diff options
Diffstat (limited to 'doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows')
8 files changed, 110 insertions, 0 deletions
diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_1_ce2355485f2610b6a7a79914dcd365be._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_1_ce2355485f2610b6a7a79914dcd365be._comment new file mode 100644 index 000000000..127d0603f --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_1_ce2355485f2610b6a7a79914dcd365be._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="edward" + subject="another example" + date="2014-12-06T21:01:03Z" + content=""" +I'm having the same problem: + +> git-annex: c:\Users\TV\annex\.git\annex\objects\566\a33\URL--quvi&chttps&c%%www.youtube.com%watch,63v,61XS-kKX9wQk0,38index,615,38list,61PLQ-uHSnFig5NCQkhJfkn8ogXFwzrP4SIf\: openTempFile: does not exist (No such file or directory) +> failed +> git-annex: init: 1 failed + +In my case the filename is slightly shorter, 154 characters, for Aaron the offending filename was 162 characters. + +I think the full filename that git annex is trying to write is 270 characters: + +> c:\Users\TV\annex\.git\annex\objects\566\a33\URL--quvi&chttps&c%%www.youtube.com%watch,63v,61XS-kKX9wQk0,38index,615,38list,61PLQ-uHSnFig5NCQkhJfkn8ogXFwzrP4SIf/URL--quvi&chttps&c%%www.youtube.com%watch,63v,61XS-kKX9wQk0,38index,615,38list,61PLQ-uHSnFig5NCQkhJfkn8ogXFwzrP4SIf +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment new file mode 100644 index 000000000..89af53b41 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment @@ -0,0 +1,25 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + subject="comment 2" + date="2015-01-06T19:18:26Z" + content=""" +On Linux and OSX, there is a maximum filename size, typically 255 bytes. git-annex always ensures that keys it generates are a maximum of 255 bytes long, no matter the platform. But, in dir/subdir/file, each of the 3 segments of the path is allowed to be that long. The limit on the total path size on Linux is a more reasonable 4096 bytes; OSX has only 1024 bytes. + +I don't know what to do about Windows having such an absurdly small `MAX_PATH` compared to more modern systems. + +The length of just a SHA512 checksum is 128 bytes; that means SHA512 backend cannot be used on windows, at all, since the paths git-annex generates will be at least twice that long, and will easily overflow `PATH_MAX`. I've confirmed this; just adding a file with --backend=SHA512 fails with a \"No such file or directory\" error when it tries to use the path. + +A SHA256 is a more manageable 64 bytes long. So a typical path to such an object will end with eg \".git\annex\objects\566\a33\SHA256E--d728a4c4727febe1c28509482ae1b7b2215798218e544eed7cb7b4dc988f838b\SHA256E--d728a4c4727febe1c28509482ae1b7b2215798218e544eed7cb7b4dc988f838b\" -- 174 bytes long (or a bit longer when there are also extension and size in the key) and leaving only 86 bytes or so for `c:\path\to\repo`. + +Perhaps git-annex should reduce its maximum key size from 255 to 64 bytes, the same as SHA256. Then url keys would work on Windows, except for in deep paths, where git-annex cannot work at all. This would be an easy change. + +git-annex could also avoid using absolute paths, which it currently uses extensively for simplicity (and possiibly robustness against renames of repositories and changes of working directory?), and use relative paths instead. This would probably solve the two examples given in the bug report, and it would make git-annex work better when in a deep path in Windows. It would not make SHA512 work though; with keys that long, the relative path is still too long. (And, it's still possible to get a relative path that has so many '../../' and subdirectories etc that it overflows `PATH_MAX`. It would probably take a really crazy repository directory structure though.) + +The MSDN article has one very interesting bit: + +> The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the \"\\?\\" prefix. For example, \"\\?\D:\very long path\". + +(It seems that, when using that prefix, `/` is not converted to `\` .. I think git-annex is quite good about getting the slashes the right way round these days.) + +So it might be possible for git-annex to use that prefix and avoid this issue entirely. Haskell's FilePath library does understand that prefix (treats it as part of the drive). Since git-annex always uses the path to the top of the Repo when constructing the problimatic FilePaths, I might be able to just change the Repo constructor to add that prefix, and everything follow from that. I tried doing that, unfortunately this makes *git* fail, with \"fatal: relative path syntax cannot be used outside working tree\" when operating on such a repo. Cause git doesn't understand that prefix. +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment new file mode 100644 index 000000000..153c48db2 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + subject="comment 3" + date="2015-01-06T20:51:20Z" + content=""" +I've started a `relativepaths` branch that uses all relative paths to the git repo. After working on it for several hours, there are still 16 test suite failures (update: 10) (update: 1). The potential for uncaught breakage is much higher than I am happy with. (Amoung other problems, git-annex does call setCurrentDirectory in several places, and this utterly breaks the relative paths). + +Using that branch on windows, I am still unable to add files with --backend=SHA512; even relative paths don't make it short enough for such keys. +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment new file mode 100644 index 000000000..3aa56f004 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + subject="comment 4" + date="2015-01-06T21:59:15Z" + content=""" +Even with relative paths, Edward's example would use a path of 253 characters, and so a slightly longer url would still break it, even with relative paths. + +So, I think reducing url key length needs to be done anyway, and I've done that. Which hardly closes this bug. +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_5_a1c8ac1d7884d676f05db588b2894603._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_5_a1c8ac1d7884d676f05db588b2894603._comment new file mode 100644 index 000000000..54c29af3b --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_5_a1c8ac1d7884d676f05db588b2894603._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + subject="comment 5" + date="2015-01-07T02:35:46Z" + content=""" +I've beat on the relativepaths branch some more and am probably as confident about it as I'm going to get. Will have to merge it and see what else it breaks. + +Also, I've documented that SHA512 and other large hashes are not recommended if one wants to interop with Windows. + +None of which completely fixes this bug, but short of teaching git about the magic filename prefix to make windows not be so broken, I don't see anything more I can do. +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_6_739b4fd2156de7570ec71417f41eb188._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_6_739b4fd2156de7570ec71417f41eb188._comment new file mode 100644 index 000000000..b20fe2ab9 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_6_739b4fd2156de7570ec71417f41eb188._comment @@ -0,0 +1,23 @@ +[[!comment format=sh + username="https://www.google.com/accounts/o8/id?id=AItOawkDqgw0JLrxLH3GIpg36Mp79F_1pxZxWxU" + nickname="Benjamin" + subject="comment 6" + date="2015-02-04T14:56:36Z" + content=""" +Hi, + +I experienced the same problem. In my case the path to the local clone is \"C:\Users\user\Documents\dev\testplay\studyforrest\". Trying to get a file within that repo (\"c:\Users\user\Documents\dev\testplay\studyforrest\stimulus\task001\annotations>git annex get german_audio_description.csv\") results in \"couldn't find path\". +More precise git annex says: \"git-annex: MoveFileEx \"..\\..\\..\\.git\\annex\\tmp\\SHA256E-s49358--49140697bfd54e0d384b30efb7256c246b99f8c2cd63a48d54078e7d03e26286.csv\" \"..\\..\\..\\.git\\annex\\objects\\885\\a97\\SHA256E-s49358-- +49140697bfd54e0d384b30efb7256c246b99f8c2cd63a48d54078e7d03e26286.csv\\SHA256E-s49358--49140697bfd54e0d384b30efb7256c246b99f8c2cd63a48d54078e7d03e26286.csv\": does not exist (Das System kann den angegeb +enen Pfad nicht finden.) +failed +git-annex: get: 1 failed\" + + +Cloning to c:\studyforrest works. + +Now, I wonder why, since none of the mentioned paths exceeds the limit of 260. But as git annex mentioned it seems to use that relative path \"..\\..\\..\\.git\\annex\ [...]\". May be it internally composes it to something like +\"c:\Users\user\Documents\dev\testplay\studyforrest\stimulus\task001\annotations\..\\..\\..\\.git\\annex\ [...]\" where the part \"stimulus\task001\annotations\..\..\..\\" is actually not needed to adress the desired file. So, if that is the case may be you could give some more room by eliminating this? + + +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_7_c561d2eb75a2579db620bf7877c98502._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_7_c561d2eb75a2579db620bf7877c98502._comment new file mode 100644 index 000000000..6243b9129 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_7_c561d2eb75a2579db620bf7877c98502._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnx8kHW66N3BqmkVpgtXDlYMvr8TJ5VvfY" + nickname="Yaroslav" + subject="comment 7" + date="2015-02-04T15:53:48Z" + content=""" +It seems that is where where having no read-only key/ directory would also be of help to shorten filename paths. (as discussed in https://github.com/datalad/datalad/issues/32) +"""]] diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_8_169423dd4f3292e503b285f088f6701f._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_8_169423dd4f3292e503b285f088f6701f._comment new file mode 100644 index 000000000..c88e3b1b5 --- /dev/null +++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_8_169423dd4f3292e503b285f088f6701f._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkDqgw0JLrxLH3GIpg36Mp79F_1pxZxWxU" + nickname="Benjamin" + subject="comment 8" + date="2015-02-05T13:05:31Z" + content=""" +Despite all things, that may or may not be possible to do to shorten the path lengths used by git-annex, it would be very helpful to properly detect that this problem is occuring and give a reasonable error message. Since windows' \"couldn't find path\"-message doesn't tell you what's going on. +"""]] |