Merge branch 'master' of ssh://git-annex.branchable.com

6 files changed, 79 insertions, 0 deletions

diff --git a/doc/bugs/CHECKPRESENT_could_check_file_size_as_well/comment_1_7bbba896da745a8a6b69ec62bf71fc89._comment b/doc/bugs/CHECKPRESENT_could_check_file_size_as_well/comment_1_7bbba896da745a8a6b69ec62bf71fc89._comment
new file mode 100644
index 000000000..e917ec998
--- /dev/null
+++ b/doc/bugs/CHECKPRESENT_could_check_file_size_as_well/comment_1_7bbba896da745a8a6b69ec62bf71fc89._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 1"
+ date="2015-01-06T17:41:45Z"
+ content="""
+The encrypted content size is not constant, and not known to git-annex.
+
+The only git-annex remote that checks the size in its checkpresent implementation is the web special remote, precisely because it's never encrypted. Also because files on the web change content from time to time and so that needs to be detected.
+
+What would make sense is to extend the reply to `CHECKPRESENT-SUCCESS Key [size]` or perhaps `CHECKPRESENT-SIZE Key size`. git-annex can then compare the value with the key's known size, if any. If the key is encrypted, it would need to skip this check.
+
+Note that chunk keys currently have their keySize inherited from the parent key, and the keyChunkSize of each chunk key is set to the key size. The last chunk of a key will typically be shorter than its keyChunkSize. That would need to be cleaned up.
+"""]]
diff --git a/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment
new file mode 100644
index 000000000..89af53b41
--- /dev/null
+++ b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_2_dfc398002e2ffbe0b63ce422a1e16d67._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 2"
+ date="2015-01-06T19:18:26Z"
+ content="""
+On Linux and OSX, there is a maximum filename size, typically 255 bytes. git-annex always ensures that keys it generates are a maximum of 255 bytes long, no matter the platform. But, in dir/subdir/file, each of the 3 segments of the path is allowed to be that long. The limit on the total path size on Linux is a more reasonable 4096 bytes; OSX has only 1024 bytes. 
+
+I don't know what to do about Windows having such an absurdly small `MAX_PATH` compared to more modern systems.
+
+The length of just a SHA512 checksum is 128 bytes; that means SHA512 backend cannot be used on windows, at all, since the paths git-annex generates will be at least twice that long, and will easily overflow `PATH_MAX`. I've confirmed this; just adding a file with --backend=SHA512 fails with a \"No such file or directory\" error when it tries to use the path.
+
+A SHA256 is a more manageable 64 bytes long. So a typical path to such an object will end with eg \".git\annex\objects\566\a33\SHA256E--d728a4c4727febe1c28509482ae1b7b2215798218e544eed7cb7b4dc988f838b\SHA256E--d728a4c4727febe1c28509482ae1b7b2215798218e544eed7cb7b4dc988f838b\" -- 174 bytes long (or a bit longer when there are also extension and size in the key) and leaving only 86 bytes or so for `c:\path\to\repo`.
+
+Perhaps git-annex should reduce its maximum key size from 255 to 64 bytes, the same as SHA256. Then url keys would work on Windows, except for in deep paths, where git-annex cannot work at all. This would be an easy change.
+
+git-annex could also avoid using absolute paths, which it currently uses extensively for simplicity (and possiibly robustness against renames of repositories and changes of working directory?), and use relative paths instead. This would probably solve the two examples given in the bug report, and it would make git-annex work better when in a deep path in Windows. It would not make SHA512 work though; with keys that long, the relative path is still too long. (And, it's still possible to get a relative path that has so many '../../' and subdirectories etc that it overflows `PATH_MAX`. It would probably take a really crazy repository directory structure though.)
+
+The MSDN article has one very interesting bit:
+
+> The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the \"\\?\\" prefix. For example, \"\\?\D:\very long path\".
+
+(It seems that, when using that prefix, `/` is not converted to `\` .. I think git-annex is quite good about getting the slashes the right way round these days.)
+
+So it might be possible for git-annex to use that prefix and avoid this issue entirely. Haskell's FilePath library does understand that prefix (treats it as part of the drive). Since git-annex always uses the path to the top of the Repo when constructing the problimatic FilePaths, I might be able to just change the Repo constructor to add that prefix, and everything follow from that. I tried doing that, unfortunately this makes *git* fail, with \"fatal: relative path syntax cannot be used outside working tree\" when operating on such a repo. Cause git doesn't understand that prefix.
+"""]]
diff --git a/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment
new file mode 100644
index 000000000..153c48db2
--- /dev/null
+++ b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_3_1d23e9760782a8d6d2ea2dd5a4c6253a._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 3"
+ date="2015-01-06T20:51:20Z"
+ content="""
+I've started a `relativepaths` branch that uses all relative paths to the git repo. After working on it for several hours, there are still 16 test suite failures (update: 10) (update: 1). The potential for uncaught breakage is much higher than I am happy with. (Amoung other problems, git-annex does call setCurrentDirectory in several places, and this utterly breaks the relative paths).
+
+Using that branch on windows, I am still unable to add files with --backend=SHA512; even relative paths don't make it short enough for such keys.
+"""]]
diff --git a/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment
new file mode 100644
index 000000000..3aa56f004
--- /dev/null
+++ b/doc/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/comment_4_108f3e4449fc9591bcdeb490b486357f._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 4"
+ date="2015-01-06T21:59:15Z"
+ content="""
+Even with relative paths, Edward's example would use a path of 253 characters, and so a slightly longer url would still break it, even with relative paths.
+
+So, I think reducing url key length needs to be done anyway, and I've done that. Which hardly closes this bug.
+"""]]
diff --git a/doc/bugs/__91__Android__93___5.0_needs_PIE_executables___40__git_annex_does_not_work_on_android_5.0__41__/comment_7_c9104ad6d9cfec93f561b9cd514b6f48._comment b/doc/bugs/__91__Android__93___5.0_needs_PIE_executables___40__git_annex_does_not_work_on_android_5.0__41__/comment_7_c9104ad6d9cfec93f561b9cd514b6f48._comment
new file mode 100644
index 000000000..3d4f7e230
--- /dev/null
+++ b/doc/bugs/__91__Android__93___5.0_needs_PIE_executables___40__git_annex_does_not_work_on_android_5.0__41__/comment_7_c9104ad6d9cfec93f561b9cd514b6f48._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 7"
+ date="2015-01-06T17:43:45Z"
+ content="""
+I verified with `eu-readelf --file-header` that the git-annex binary is DYN; ie linked PIE.
+
+It might be that I also need to tell the C compiler to build it with PIE options. I have now updated the build to include that. Please test the new build.
+
+It occurs to me that the problem might be not git-annex, but one of the other binaries, like busybox. Does the android app install to the point that there is a working terminal app with a shell?
+
+It also seems possible that the entire haskell library stack might need to be built with PIE options. If so, that will be a massive pain; I'd need an entire separate autobuilder instance.
+"""]]
diff --git a/doc/forum/How_To_Permanently_Delete_a_File__63__/comment_7_1a6fab467a1c4db2dee39ef464f6cc7b._comment b/doc/forum/How_To_Permanently_Delete_a_File__63__/comment_7_1a6fab467a1c4db2dee39ef464f6cc7b._comment
new file mode 100644
index 000000000..0b1f6812b
--- /dev/null
+++ b/doc/forum/How_To_Permanently_Delete_a_File__63__/comment_7_1a6fab467a1c4db2dee39ef464f6cc7b._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmwjQzWgiD7_I3zw-_91rMRf_6qoThupis"
+ nickname="Mike"
+ subject="comment 7"
+ date="2015-01-06T18:01:40Z"
+ content="""
+Unfortunately, that is not useful for this at all. We are talking about millions of files here, and the issue is leaving behind old hard links, so that program just won't work at all.
+
+Furthermore, this questions has all ready been answered in previous comments.
+"""]]