diff options
Diffstat (limited to 'doc/bugs/Whereis_reports_same_UUID_multiple_times')
6 files changed, 0 insertions, 132 deletions
diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_1_4af19e393212aa5cd5b64c3bb81e8268._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_1_4af19e393212aa5cd5b64c3bb81e8268._comment deleted file mode 100644 index 61133235b..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_1_4af19e393212aa5cd5b64c3bb81e8268._comment +++ /dev/null @@ -1,16 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-05-17T17:51:11Z" - content=""" -This is quite odd. Normally, multiple instances of the same UUID can occur -but will get compacted down to 1. - -That it only lists the remote name/description for one instance of the UUID -is probably a clue to whatever the problem is. - -Can you paste the output of this command: `git show git-annex:uuid.log` - -(It would be very helpful if I could get access to a clone of the git-annex -branch of your repository.) -"""]] diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_2_148e8a83da7f9208ab7e0619b70b7093._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_2_148e8a83da7f9208ab7e0619b70b7093._comment deleted file mode 100644 index 2c9b91791..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_2_148e8a83da7f9208ab7e0619b70b7093._comment +++ /dev/null @@ -1,43 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 2""" - date="2016-05-27T14:26:18Z" - content=""" -Received a clone of this repository (in git-annex-test-repos/annex.bundle -here), and was able to reproduce the bug. - -Looking at one duplicate UUID for one file, I see: - - 1444995510.830128s 1 0866153a-19e5-4382-aeb6-30e8210706cc - 1444995510.830128s 1 0866153a-19e5-4382-aeb6-30e8210706cc - 1444995510.830128s 1 0866153a-19e5-4382-aeb6-30e8210706cc - 1444995510.830128s 1 0866153a-19e5-4382-aeb6-30e8210706cc - 1444995510.830128s 1 0866153a-19e5-4382-aeb6-30e8210706cc - -The notable thing here is not that there are multiple lines for a UUID, but -that they somehow have the *exact* same timestamp down to the -microsecond. - -I'm a) unsure how this could happen and b) afraid that the log file -compaction fails in this case, with catastrophic results. - -Regarding how this could happen, git blame shows a single commit -adding duplicate lines with the same timestamp. Commit message was -"update". The commit touched a wide swath of the repository, including even -non-location-log files like trust.log, which also got duplicate lines with -the same timestamp. - -Some of the lines were entirely new, but some existing lines also -got duplicated. - -There were some duplicate lines before this commit, so it was not an -isolated incident. - -Clearly, log compaction needs to collapse down lines that are identical except -for timestamp. The location log code also needs to throw out all but one -current item for a given uuid, since other code treats each returned location -as a copy, expecting there to not be any duplicate UUIDs. With these changes, -whatever caused these duplicate lines to occur in the first place at least -won't result in weird output or data loss. I have not verified yet if data -loss can actually occur in this case. -"""]] diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_3_96eeccef3cf2678b30b892cb264bc7e1._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_3_96eeccef3cf2678b30b892cb264bc7e1._comment deleted file mode 100644 index 7c4b48398..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_3_96eeccef3cf2678b30b892cb264bc7e1._comment +++ /dev/null @@ -1,30 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 3""" - date="2016-05-27T14:47:40Z" - content=""" -I tried reproducing this artificially by duplicating a presence log line. -That didn't work; whereis showed only 1 copy, not multiple ones. -Which makes sense, because the log reader makes a map that has the UUID as -the key. So a single UUID can't appear more than once at that point. Good! - -So, there's something else going on in the problem repository that -allows this behavior to occur. - -Aha! It's sequences of one or more `\r` at the end of the line. - - fromList [("0866153a-19e5-4382-aeb6-30e8210706cc",LogLine {date = 1444995329.589s, status = InfoPresent, info = "0866153a-19e5-4382-aeb6-30e8210706cc"}),("0866153a-19e5-4382-aeb6-30e8210706cc\r",LogLine {date = 1444995329.589s, status = InfoPresent, info = "0866153a-19e5-4382-aeb6-30e8210706cc\r"}),("0866153a-19e5-4382-aeb6-30e8210706cc\r\r",LogLine {date = 1444995329.589s, status = InfoPresent, info = "0866153a-19e5-4382-aeb6-30e8210706cc\r\r"}) ... - -So, 0866153a-19e5-4382-aeb6-30e8210706cc seems to appear multiple times, but it's really due to these `\r`'s. - -Suspect this means that this problem only impacts display. It should not lead -to data loss, because no remote will have a UUID ending in `\r', so there -should be no way for git-annex to somehow count a remote twice as containing -a copy of a file when dropping. Indeed, we can see in the whereis output that -it only matches up some instances of the "duplicate" uuid with remotes -- because -the other instances have carriage returns appended. - -Also, this suggests that the reason the duplicate lines occurred in the first -place was something to do with a windows system, which presumably added some -`\r\n` that is being stumbled over. -"""]] diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_4_d26052bc19fbb6eaa84b2d72c280f11a._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_4_d26052bc19fbb6eaa84b2d72c280f11a._comment deleted file mode 100644 index 92f711ed6..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_4_d26052bc19fbb6eaa84b2d72c280f11a._comment +++ /dev/null @@ -1,18 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 4""" - date="2016-05-27T15:02:05Z" - content=""" -In git-annex version 5.20140606, we have: - - * Windows: Fix bug introduced in last release that caused files - in the git-annex branch to have lines teminated with \r. - -There was only 1 week where git-annex had that bug AFAIK, but of -course the broken version could have kept on being used for some time. - -It's also always possible that this same problem popped up again in the -code. The fix in 1ab3d7c81049e4ce7e8e47800e2ef1fecb3a9ab4 was to -Annex.Journal and that still seems ok. The merge code is another place -this could perhaps happen. -"""]] diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_5_2355fd90d4ff8dd440bf11348a8daa73._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_5_2355fd90d4ff8dd440bf11348a8daa73._comment deleted file mode 100644 index 5ae6c2b1f..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_5_2355fd90d4ff8dd440bf11348a8daa73._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 5""" - date="2016-05-27T15:40:16Z" - content=""" -I've developed a patch which yields a good whereis display in this repo. - -Still remains to be seen if there's some code path that currently causes -'\r' to get added in the current version of git-annex on Windows. -"""]] diff --git a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_6_d3adcfad215b3f39cc5bc3ff248e4b86._comment b/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_6_d3adcfad215b3f39cc5bc3ff248e4b86._comment deleted file mode 100644 index 30951378e..000000000 --- a/doc/bugs/Whereis_reports_same_UUID_multiple_times/comment_6_d3adcfad215b3f39cc5bc3ff248e4b86._comment +++ /dev/null @@ -1,15 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2016-05-27T19:03:05Z" - content=""" -Union merge on windows does indeed add \r onto lines. - -Looks like hashBlob is at fault; it writes a string to a temp file, -and the IO layer does CRLF conversion at that point. - -The git-annex branch transition code also uses hashBlob so would also -do it. - -So I've reproduced the root cause of this now. Fixing.. -"""]] |