1 files changed, 215 insertions, 0 deletions

diff --git a/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn
new file mode 100644
index 000000000..3223c61c0
--- /dev/null
+++ b/doc/bugs/sync_claims_data_loss_but_seems_to_just_lose_tracking.mdwn
@@ -0,0 +1,215 @@
+### Please describe the problem.
+Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available.
+
+
+### What steps will reproduce the problem?
+See a script below...
+
+``` sh
+#!/bin/sh
+
+TMP=$(mktemp -d)
+
+
+##########################################
+# Set up the repos
+##########################################
+
+
+###
+# Create working directories
+cd ${TMP}
+mkdir source transfer
+
+
+###
+# Set up the source repo
+cd ${TMP}/source
+git init
+git annex init source
+git-annex group here source
+git annex wanted here standard
+
+
+###
+# Set up the transfer repo
+# N.B. It can see the source repo
+cd ${TMP}/transfer
+git init
+git remote add source ${TMP}/source
+git annex init transfer
+git-annex group here transfer
+git annex wanted here \
+  'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)'
+
+
+##########################################
+# Add some files and run the assistant
+##########################################
+
+###
+# Make sure it works by adding some files to the source...
+cd ${TMP}/source
+date > file.1
+date > file.2
+date > file.3
+
+# Run the assistant in the source...
+(cd ${TMP}/source && \
+    git annex assistant && \
+    sleep 30 && \
+    git annex assistant --stop && \
+    git annex whereis file.*)
+# PASS: All files are in source.
+
+# Now run it in the transfer group
+(cd ${TMP}/transfer && \
+    git annex assistant && \
+    sleep 30 && \
+    git annex assistant --stop && \
+    git annex whereis file.*)
+# PASS: All files are in transfer
+
+# Check the source
+(cd ${TMP}/source && \
+    git annex whereis file.*)
+# FAIL: Only file.1 was updated to be in source.
+
+# whereis file.1 (1 copy)
+#        64b193e0-86f0-4349-b073-70af919ce628 -- transfer
+# ok
+# whereis file.2 (0 copies) failed
+# whereis file.3 (0 copies) failed
+# git-annex: whereis: 2 failed
+
+
+##########################################
+# Start to scratch head and ask 'Why?'
+##########################################
+
+##########
+# What's happening???
+
+cd ${TMP}/source && git annex sync
+# Source thinks it is updated - working tree clean
+
+cd ${TMP}/transfer && git annex sync
+# Transfer pulls from source, so run again
+git annex sync
+# OK... working tree clean
+
+
+# Check again
+cd ${TMP}/source && git annex whereis file.*
+# whereis file.1 (1 copy)
+#         64b193e0-86f0-4349-b073-70af919ce628 -- transfer
+# ok
+# whereis file.2 (0 copies) failed
+# whereis file.3 (0 copies) failed
+# git-annex: whereis: 2 failed
+
+cd ${TMP}/transfer && git annex whereis file.*
+# whereis file.1 (1 copy)
+#         64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here]
+# ok
+# whereis file.2 (0 copies) failed
+# whereis file.3 (0 copies) failed
+# git-annex: whereis: 2 failed
+
+
+# HOWEVER...  Files are definitely in the transfer repo
+
+cd ${TMP}/source && cat file.*
+# cat: file.1: No such file or directory
+# cat: file.2: No such file or directory
+# cat: file.3: No such file or directory
+
+
+cd ${TMP}/transfer && cat file.*
+# Sun May 14 21:22:02 PDT 2017
+# Sun May 14 21:22:03 PDT 2017
+# Sun May 14 21:22:06 PDT 2017
+```
+
+
+
+### What version of git-annex are you using? On what operating system?
+
+#### Version:
+```
+git-annex version: 6.20170510
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 5
+supported repository versions: 3 5 6
+upgrade supported from repository versions: 0 1 2 3 4 5
+operating system: darwin x86_64
+```
+
+#### OS:
+MacOS 10.12.4, but seems to apply to Debian Jessie too.
+
+
+### Please provide any additional information below.
+
+Annex assistant daemon.log for `source` repo:
+
+``` sh
+[2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510
+[2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data.
+(scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan
+(started...)
+[2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3
+add file.1 ok
+add file.2 ok
+add file.3 ok
+[2017-05-15 14:11:45.544277] Committer: Committing changes to git
+(recording state in git...)
+(recording state in git...)
+```
+
+Annex assistant daemon.log for `transfer` repo:
+
+``` sh
+[2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510
+[2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data.
+[2017-05-15 14:12:09.177452] TransferScanner: Syncing with source
+(scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan
+(started...)
+warning: no common commits
+From /Users/olaf/tmp/git-annex/play/bug-play/source
+ * [new branch]      git-annex     -> source/git-annex
+ * [new branch]      master        -> source/master
+ * [new branch]      synced/master -> source/synced/master
+(merging source/git-annex into git-annex...)
+(recording state in git...)
+
+
+Already up-to-date.
+[2017-05-15 14:12:10.256195] Committer: Committing changes to git
+(recording state in git...)
+(checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1
+[2017-05-15 14:12:11.17664] Pusher: Syncing with source
+(recording state in git...)
+[2017-05-15 14:12:11.356447] Committer: Committing changes to git
+(recording state in git...)
+To /Users/olaf/tmp/git-annex/play/bug-play/source
+ * [new branch]      git-annex -> synced/git-annex
+(checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2
+(checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3
+drop source file.1 ok
+drop source file.2 ok
+drop source file.3 ok
+[2017-05-15 14:12:13.477636] Pusher: Syncing with source
+(recording state in git...)
+To /Users/olaf/tmp/git-annex/play/bug-play/source
+   825d3dd..60c96cb  git-annex -> synced/git-annex
+```
+
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes.  Love it.  Donated.  Have been using it for years.  Recommend it and get(/force) my collaborators to use it.  ;-)
+