### Please describe the problem. Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available. ### What steps will reproduce the problem? See a script below... ``` sh #!/bin/sh TMP=$(mktemp -d) ########################################## # Set up the repos ########################################## ### # Create working directories cd ${TMP} mkdir source transfer ### # Set up the source repo cd ${TMP}/source git init git annex init source git-annex group here source git annex wanted here standard ### # Set up the transfer repo # N.B. It can see the source repo cd ${TMP}/transfer git init git remote add source ${TMP}/source git annex init transfer git-annex group here transfer git annex wanted here \ 'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)' ########################################## # Add some files and run the assistant ########################################## ### # Make sure it works by adding some files to the source... cd ${TMP}/source date > file.1 date > file.2 date > file.3 # Run the assistant in the source... (cd ${TMP}/source && \ git annex assistant && \ sleep 30 && \ git annex assistant --stop && \ git annex whereis file.*) # PASS: All files are in source. # Now run it in the transfer group (cd ${TMP}/transfer && \ git annex assistant && \ sleep 30 && \ git annex assistant --stop && \ git annex whereis file.*) # PASS: All files are in transfer # Check the source (cd ${TMP}/source && \ git annex whereis file.*) # FAIL: Only file.1 was updated to be in source. # whereis file.1 (1 copy) # 64b193e0-86f0-4349-b073-70af919ce628 -- transfer # ok # whereis file.2 (0 copies) failed # whereis file.3 (0 copies) failed # git-annex: whereis: 2 failed ########################################## # Start to scratch head and ask 'Why?' ########################################## ########## # What's happening??? cd ${TMP}/source && git annex sync # Source thinks it is updated - working tree clean cd ${TMP}/transfer && git annex sync # Transfer pulls from source, so run again git annex sync # OK... working tree clean # Check again cd ${TMP}/source && git annex whereis file.* # whereis file.1 (1 copy) # 64b193e0-86f0-4349-b073-70af919ce628 -- transfer # ok # whereis file.2 (0 copies) failed # whereis file.3 (0 copies) failed # git-annex: whereis: 2 failed cd ${TMP}/transfer && git annex whereis file.* # whereis file.1 (1 copy) # 64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here] # ok # whereis file.2 (0 copies) failed # whereis file.3 (0 copies) failed # git-annex: whereis: 2 failed # HOWEVER... Files are definitely in the transfer repo cd ${TMP}/source && cat file.* # cat: file.1: No such file or directory # cat: file.2: No such file or directory # cat: file.3: No such file or directory cd ${TMP}/transfer && cat file.* # Sun May 14 21:22:02 PDT 2017 # Sun May 14 21:22:03 PDT 2017 # Sun May 14 21:22:06 PDT 2017 ``` ### What version of git-annex are you using? On what operating system? #### Version: ``` git-annex version: 6.20170510 build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external local repository version: 5 supported repository versions: 3 5 6 upgrade supported from repository versions: 0 1 2 3 4 5 operating system: darwin x86_64 ``` #### OS: MacOS 10.12.4, but seems to apply to Debian Jessie too. ### Please provide any additional information below. Annex assistant daemon.log for `source` repo: ``` sh [2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510 [2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data. (scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan (started...) [2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3 add file.1 ok add file.2 ok add file.3 ok [2017-05-15 14:11:45.544277] Committer: Committing changes to git (recording state in git...) (recording state in git...) ``` Annex assistant daemon.log for `transfer` repo: ``` sh [2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510 [2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data. [2017-05-15 14:12:09.177452] TransferScanner: Syncing with source (scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan (started...) warning: no common commits From /Users/olaf/tmp/git-annex/play/bug-play/source * [new branch] git-annex -> source/git-annex * [new branch] master -> source/master * [new branch] synced/master -> source/synced/master (merging source/git-annex into git-annex...) (recording state in git...) Already up-to-date. [2017-05-15 14:12:10.256195] Committer: Committing changes to git (recording state in git...) (checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1 [2017-05-15 14:12:11.17664] Pusher: Syncing with source (recording state in git...) [2017-05-15 14:12:11.356447] Committer: Committing changes to git (recording state in git...) To /Users/olaf/tmp/git-annex/play/bug-play/source * [new branch] git-annex -> synced/git-annex (checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2 (checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3 drop source file.1 ok drop source file.2 ok drop source file.3 ok [2017-05-15 14:12:13.477636] Pusher: Syncing with source (recording state in git...) To /Users/olaf/tmp/git-annex/play/bug-play/source 825d3dd..60c96cb git-annex -> synced/git-annex ``` ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) Yes. Love it. Donated. Have been using it for years. Recommend it and get(/force) my collaborators to use it. ;-)