blob: 3223c61c05762d5681c106a0c5bd86ac463e00e8 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
|
### Please describe the problem.
Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available.
### What steps will reproduce the problem?
See a script below...
``` sh
#!/bin/sh
TMP=$(mktemp -d)
##########################################
# Set up the repos
##########################################
###
# Create working directories
cd ${TMP}
mkdir source transfer
###
# Set up the source repo
cd ${TMP}/source
git init
git annex init source
git-annex group here source
git annex wanted here standard
###
# Set up the transfer repo
# N.B. It can see the source repo
cd ${TMP}/transfer
git init
git remote add source ${TMP}/source
git annex init transfer
git-annex group here transfer
git annex wanted here \
'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)'
##########################################
# Add some files and run the assistant
##########################################
###
# Make sure it works by adding some files to the source...
cd ${TMP}/source
date > file.1
date > file.2
date > file.3
# Run the assistant in the source...
(cd ${TMP}/source && \
git annex assistant && \
sleep 30 && \
git annex assistant --stop && \
git annex whereis file.*)
# PASS: All files are in source.
# Now run it in the transfer group
(cd ${TMP}/transfer && \
git annex assistant && \
sleep 30 && \
git annex assistant --stop && \
git annex whereis file.*)
# PASS: All files are in transfer
# Check the source
(cd ${TMP}/source && \
git annex whereis file.*)
# FAIL: Only file.1 was updated to be in source.
# whereis file.1 (1 copy)
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
# ok
# whereis file.2 (0 copies) failed
# whereis file.3 (0 copies) failed
# git-annex: whereis: 2 failed
##########################################
# Start to scratch head and ask 'Why?'
##########################################
##########
# What's happening???
cd ${TMP}/source && git annex sync
# Source thinks it is updated - working tree clean
cd ${TMP}/transfer && git annex sync
# Transfer pulls from source, so run again
git annex sync
# OK... working tree clean
# Check again
cd ${TMP}/source && git annex whereis file.*
# whereis file.1 (1 copy)
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
# ok
# whereis file.2 (0 copies) failed
# whereis file.3 (0 copies) failed
# git-annex: whereis: 2 failed
cd ${TMP}/transfer && git annex whereis file.*
# whereis file.1 (1 copy)
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here]
# ok
# whereis file.2 (0 copies) failed
# whereis file.3 (0 copies) failed
# git-annex: whereis: 2 failed
# HOWEVER... Files are definitely in the transfer repo
cd ${TMP}/source && cat file.*
# cat: file.1: No such file or directory
# cat: file.2: No such file or directory
# cat: file.3: No such file or directory
cd ${TMP}/transfer && cat file.*
# Sun May 14 21:22:02 PDT 2017
# Sun May 14 21:22:03 PDT 2017
# Sun May 14 21:22:06 PDT 2017
```
### What version of git-annex are you using? On what operating system?
#### Version:
```
git-annex version: 6.20170510
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi
dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
operating system: darwin x86_64
```
#### OS:
MacOS 10.12.4, but seems to apply to Debian Jessie too.
### Please provide any additional information below.
Annex assistant daemon.log for `source` repo:
``` sh
[2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510
[2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data.
(scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan
(started...)
[2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3
add file.1 ok
add file.2 ok
add file.3 ok
[2017-05-15 14:11:45.544277] Committer: Committing changes to git
(recording state in git...)
(recording state in git...)
```
Annex assistant daemon.log for `transfer` repo:
``` sh
[2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510
[2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data.
[2017-05-15 14:12:09.177452] TransferScanner: Syncing with source
(scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan
(started...)
warning: no common commits
From /Users/olaf/tmp/git-annex/play/bug-play/source
* [new branch] git-annex -> source/git-annex
* [new branch] master -> source/master
* [new branch] synced/master -> source/synced/master
(merging source/git-annex into git-annex...)
(recording state in git...)
Already up-to-date.
[2017-05-15 14:12:10.256195] Committer: Committing changes to git
(recording state in git...)
(checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1
[2017-05-15 14:12:11.17664] Pusher: Syncing with source
(recording state in git...)
[2017-05-15 14:12:11.356447] Committer: Committing changes to git
(recording state in git...)
To /Users/olaf/tmp/git-annex/play/bug-play/source
* [new branch] git-annex -> synced/git-annex
(checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2
(checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3
drop source file.1 ok
drop source file.2 ok
drop source file.3 ok
[2017-05-15 14:12:13.477636] Pusher: Syncing with source
(recording state in git...)
To /Users/olaf/tmp/git-annex/play/bug-play/source
825d3dd..60c96cb git-annex -> synced/git-annex
```
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes. Love it. Donated. Have been using it for years. Recommend it and get(/force) my collaborators to use it. ;-)
|
