aboutsummaryrefslogtreecommitdiff
path: root/doc/bugs/Prefered_Content_not_Taken_into_Account.mdwn
blob: bdb3c9d97420a90a05e7a5e5eadc94f140de4565 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
### Please describe the problem.

This is a follow up to the question [1]. I've created 4 repos with the following vicfg.

[[!format sh """
# git-annex configuration
# 
# Changes saved to this file will be recorded in the git-annex branch.
# 
# Lines in this file have the format:
#   setting field = value

# Repository trust configuration
# (Valid trust levels: trusted semitrusted untrusted dead)
# (for web)
#trust 00000000-0000-0000-0000-000000000001 = semitrusted
# (for bittorrent)
#trust 00000000-0000-0000-0000-000000000002 = semitrusted
# (for repo2)
#trust 2837f3d7-7c58-4177-8877-620213cf5146 = semitrusted
# (for repo3)
#trust b3cbb656-e797-45ba-bf43-08523d463146 = semitrusted
# (for repo1)
#trust e7673de4-d465-4557-be8f-b1400acf923e = semitrusted

# Repository groups
# (Standard groups: client transfer backup incrementalbackup smallarchive archive source manual public unwanted)
# (Separate group names with spaces)
# (for repo3)
group b3cbb656-e797-45ba-bf43-08523d463146 = PodA
# (for repo1)
group e7673de4-d465-4557-be8f-b1400acf923e = PodA
# (for repo2)
group 2837f3d7-7c58-4177-8877-620213cf5146 = PodB
# (for web)
#group 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#group 00000000-0000-0000-0000-000000000002 = 

# Repository preferred contents
# (Set to "standard" to use a repository's group's preferred contents)
# (for repo2)
wanted 2837f3d7-7c58-4177-8877-620213cf5146 = groupwanted
# (for repo1)
wanted e7673de4-d465-4557-be8f-b1400acf923e = groupwanted
# (for web)
#wanted 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#wanted 00000000-0000-0000-0000-000000000002 = 
# (for repo3)
#wanted b3cbb656-e797-45ba-bf43-08523d463146 = 

# Group preferred contents
# (Used by repositories with "groupwanted" in their preferred contents)
groupwanted PodA = not copies=PodA:1
groupwanted PodB = not copies=PodB:1
#groupwanted archive = 
#groupwanted backup = 
#groupwanted client = 
#groupwanted incrementalbackup = 
#groupwanted manual = 
#groupwanted public = 
#groupwanted smallarchive = 
#groupwanted source = 
#groupwanted transfer = 
#groupwanted unwanted = 

# Standard preferred contents
# (Used by wanted or groupwanted expressions containing "standard")
# (For reference only; built-in and cannot be changed!)
# standard client = (include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1
# standard transfer = (not (inallgroup=client and copies=client:2) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)) or approxlackingcopies=1
# standard backup = anything
# standard incrementalbackup = ((not copies=backup:1) and (not copies=incrementalbackup:1)) or approxlackingcopies=1
# standard smallarchive = ((include=*/archive/* or include=archive/*) and ((not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1)) or approxlackingcopies=1
# standard archive = (not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1
# standard source = not (copies=1)
# standard manual = present and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)
# standard public = inpreferreddir
# standard unwanted = not anything

# Repository required contents
# (for web)
#required 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#required 00000000-0000-0000-0000-000000000002 = 
# (for repo2)
#required 2837f3d7-7c58-4177-8877-620213cf5146 = 
# (for repo3)
#required b3cbb656-e797-45ba-bf43-08523d463146 = 
# (for repo1)
#required e7673de4-d465-4557-be8f-b1400acf923e = 

# Scheduled activities
# (Separate multiple activities with "; ")
# (for web)
#schedule 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#schedule 00000000-0000-0000-0000-000000000002 = 
# (for repo2)
#schedule 2837f3d7-7c58-4177-8877-620213cf5146 = 
# (for repo3)
#schedule b3cbb656-e797-45ba-bf43-08523d463146 = 
# (for repo1)
#schedule e7673de4-d465-4557-be8f-b1400acf923e = 
"""]]

This works as expected each file lands in 1 on of the repos on the group, not both. My original repo one with the problem has the following vicfg.

[[!format sh """
# git-annex configuration
# 
# Changes saved to this file will be recorded in the git-annex branch.
# 
# Lines in this file have the format:
#   setting field = value

# Repository trust configuration
# (Valid trust levels: trusted semitrusted untrusted dead)
# (for buse [origin])
trust bff7238e-bd92-4929-88a8-c59c1a1dcf03 = semitrusted
# (for web)
trust 00000000-0000-0000-0000-000000000001 = untrusted
# (for bittorrent)
trust 00000000-0000-0000-0000-000000000002 = untrusted
# (for )
trust 03b19bdc-8497-4234-9ef6-c80d78e309f8 = dead
# (for )
trust 0b1a34ce-a8e3-4002-b46a-83e96626305f = dead
# (for )
trust 1459a238-6f60-4b43-8459-f7055b49e3c5 = dead
# (for )
trust 14738d0f-2f50-4f93-a8e4-c29870d02ba1 = dead
# (for )
trust 1512a830-7789-4bc8-a4a0-4c51eedd1109 = dead
# (for )
trust 174d69cb-9b97-44df-9f15-aea00f66a61c = dead
# (for )
trust 1ade7f29-2ad0-4ab0-a816-de55ffde389e = dead
# (for )
trust 1ba9d14b-deae-46e6-b451-dbf0fa8aa934 = dead
# (for )
trust 1dda84ca-4e8c-49b4-b673-35ec2b11557b = dead
# (for )
trust 29aad234-82e6-45c9-9822-b650290c2264 = dead
# (for )
trust 2be091a2-e721-4a86-a759-c5df7a41a61d = dead
# (for )
trust 3540b01a-03f0-4a58-8ea1-7ca80002a22d = dead
# (for )
trust 39b4d431-9d55-432d-a801-eaf4cf643de1 = dead
# (for )
trust 3a443e72-e8e0-11e2-b7db-8feb865236d4 = dead
# (for )
trust 42023cde-e95f-483c-8389-54b823aed789 = dead
# (for )
trust 54a819c1-65a3-44e9-86c3-0eb602223ee8 = dead
# (for )
trust 68846c04-912d-4bfc-80f3-3b8ec1e7a3c7 = dead
# (for )
trust 69b10f7e-0214-4fd0-8138-ff4717d1f39f = dead
# (for )
trust 7b236779-a961-4e72-abfc-56b764e53a9c = dead
# (for )
trust 7cde3feb-3876-4c2a-a772-d4541eac8614 = dead
# (for )
trust 8f3e7b2b-f8de-4114-bcdd-2ef3ed932c18 = dead
# (for )
trust a3bd1546-e8df-11e2-bc54-a7edbf29f5da = dead
# (for )
trust aa830256-4c63-44e3-9a3f-33b377de79d5 = dead
# (for )
trust ad46e36b-df9c-4d45-9af1-58d17e798afa = dead
# (for )
trust b5464382-b5f8-4ec3-b36d-a378b1202497 = dead
# (for )
trust b6317827-97d3-44bd-8197-91cae0b56a19 = dead
# (for )
trust bbb5dd98-f09d-11e2-a57b-d7185df9eef1 = dead
# (for )
trust c6dfb856-a0cc-4e79-acbd-3600f4bce158 = dead
# (for )
trust ccb11b54-ecc8-11e2-ad6c-af6445c8eedc = dead
# (for )
trust cdbf33bc-17bc-49e3-b0d1-afa653a2c1f7 = dead
# (for )
trust d8cf3fb5-4e8e-4732-b566-dab6f2d9dc4c = dead
# (for )
trust e160a8ef-3b3d-4f0e-88eb-9ceba11b2f57 = dead
# (for )
trust e39439a0-1486-55a0-ada3-c9785c08d650 = dead
# (for )
trust e8bcb64b-17df-4d79-a5b6-9d444e4afa51 = dead
# (for )
trust eae76b0e-ada9-4c5e-b9e3-db5998324e70 = dead
# (for )
trust ec71b4bd-1b7b-408c-906f-91c4168fd89f = dead
# (for )
trust f936015b-d62b-4899-a3f0-8d91efb385b5 = dead
# (for damla)
#trust 132503d4-dcde-4790-aabb-ee5ba539a3a0 = semitrusted
# (for ozge)
#trust 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = semitrusted
# (for mini)
#trust 7deafa14-8add-4316-984f-9b24eb713770 = semitrusted
# (for irem)
#trust aabc3536-a423-42b6-a234-5f110607296e = semitrusted
# (for yesim)
#trust ba3593c0-ddf1-4433-9916-aa25d1a52895 = semitrusted
# (for hubic)
#trust f980f309-0ebd-41c8-9303-73aff6409365 = semitrusted

# Repository groups
# (Standard groups: client transfer backup incrementalbackup smallarchive archive source manual public unwanted)
# (Separate group names with spaces)
# (for damla)
group 132503d4-dcde-4790-aabb-ee5ba539a3a0 = PodA
# (for ozge)
group 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = PodA
# (for irem)
group aabc3536-a423-42b6-a234-5f110607296e = PodA
# (for yesim)
group ba3593c0-ddf1-4433-9916-aa25d1a52895 = PodA
# (for buse [origin])
group bff7238e-bd92-4929-88a8-c59c1a1dcf03 = PodA
# (for hubic)
group f980f309-0ebd-41c8-9303-73aff6409365 = PodB
# (for mini)
group 7deafa14-8add-4316-984f-9b24eb713770 = manual
# (for web)
#group 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#group 00000000-0000-0000-0000-000000000002 = 

# Repository preferred contents
# (Set to "standard" to use a repository's group's preferred contents)
# (for damla)
wanted 132503d4-dcde-4790-aabb-ee5ba539a3a0 = groupwanted
# (for ozge)
wanted 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = groupwanted
# (for irem)
wanted aabc3536-a423-42b6-a234-5f110607296e = groupwanted
# (for yesim)
wanted ba3593c0-ddf1-4433-9916-aa25d1a52895 = groupwanted
# (for buse [origin])
wanted bff7238e-bd92-4929-88a8-c59c1a1dcf03 = groupwanted
# (for hubic)
wanted f980f309-0ebd-41c8-9303-73aff6409365 = groupwanted
# (for mini)
wanted 7deafa14-8add-4316-984f-9b24eb713770 = standard
# (for web)
#wanted 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#wanted 00000000-0000-0000-0000-000000000002 = 

# Group preferred contents
# (Used by repositories with "groupwanted" in their preferred contents)
groupwanted storagePodA = 
groupwanted storagePodB = 
groupwanted PodA = not copies=PodA:1
groupwanted PodB = not copies=PodB:1
#groupwanted archive = 
#groupwanted backup = 
#groupwanted client = 
#groupwanted incrementalbackup = 
#groupwanted manual = 
#groupwanted public = 
#groupwanted smallarchive = 
#groupwanted source = 
#groupwanted transfer = 
#groupwanted unwanted = 

# Standard preferred contents
# (Used by wanted or groupwanted expressions containing "standard")
# (For reference only; built-in and cannot be changed!)
# standard client = (include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1
# standard transfer = (not (inallgroup=client and copies=client:2) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)) or approxlackingcopies=1
# standard backup = anything
# standard incrementalbackup = ((not copies=backup:1) and (not copies=incrementalbackup:1)) or approxlackingcopies=1
# standard smallarchive = ((include=*/archive/* or include=archive/*) and ((not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1)) or approxlackingcopies=1
# standard archive = (not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1
# standard source = not (copies=1)
# standard manual = present and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)
# standard public = inpreferreddir
# standard unwanted = not anything

# Repository required contents
# (for damla)
required 132503d4-dcde-4790-aabb-ee5ba539a3a0 = 
# (for ozge)
required 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = 
# (for irem)
required aabc3536-a423-42b6-a234-5f110607296e = 
# (for yesim)
required ba3593c0-ddf1-4433-9916-aa25d1a52895 = 
# (for buse [origin])
required bff7238e-bd92-4929-88a8-c59c1a1dcf03 = 
# (for web)
#required 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#required 00000000-0000-0000-0000-000000000002 = 
# (for mini)
#required 7deafa14-8add-4316-984f-9b24eb713770 = 
# (for hubic)
#required f980f309-0ebd-41c8-9303-73aff6409365 = 

# Scheduled activities
# (Separate multiple activities with "; ")
# (for web)
#schedule 00000000-0000-0000-0000-000000000001 = 
# (for bittorrent)
#schedule 00000000-0000-0000-0000-000000000002 = 
# (for damla)
#schedule 132503d4-dcde-4790-aabb-ee5ba539a3a0 = 
# (for ozge)
#schedule 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = 
# (for mini)
#schedule 7deafa14-8add-4316-984f-9b24eb713770 = 
# (for irem)
#schedule aabc3536-a423-42b6-a234-5f110607296e = 
# (for yesim)
#schedule ba3593c0-ddf1-4433-9916-aa25d1a52895 = 
# (for buse [origin])
#schedule bff7238e-bd92-4929-88a8-c59c1a1dcf03 = 
# (for hubic)
#schedule f980f309-0ebd-41c8-9303-73aff6409365 = 

"""]]

With this settings all the repos tries get all of the files files that are present in other groups. Is there a way to debug this problem and tell why git-annex is trying to get the files? According to the settings it should not. I've also tried deleting all history this is a big (around 7 TB) old repo I thought maybe something got messed up along the way but it did not fix it.

### What steps will reproduce the problem?


### What version of git-annex are you using? On what operating system?

[[!format sh """
git-annex version: 5.20150825-g7826f84
build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA Database                                                                                                     
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SKEIN256E SKEIN256 SKEIN512E SKEIN512
 SHA1E SHA1 MD5E MD5 WORM URL                                                                                       
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external local repository version: 5
supported repository version: 5
upgrade supported from repository versions: 0 1 2 4
"""]]

On Ubuntu 14.04. All repos are on external USB drives on the same machine except the one called "mini".

### Please provide any additional information below.

[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log


# End of transcript or log.
"""]]

### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)


[1] https://git-annex.branchable.com/forum/git-annex_does_not_respect_preferred_content_settings/

> [[fixed|done]] --[[Joey]]