diff options
-rw-r--r-- | doc/bugs/Prefered_Content_not_Taken_into_Account.mdwn | 362 |
1 files changed, 362 insertions, 0 deletions
diff --git a/doc/bugs/Prefered_Content_not_Taken_into_Account.mdwn b/doc/bugs/Prefered_Content_not_Taken_into_Account.mdwn new file mode 100644 index 000000000..818181dff --- /dev/null +++ b/doc/bugs/Prefered_Content_not_Taken_into_Account.mdwn @@ -0,0 +1,362 @@ +### Please describe the problem. + +This is a follow up to the question [1]. I've created 4 repos with the following vicfg. + +[[!format sh """ +# git-annex configuration +# +# Changes saved to this file will be recorded in the git-annex branch. +# +# Lines in this file have the format: +# setting field = value + +# Repository trust configuration +# (Valid trust levels: trusted semitrusted untrusted dead) +# (for web) +#trust 00000000-0000-0000-0000-000000000001 = semitrusted +# (for bittorrent) +#trust 00000000-0000-0000-0000-000000000002 = semitrusted +# (for repo2) +#trust 2837f3d7-7c58-4177-8877-620213cf5146 = semitrusted +# (for repo3) +#trust b3cbb656-e797-45ba-bf43-08523d463146 = semitrusted +# (for repo1) +#trust e7673de4-d465-4557-be8f-b1400acf923e = semitrusted + +# Repository groups +# (Standard groups: client transfer backup incrementalbackup smallarchive archive source manual public unwanted) +# (Separate group names with spaces) +# (for repo3) +group b3cbb656-e797-45ba-bf43-08523d463146 = PodA +# (for repo1) +group e7673de4-d465-4557-be8f-b1400acf923e = PodA +# (for repo2) +group 2837f3d7-7c58-4177-8877-620213cf5146 = PodB +# (for web) +#group 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#group 00000000-0000-0000-0000-000000000002 = + +# Repository preferred contents +# (Set to "standard" to use a repository's group's preferred contents) +# (for repo2) +wanted 2837f3d7-7c58-4177-8877-620213cf5146 = groupwanted +# (for repo1) +wanted e7673de4-d465-4557-be8f-b1400acf923e = groupwanted +# (for web) +#wanted 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#wanted 00000000-0000-0000-0000-000000000002 = +# (for repo3) +#wanted b3cbb656-e797-45ba-bf43-08523d463146 = + +# Group preferred contents +# (Used by repositories with "groupwanted" in their preferred contents) +groupwanted PodA = not copies=PodA:1 +groupwanted PodB = not copies=PodB:1 +#groupwanted archive = +#groupwanted backup = +#groupwanted client = +#groupwanted incrementalbackup = +#groupwanted manual = +#groupwanted public = +#groupwanted smallarchive = +#groupwanted source = +#groupwanted transfer = +#groupwanted unwanted = + +# Standard preferred contents +# (Used by wanted or groupwanted expressions containing "standard") +# (For reference only; built-in and cannot be changed!) +# standard client = (include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1 +# standard transfer = (not (inallgroup=client and copies=client:2) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)) or approxlackingcopies=1 +# standard backup = anything +# standard incrementalbackup = ((not copies=backup:1) and (not copies=incrementalbackup:1)) or approxlackingcopies=1 +# standard smallarchive = ((include=*/archive/* or include=archive/*) and ((not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1)) or approxlackingcopies=1 +# standard archive = (not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1 +# standard source = not (copies=1) +# standard manual = present and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1) +# standard public = inpreferreddir +# standard unwanted = not anything + +# Repository required contents +# (for web) +#required 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#required 00000000-0000-0000-0000-000000000002 = +# (for repo2) +#required 2837f3d7-7c58-4177-8877-620213cf5146 = +# (for repo3) +#required b3cbb656-e797-45ba-bf43-08523d463146 = +# (for repo1) +#required e7673de4-d465-4557-be8f-b1400acf923e = + +# Scheduled activities +# (Separate multiple activities with "; ") +# (for web) +#schedule 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#schedule 00000000-0000-0000-0000-000000000002 = +# (for repo2) +#schedule 2837f3d7-7c58-4177-8877-620213cf5146 = +# (for repo3) +#schedule b3cbb656-e797-45ba-bf43-08523d463146 = +# (for repo1) +#schedule e7673de4-d465-4557-be8f-b1400acf923e = +"""]] + +This works as expected each file lands in 1 on of the repos on the group, not both. My original repo one with the problem has the following vicfg. + +[[!format sh """ +# git-annex configuration +# +# Changes saved to this file will be recorded in the git-annex branch. +# +# Lines in this file have the format: +# setting field = value + +# Repository trust configuration +# (Valid trust levels: trusted semitrusted untrusted dead) +# (for buse [origin]) +trust bff7238e-bd92-4929-88a8-c59c1a1dcf03 = semitrusted +# (for web) +trust 00000000-0000-0000-0000-000000000001 = untrusted +# (for bittorrent) +trust 00000000-0000-0000-0000-000000000002 = untrusted +# (for ) +trust 03b19bdc-8497-4234-9ef6-c80d78e309f8 = dead +# (for ) +trust 0b1a34ce-a8e3-4002-b46a-83e96626305f = dead +# (for ) +trust 1459a238-6f60-4b43-8459-f7055b49e3c5 = dead +# (for ) +trust 14738d0f-2f50-4f93-a8e4-c29870d02ba1 = dead +# (for ) +trust 1512a830-7789-4bc8-a4a0-4c51eedd1109 = dead +# (for ) +trust 174d69cb-9b97-44df-9f15-aea00f66a61c = dead +# (for ) +trust 1ade7f29-2ad0-4ab0-a816-de55ffde389e = dead +# (for ) +trust 1ba9d14b-deae-46e6-b451-dbf0fa8aa934 = dead +# (for ) +trust 1dda84ca-4e8c-49b4-b673-35ec2b11557b = dead +# (for ) +trust 29aad234-82e6-45c9-9822-b650290c2264 = dead +# (for ) +trust 2be091a2-e721-4a86-a759-c5df7a41a61d = dead +# (for ) +trust 3540b01a-03f0-4a58-8ea1-7ca80002a22d = dead +# (for ) +trust 39b4d431-9d55-432d-a801-eaf4cf643de1 = dead +# (for ) +trust 3a443e72-e8e0-11e2-b7db-8feb865236d4 = dead +# (for ) +trust 42023cde-e95f-483c-8389-54b823aed789 = dead +# (for ) +trust 54a819c1-65a3-44e9-86c3-0eb602223ee8 = dead +# (for ) +trust 68846c04-912d-4bfc-80f3-3b8ec1e7a3c7 = dead +# (for ) +trust 69b10f7e-0214-4fd0-8138-ff4717d1f39f = dead +# (for ) +trust 7b236779-a961-4e72-abfc-56b764e53a9c = dead +# (for ) +trust 7cde3feb-3876-4c2a-a772-d4541eac8614 = dead +# (for ) +trust 8f3e7b2b-f8de-4114-bcdd-2ef3ed932c18 = dead +# (for ) +trust a3bd1546-e8df-11e2-bc54-a7edbf29f5da = dead +# (for ) +trust aa830256-4c63-44e3-9a3f-33b377de79d5 = dead +# (for ) +trust ad46e36b-df9c-4d45-9af1-58d17e798afa = dead +# (for ) +trust b5464382-b5f8-4ec3-b36d-a378b1202497 = dead +# (for ) +trust b6317827-97d3-44bd-8197-91cae0b56a19 = dead +# (for ) +trust bbb5dd98-f09d-11e2-a57b-d7185df9eef1 = dead +# (for ) +trust c6dfb856-a0cc-4e79-acbd-3600f4bce158 = dead +# (for ) +trust ccb11b54-ecc8-11e2-ad6c-af6445c8eedc = dead +# (for ) +trust cdbf33bc-17bc-49e3-b0d1-afa653a2c1f7 = dead +# (for ) +trust d8cf3fb5-4e8e-4732-b566-dab6f2d9dc4c = dead +# (for ) +trust e160a8ef-3b3d-4f0e-88eb-9ceba11b2f57 = dead +# (for ) +trust e39439a0-1486-55a0-ada3-c9785c08d650 = dead +# (for ) +trust e8bcb64b-17df-4d79-a5b6-9d444e4afa51 = dead +# (for ) +trust eae76b0e-ada9-4c5e-b9e3-db5998324e70 = dead +# (for ) +trust ec71b4bd-1b7b-408c-906f-91c4168fd89f = dead +# (for ) +trust f936015b-d62b-4899-a3f0-8d91efb385b5 = dead +# (for damla) +#trust 132503d4-dcde-4790-aabb-ee5ba539a3a0 = semitrusted +# (for ozge) +#trust 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = semitrusted +# (for mini) +#trust 7deafa14-8add-4316-984f-9b24eb713770 = semitrusted +# (for irem) +#trust aabc3536-a423-42b6-a234-5f110607296e = semitrusted +# (for yesim) +#trust ba3593c0-ddf1-4433-9916-aa25d1a52895 = semitrusted +# (for hubic) +#trust f980f309-0ebd-41c8-9303-73aff6409365 = semitrusted + +# Repository groups +# (Standard groups: client transfer backup incrementalbackup smallarchive archive source manual public unwanted) +# (Separate group names with spaces) +# (for damla) +group 132503d4-dcde-4790-aabb-ee5ba539a3a0 = PodA +# (for ozge) +group 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = PodA +# (for irem) +group aabc3536-a423-42b6-a234-5f110607296e = PodA +# (for yesim) +group ba3593c0-ddf1-4433-9916-aa25d1a52895 = PodA +# (for buse [origin]) +group bff7238e-bd92-4929-88a8-c59c1a1dcf03 = PodA +# (for hubic) +group f980f309-0ebd-41c8-9303-73aff6409365 = PodB +# (for mini) +group 7deafa14-8add-4316-984f-9b24eb713770 = manual +# (for web) +#group 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#group 00000000-0000-0000-0000-000000000002 = + +# Repository preferred contents +# (Set to "standard" to use a repository's group's preferred contents) +# (for damla) +wanted 132503d4-dcde-4790-aabb-ee5ba539a3a0 = groupwanted +# (for ozge) +wanted 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = groupwanted +# (for irem) +wanted aabc3536-a423-42b6-a234-5f110607296e = groupwanted +# (for yesim) +wanted ba3593c0-ddf1-4433-9916-aa25d1a52895 = groupwanted +# (for buse [origin]) +wanted bff7238e-bd92-4929-88a8-c59c1a1dcf03 = groupwanted +# (for hubic) +wanted f980f309-0ebd-41c8-9303-73aff6409365 = groupwanted +# (for mini) +wanted 7deafa14-8add-4316-984f-9b24eb713770 = standard +# (for web) +#wanted 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#wanted 00000000-0000-0000-0000-000000000002 = + +# Group preferred contents +# (Used by repositories with "groupwanted" in their preferred contents) +groupwanted storagePodA = +groupwanted storagePodB = +groupwanted PodA = not copies=PodA:1 +groupwanted PodB = not copies=PodB:1 +#groupwanted archive = +#groupwanted backup = +#groupwanted client = +#groupwanted incrementalbackup = +#groupwanted manual = +#groupwanted public = +#groupwanted smallarchive = +#groupwanted source = +#groupwanted transfer = +#groupwanted unwanted = + +# Standard preferred contents +# (Used by wanted or groupwanted expressions containing "standard") +# (For reference only; built-in and cannot be changed!) +# standard client = (include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1 +# standard transfer = (not (inallgroup=client and copies=client:2) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)) or approxlackingcopies=1 +# standard backup = anything +# standard incrementalbackup = ((not copies=backup:1) and (not copies=incrementalbackup:1)) or approxlackingcopies=1 +# standard smallarchive = ((include=*/archive/* or include=archive/*) and ((not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1)) or approxlackingcopies=1 +# standard archive = (not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1 +# standard source = not (copies=1) +# standard manual = present and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1) +# standard public = inpreferreddir +# standard unwanted = not anything + +# Repository required contents +# (for damla) +required 132503d4-dcde-4790-aabb-ee5ba539a3a0 = +# (for ozge) +required 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = +# (for irem) +required aabc3536-a423-42b6-a234-5f110607296e = +# (for yesim) +required ba3593c0-ddf1-4433-9916-aa25d1a52895 = +# (for buse [origin]) +required bff7238e-bd92-4929-88a8-c59c1a1dcf03 = +# (for web) +#required 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#required 00000000-0000-0000-0000-000000000002 = +# (for mini) +#required 7deafa14-8add-4316-984f-9b24eb713770 = +# (for hubic) +#required f980f309-0ebd-41c8-9303-73aff6409365 = + +# Scheduled activities +# (Separate multiple activities with "; ") +# (for web) +#schedule 00000000-0000-0000-0000-000000000001 = +# (for bittorrent) +#schedule 00000000-0000-0000-0000-000000000002 = +# (for damla) +#schedule 132503d4-dcde-4790-aabb-ee5ba539a3a0 = +# (for ozge) +#schedule 1e1d0c4e-b1da-465f-9140-7128a7e3ee13 = +# (for mini) +#schedule 7deafa14-8add-4316-984f-9b24eb713770 = +# (for irem) +#schedule aabc3536-a423-42b6-a234-5f110607296e = +# (for yesim) +#schedule ba3593c0-ddf1-4433-9916-aa25d1a52895 = +# (for buse [origin]) +#schedule bff7238e-bd92-4929-88a8-c59c1a1dcf03 = +# (for hubic) +#schedule f980f309-0ebd-41c8-9303-73aff6409365 = + +"""]] + +With this settings all the repos tries get all of the files files that are present in other groups. Is there a way to debug this problem and tell why git-annex is trying to get the files? According to the settings it should not. I've also tried deleting all history this is a big (around 7 TB) old repo I thought maybe something got messed up along the way but it did not fix it. + +### What steps will reproduce the problem? + + +### What version of git-annex are you using? On what operating system? + +[[!format sh """ +git-annex version: 5.20150825-g7826f84 +build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA Database +key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 + SHA1E SHA1 MD5E MD5 WORM URL +remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external local repository version: 5 +supported repository version: 5 +upgrade supported from repository versions: 0 1 2 4 +"""]] + +On Ubuntu 14.04. All repos are on external USB drives on the same machine except the one called "mini". + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + + +[1] https://git-annex.branchable.com/forum/git-annex_does_not_respect_preferred_content_settings/ |