summaryrefslogtreecommitdiff
path: root/doc/bugs/git-annex_losing_rsync_remotes_with_encryption_enabled.mdwn
blob: 93588b49c12a60503e0f0353dba2a64c98f2f251 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
Somehow git-annex has again lost a complete rsync remote with encryption enabled...

git-annex version was 3.20111111

> "once again" ? When did it do it before?

>> It's the second time i uploaded all the files to an encrypted rsync remote and git-annex is not able to find it anymore. --[[gebi]]

> "lost" ? How is the remote lost?

>> git-annex is not able to find any files on the encrypted rsync remote anymore.
>> Copy does not copy the content again but drop doesn't find it, thus it's somehow "lost" and in an strange state.
>> I've also had the state where the content was already on the remote side but git-annex copy would copy it again,
>> ignoring all the data on the remote side. --[[gebi]]

Both *remoteserver* and *localserver* are rsync remotes with enabled encryption.
All commands are executed on the git repository on my laptop.
Target of origin is a gitolite repository without annex support (thus the two rsync remotes).

Is there a way in git-annex to verify that all files fullfill the numcopies, in my case
numcopies=2, and can be read from the remotes their are on?
I thought that *copy* would verify that, but seems not.
    
    % g a copy --to remoteserver tools
    copy tools/md5_sha1_utility.exe (gpg) (checking remoteserver...) ok
    copy tools/win32diskimager-RELEASE-0.2-r23-win32.zip (checking remoteserver...) ok
    
    % g a copy --to localserver tools
    copy tools/md5_sha1_utility.exe (gpg) (checking localserver...) ok
    copy tools/win32diskimager-RELEASE-0.2-r23-win32.zip (checking localserver...) ok
    
    % g a drop tools
    drop tools/md5_sha1_utility.exe (gpg) (checking localserver...) (checking remoteserver...) (unsafe)
      Could only verify the existence of 1 out of 2 necessary copies
      
      Try making some of these repositories available:
            718a9b5c-1b4a-11e1-8211-6f094f20e050 -- remoteserver (remote backupserver)
      
      (Use --force to override this check, or adjust annex.numcopies.)
    failed
    drop tools/win32diskimager-RELEASE-0.2-r23-win32.zip (checking localserver...) (checking remoteserver...) (unsafe)
      Could only verify the existence of 1 out of 2 necessary copies
      
      Try making some of these repositories available:
            718a9b5c-1b4a-11e1-8211-6f094f20e050 -- remoteserver (remote backupserver)
      
      (Use --force to override this check, or adjust annex.numcopies.)
    failed
    git-annex: drop: 2 failed
    
    % g a fsck tools
    fsck tools/md5_sha1_utility.exe (checksum...) ok
    fsck tools/win32diskimager-RELEASE-0.2-r23-win32.zip (checksum...) ok

> Copy does do an explicit check that the content is present on remoteserver,
> and based on the above, the content was found to be already there,
> which is why it did not copy it again.
> 
> Drop does an indentical check that the content is present, and 
> since it failed to find it, I am left thinking something must have
> happened to the remove in between the copy and the drop to cause the
> content to go away.
> 
> What happens if you copy the data to remoteserver again? --[[Joey]]

The commands above are executed within a few seconds and completely repeatable. --[[gebi]]

> In that case, why don't you run the commands with `-d` to see the actual
> rsync command it's running to check if the content is present. 
> Then you can try repeatedly running the command by hand and see why it
> sometimes succeeds and sometimes fail.
> 
> The command will be something like this:
> `rsync --quiet hostname:/dir/file 2>/dev/null`
> 
> The exit status is what's used to see if content is present -- and
> currently any failure even a failure to connect is taken to mean it's not
> present. --[[Joey]]