1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
|
Currently, git-annex takes a very lazy approch to displaying
progress into. It just lets rsync or whatever display the progress
for it, in the terminal.
Something better is needed for the [[webapp]]. There needs to be a
way for the web app to know what the current progress is of all transfers.
This is one of those potentially hidden but time consuming problems.
## downloads
* Watch temp file as it's coming in and use its size.
This is the only option for some special remotes (ie, non-rsync).
Can either poll every .5 seconds or so to check file size, or
could use inotify. **done**
## uploads
Each individual remote type needs to implement its own support for calling
the ProgressCallback as the upload progresses.
* git: Done, with one exception: `git-annex-shell sendkey` runs `rsync
--server --sender` and in that mode it does not report progress info.
So downloads initiated by other repos do not show progress in the repo
doing the uploading.
Maybe I should
write a proxy for the rsync wire protocol that can tell what chunk of the
file is being sent, and shim it in front of the rsync server? Sadly,
the protocol is insane.
Another idea: Invert things. Make `git-annex-shell sendkey` run
`rsync -e 'cat'`, so it treats the incoming ssh connection as the server.
(cat probably won't really work; bidirectional pipe needed).
Run rsync in `--server` mode on the *client* side, piped to ssh.
Now the `git-annex` side doesn't have a progress bar (but it can poll the
file size and produce its own), `git-annex-shell` side does have a progress
bar.
Less crazy, but probably harder idea: Multiplex progress info from client
back to server over the ssh connection, and demultiplex at server end.
Or, use a separate ssh connection, and let ssh connection caching handle
the multiplexing.
* rsync: **done**
* directory
* web: Not applicable; does not upload
* S3
* bup
* hook: Would require the hook interface to somehow do this, which seems
too complicated. So skipping.
## communication
It may be worth using a better communication channel than files on disk for
the transfer progress. Shared memory could be used, or http posts to the
webapp.
|
