summaryrefslogtreecommitdiff
path: root/Utility/Rsync.hs
Commit message (Collapse)AuthorAge
* Added new toMSYS2Path function for use with rsync on Windows.Gravatar Pieter Kitslaar2016-01-11
|
* remove Params constructor from Utility.SafeCommandGravatar Joey Hess2015-06-01
| | | | | | | | | | | | | | | | | | This removes a bit of complexity, and should make things faster (avoids tokenizing Params string), and probably involve less garbage collection. In a few places, it was useful to use Params to avoid needing a list, but that is easily avoided. Problems noticed while doing this conversion: * Some uses of Params "oneword" which was entirely unnecessary overhead. * A few places that built up a list of parameters with ++ and then used Params to split it! Test suite passes.
* well along the way to fully quiet --quietGravatar Joey Hess2015-04-04
| | | | | | | Came up with a generic way to filter out progress messages while keeping errors, for commands that use stderr for both. --json mode will disable command outputs too.
* WIP on making --quiet silence progress, and infra for concurrent progress barsGravatar Joey Hess2015-04-03
|
* update my email address and homepage urlGravatar Joey Hess2015-01-21
|
* Windows: Got the rsync special remote working.Gravatar Joey Hess2014-12-30
| | | | | | | | | More aggressive rsync params fixup for windows. Param may contain a url, or a file path, so check if it looks like a local file path and if so, fix it up. On windows only, rsyncUrlIsPath will treat c:foo as a path, rather than as a rsyncurl starting with a host "c".
* Windows: Fix local rsync filepath munging (fixes 26 test suite failures).Gravatar Joey Hess2014-12-30
|
* refactorGravatar Joey Hess2014-12-17
|
* fix some mixed space+tab indentationGravatar Joey Hess2014-10-09
| | | | | | | | | This fixes all instances of " \t" in the code base. Most common case seems to be after a "where" line; probably vim copied the two space layout of that line. Done as a background task while listening to episode 2 of the Type Theory podcast.
* move ugly rsync zombie workaroundGravatar Joey Hess2014-08-03
| | | | | | | | | | | | This reaping of any processes came to cause me problems when redoing the rsync special remote -- a gpg process that was running gets waited on and the place that then checks its return code fails. I cannot reproduce any zombies when using the rsync special remote. But I still can when using a normal git remote, accessed over ssh. There is 1 zombie per file downloaded without this horrible hack enabled. So, move the hack to only be used in that case.
* relicense general utility library code to BSDGravatar Joey Hess2014-05-10
| | | | | Omitted a couple of files what have had significant contributions from others.
* Fix rsync progress parsing in locales that use comma in number display. ↵Gravatar Joey Hess2014-04-10
| | | | Closes: #744148
* rsync special remote: Fix fallback mode for rsync remotes that use ↵Gravatar Joey Hess2013-12-02
| | | | hashDirMixed. Closes: #731142
* deal with Cygwin rsync paths issueGravatar Joey Hess2013-05-14
|
* rsync special remotes: When sending from a crippled filesystem, use the ↵Gravatar Joey Hess2013-05-09
| | | | destination's default file permissions, as the local ones can be arbitrarily broken. (Ie, ----rwxr-x for files on Android)
* expose Control.Monad.joinGravatar Joey Hess2013-04-22
| | | | | I think I've been looking for that function for some time. Ie, I remember wanting to collapse Just Nothing to Nothing.
* minor refactoringGravatar Joey Hess2013-03-30
|
* don't refer to git-annex, as this is a generic utility libraryGravatar Joey Hess2013-03-30
|
* Make git-annex-shell call the command with its (safe) options.Gravatar guilhem2013-03-30
|
* webapp: Progess bar fixes for many types of special remotes.Gravatar Joey Hess2013-03-28
| | | | | | | | | | | | | There was confusion in different parts of the progress bar code about whether an update contained the total number of bytes transferred, or the number of bytes transferred since the last update. One way this bug showed up was progress bars that seemed to stick at zero for a long time. In order to fix it comprehensively, I add a new BytesProcessed data type, that is explicitly a total quantity of bytes, not a delta. Note that this doesn't necessarily fix every problem with progress bars. Particularly, buffering can now cause progress bars to seem to run ahead of transfers, reaching 100% when data is still being uploaded.
* Avoid passing -p to rsync, to interoperate with crippled filesystems.Gravatar Joey Hess2013-02-22
| | | | | | | | | In general, git-annex does not try to preserve file permissions. For example, they don't round trip through special remotes. So it's ok to not preserve them for git remotes either. On crippled filesystems, rsync has been observed failing after the file was transferred because it couldn't set some permission or other.
* finished where indentation changesGravatar Joey Hess2012-12-13
|
* better fix for zombie problem, which turns out to be a zombie ssh started by ↵Gravatar Joey Hess2012-10-17
| | | | | | | | | | | | | | | | | | | | | rsync When rsyncProgress pipes rsync's stdout, this turns out to cause a ssh process started by rsync to be left behind as a zombie. I don't know why, but my recent zombie reaping cleanup was correct, it's just that this other zombie, that's not directly started by git-annex, was no longer reaped due to changes in the cleanup. Make rsyncProgress reap the zombie started by rsync, as a workaround. FWIW, the process tree looks like this. It seems like the rsync child is for some reason starting but not waiting on this extra ssh process. Ssh connection caching may be involved -- disabling it seemed to change the shape of the tree, but did not eliminate the zombie. 9378 pts/14 S+ 0:00 | \_ rsync -p --progress --inplace -4 -e 'ssh' '-S' ... 9379 pts/14 S+ 0:00 | | \_ ssh ... 9380 pts/14 S+ 0:00 | | \_ rsync -p --progress --inplace -4 -e 'ssh' '-S' ... 9381 pts/14 Z+ 0:00 | \_ [ssh] <defunct>
* bring back default SIGINT handlerGravatar Joey Hess2012-10-01
| | | | | | | | | | | | This seems to fix a problem I've recently seen where ctrl-c during rsync leads to `git annex get` moving on to the next thing rather than exiting. Seems likely that started happening with the switch to System.Process (d1da9cf221aeea5c7ac8a313a18b559791a04f12), as the old code took care to install a default SIGINT handler. Note that since the bug was only occurring sometimes, I am not 100% sure I've squashed it, although I seem to have.
* avoid calling the progress callback when the bytes sent have not changedGravatar Joey Hess2012-09-20
| | | | | Does rsync stall and update its progress display? Dunno, but this was an easy optimisation to throw in.
* optimised rsync output reader to read whole blocks at a timeGravatar Joey Hess2012-09-20
|
* updateGravatar Joey Hess2012-09-20
|
* better parameter nameGravatar Joey Hess2012-09-19
|
* rsync progress interceptionGravatar Joey Hess2012-09-19
| | | | | | | | Current implementation parses rsync's output a character a time, which is hardly efficient. It could be sped up a lot by using hGetBufSome, but that would require going really lowlevel, down to raw C style buffers (good example of that here: http://users.aber.ac.uk/afc/stricthaskell.html) But rsync doesn't output very much, so currently it seems ok.
* parser for rsync progress outputGravatar Joey Hess2012-09-19
|
* renamed RsyncFile -> RsyncGravatar Joey Hess2012-09-19