summaryrefslogtreecommitdiff
path: root/Logs/Transfer.hs
Commit message (Collapse)AuthorAge
* connect existing meters to the transfer log for downloadsGravatar Joey Hess2013-04-11
| | | | | | | | | | | | | | Most remotes have meters in their implementations of retrieveKeyFile already. Simply hooking these up to the transfer log makes that information available. Easy peasy. This is particularly valuable information for encrypted remotes, which otherwise bypass the assistant's polling of temp files, and so don't have good progress bars yet. Still some work to do here (see progressbars.mdwn changes), but this is entirely an improvement from the lack of progress bars for encrypted downloads.
* hlintGravatar Joey Hess2013-04-03
|
* show bytesCompleteGravatar Joey Hess2013-04-02
|
* webapp: Progess bar fixes for many types of special remotes.Gravatar Joey Hess2013-03-28
| | | | | | | | | | | | | There was confusion in different parts of the progress bar code about whether an update contained the total number of bytes transferred, or the number of bytes transferred since the last update. One way this bug showed up was progress bars that seemed to stick at zero for a long time. In order to fix it comprehensively, I add a new BytesProcessed data type, that is explicitly a total quantity of bytes, not a delta. Note that this doesn't necessarily fix every problem with progress bars. Particularly, buffering can now cause progress bars to seem to run ahead of transfers, reaching 100% when data is still being uploaded.
* get, copy, move: Display an error message when an identical transfer is ↵Gravatar Joey Hess2013-03-19
| | | | already in progress, rather than failing with no indication why.
* optimisation for transfers to drives that are not plugged inGravatar Joey Hess2013-03-18
| | | | | | Rather than forking a git-annex transferkey only to have it fail, just immediately record the failed transfer (so when the drive is plugged in, the scan will retry it).
* add additional debug info about reasons for transfersGravatar Joey Hess2013-03-01
|
* improve importsGravatar Joey Hess2013-02-27
|
* move Arbitrary instances out of Test and into modules that define the typesGravatar Joey Hess2013-02-27
| | | | This is possible now that we build-depend on QuickCheck.
* more quickcheck funGravatar Joey Hess2012-12-19
| | | | and the code gets better..
* quickcheck test for transfer info read/write codeGravatar Joey Hess2012-12-19
| | | | Fixed a bug the quickcheck turned up.
* Bugfix: Fixed bug parsing transfer info filesGravatar Joey Hess2012-12-19
| | | | | | | | | | | | | | | The newline after the filename was included in it. This was generally benign -- mostly these filenames are just displayed, and the newline didn't matter. But in the assistant, it caused unexpected dropping of preferred content. A characteristic of this bug is that the drop was displayed like this: drop some_file ok
* where indentingGravatar Joey Hess2012-11-11
|
* bugfix: Don't fail transferring content from read-only repos. Closes: #691341Gravatar Joey Hess2012-10-24
| | | | | This used to work, but got broken when the transfer info files were added, as it failed writing them on the readonly filesystem.
* !! removalGravatar Joey Hess2012-10-21
|
* fixes for transfer resumeGravatar Joey Hess2012-09-24
| | | | | | | | | Fix resuming of downloads, which do not have a transfer info file to read. When checking upload progress, use the MVar, rather than re-reading the info file. Catch exceptions in the transfer action. Required a tryAnnex.
* make the assistant retry failed transfersGravatar Joey Hess2012-09-23
| | | | | | | When a transfer fails, the progress info can be used to intelligently retry it. If the transfer managed to make some progress, but did not fully complete, then there's a good chance that a retry will finish it (or at least make more progress).
* git-annex-shell transferinfo commandGravatar Joey Hess2012-09-21
| | | | | TODO: Use this when running sendkey, to feed back transfer info from the client side rsync.
* avoid updating transfer info file until another 1% of the total has been ↵Gravatar Joey Hess2012-09-21
| | | | transferred
* unify typesGravatar Joey Hess2012-09-21
|
* fix reading of transfer info files with a bytesComplete valueGravatar Joey Hess2012-09-20
|
* add a progress callback to storeKey, and threaded it all the way throughGravatar Joey Hess2012-09-19
| | | | | | | | Transfer info files are updated when the callback is called, updating the number of bytes transferred. Left unused p variables at every place the callback should be used. Which is rather a lot..
* make other repositories list list all autostarted reposGravatar Joey Hess2012-09-18
| | | | And add a form to add another, unrelated repository
* cleanupGravatar Joey Hess2012-09-17
|
* flip catchDefaultIOGravatar Joey Hess2012-09-17
|
* Avoid crashing on encoding errors in filenames when writing transfer info ↵Gravatar Joey Hess2012-09-16
| | | | files and reading from checksum commands.
* stupid typoGravatar Joey Hess2012-08-29
|
* when canceling a transfer, also cancel all other downloads of the same keyGravatar Joey Hess2012-08-29
|
* fix resume buttonGravatar Joey Hess2012-08-29
| | | | | Change alterTransferInfo to not merge in old values, including transferPaused.
* don't show "unknown" as the percent complete for transferinfo with no ↵Gravatar Joey Hess2012-08-28
| | | | bytesComplete value
* avoid possibly re-adding a removed transfer when updating its infoGravatar Joey Hess2012-08-28
| | | | Doesn't fix the bug I thought it'd fix, but is clearly correct.
* keep track of which remotes have been scanned in process stateGravatar Joey Hess2012-08-24
| | | | | Since it turned out to make sense to always scan all remotes on startup, there's no need to persist the info about which have been scanned.
* keep logs of failed transfers, and requeue them when doing a non-full scanGravatar Joey Hess2012-08-23
| | | | of a remote
* add transfer scanned flag filesGravatar Joey Hess2012-08-23
|
* implement pausing of transfersGravatar Joey Hess2012-08-10
| | | | | | A paused transfer's thread keeps running, keeping the slot in use. This is intentional; pausing a transfer should not let other queued transfers to run in its place.
* add routes to pause/start/cancel transfersGravatar Joey Hess2012-08-08
| | | | | | | | | | | | | | | | This commit includes a paydown on technical debt incurred two years ago, when I didn't know that it was bad to make custom Read and Show instances for types. As the routes need Read and Show for Transfer, which includes a Key, and deriving my own Read instance of key was not practical, I had to finally clean that up. So the compact Key read and show functions are now file2key and key2file, and Read and Show are now derived instances. Changed all code that used the old instances, compiler checked. (There were a few places, particularly in Command.Unused, and the test suite where the Show instance continue to be used for legitimate comparisons; ie show key_x == show key_y (though really in a bloom filter))
* Merge branch 'master' into assistantGravatar Joey Hess2012-08-07
|\
| * fix transfer log cleanup crashGravatar Joey Hess2012-08-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid crashing when "git annex get" fails to download from one location, and falls back to downloading from a second location. The problem is that git annex get calls download recursively from within itself if the first download attempt fails. So the first time through, it writes a transfer info file, which is then overwritten on the second, recursive call. Then on cleanup, it tries to delete the file twice, which of course doesn't work. Fixed both by not crashing if the transfer file is removed, and by changing Get to not run download recursively like that. It's the only thing that did so, and it just seems like a bad idea.
* | webapp now displays the real running and queued transfersGravatar Joey Hess2012-07-27
| | | | | | | | yowza!!!
| * pull in transfer log code from assistant branchGravatar Joey Hess2012-07-18
| | | | | | | | New log file format.
* | fix parsing of startedTimeGravatar Joey Hess2012-07-18
| |
* | run file transfers in threads, not processesGravatar Joey Hess2012-07-18
| | | | | | | | | | | | This should fix OSX/BSD issues with not noticing transfer information files with kqueue. Now that threads are used, the thread can manage the transfer slot allocation and deallocation by itself; much cleaner.
* | add thread id field to transferinfoGravatar Joey Hess2012-07-18
| | | | | | | | | | Also converted its timestand to posix seconds, like is used in the other log files.
* | avoid parsing lock files as transfer filesGravatar Joey Hess2012-07-17
| | | | | | | | | | This seems to happen with kqueue, not inotify. The newly added lck file triggers an add event and was then parsed as a transfer file.
* | bugfixGravatar Joey Hess2012-07-17
| |
* | change transfer lock filenames to avoid ambiguityGravatar Joey Hess2012-07-17
| | | | | | | | | | foo.lck could be a lock file for a transfer of foo, or a transfer of a key that happened to end in ".lck". Fix this by using "lck.foo" instead.
* | split transfer info and lock filesGravatar Joey Hess2012-07-07
| | | | | | | | | | | | | | Since the lock file has to be kept open, this prevented the TransferWatcher from noticing when it appeared, since inotify (and more importantly kqueue) events happen when a new file is closed. Writing a separate info file fixes that problem.
* | wait on child transfer processes, and invalidate cacheGravatar Joey Hess2012-07-06
| | | | | | | | | | | | There's still a bug; if the child updates its transfer info file, then the data from it will superscede the TransferInfo, losing the info that we should wait on this child.
* | unfinished (and unbuildable) work toward separate transfer processesGravatar Joey Hess2012-07-05
| |
* | wrote transfer threadGravatar Joey Hess2012-07-05
| | | | | | | | finally!