| Commit message (Collapse) | Author | Age |
|---|
| ... | |
| | | |
| | |
| | |
| | | |
Discovered thanks to testremote command!
|
| | | |
| | |
| | |
| | | |
Found by testremote
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This only performs some basic tests so far; no testing of chunking or
resuming. Also, the existing encryption type of the remote is used; it
would be good later to derive an encrypted and a non-encrypted version of
the remote and test them both.
This commit was sponsored by Joseph Liu.
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | |/
| |
| |
| |
| |
| | |
For example, I had a copy to a remote that was failing for an unknown
reason. This let me see the exception was createDirectory: permission
denied; the underlying problem being a permissions issue.
|
| | | |
|
| | |\ |
|
| | | |
| | |
| | |
| | |
| | | |
Probably the new ghc used on android is the root cause of needing this
constraint.
|
| | | |
| | |
| | |
| | |
| | | |
0.6.1 is in testing, and stable does not have DAV at all, so I can dispense
with this compatability code
|
| | | | |
|
| | | | |
|
| | |/ |
|
| | |
| |
| |
| |
| |
| | |
The content of unstable keys can potentially be different in different
repos, so eg, resuming a chunked upload started by another repo would
corrupt data.
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
This way, when the remote implementation neglects to update progress,
there will still be a somewhat useful progress display, as long as chunks
are used.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
encryption are not being used
No need to read whole FileContent only to write it back out to a file in
this case. Can just rename! Yay.
Also indidentially, fixed an attempt to open a file for write that was
already opened for write, which caused a crash and deadlock.
|
| | |
| |
| |
| |
| |
| |
| | |
Removing code and at the same time adding great features, including
upload/download resuming.
This commit was sponsored by Romain Lenglet.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Putting a callback in the Retriever type allows for the callback to
remove the retrieved file when it's done with it.
I did not really want to make Retriever be fixed to Annex Bool,
but when I tried to use Annex a, I got into some type of type mess.
|
| | |
| |
| |
| |
| |
| |
| |
| | |
Needed for eg, Remote.External.
Generally, any Retriever that stores content in a file is responsible for
updating the meter, while ones that procude a lazy bytestring cannot update
the meter, so are not asked to.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Some remotes like External need to run store and retrieve actions in Annex,
not IO. In order to do that lift, I had to dive pretty deep into the
utilities, making Utility.Gpg and Utility.Tmp be partly converted to using
MonadIO, and Control.Monad.Catch for exception handling.
There should be no behavior changes in this commit.
This commit was sponsored by Michael Barabanov.
|
| | | |
|
| | |
| |
| |
| |
| | |
Note that currently nothing cleans up a ContentSource's file, when eg,
retrieving chunks.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |\ |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
present until it's all done being stored
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Leverage the new chunked remotes to automatically resume uploads.
Sort of like rsync, although of course not as efficient since this
needs to start at a chunk boundry.
But, unlike rsync, this method will work for S3, WebDAV, external
special remotes, etc, etc. Only directory special remotes so far,
but many more soon!
This implementation will also allow starting an upload from one repository,
interrupting it, and then resuming the upload to the same remote from
an entirely different repository.
Note that I added a comment that storeKey should atomically move the content
into place once it's all received. This was already an undocumented
requirement -- it's necessary for hasKey to work reliably. This resume code
just uses hasKey to find the first chunk that's missing.
Note that if there are two uploads of the same key to the same chunked remote,
one might resume at the point the other had gotten to, but both will then
redundantly upload. As before.
In the non-resume case, this adds one hasKey call per storeKey, and only
if the remote is configured to use chunks. Future work: Try to eliminate that
hasKey. Notice that eg, `git annex copy --to` checks if the key is present
before sending it, so is already running hasKey.. which could perhaps
be cached and reused.
However, this additional overhead is not very large compared with
transferring an entire large file, and the ability to resume
is certianly worth it. There is an optimisation in place for small files,
that avoids trying to resume if the whole file fits within one chunk.
This commit was sponsored by Georg Bauer.
|
| | | | |
|
Joey Hess
gernot
https://www.google.com/accounts/o8/id?id=AItOawmAINLSovhWM_4_KrbngOcxduIbBuKv8ZA
Xyem
zardoz
https://www.google.com/accounts/o8/id?id=AItOawm8BAEUyzYhORZmMuocRTk4M-3IumDm5VU
http://olivier.mehani.name/
https://www.google.com/accounts/o8/id?id=AItOawn-TDneVW-8kwb1fyTRAJfH3l1xs2VSEmk
https://www.google.com/accounts/o8/id?id=AItOawmwjQzWgiD7_I3zw-_91rMRf_6qoThupis
http://joeyh.name/
https://www.google.com/accounts/o8/id?id=AItOawlYGpCG2u-0fVbpQhIDMX_ZfE64E-K2EMI
http://svario.it/gioele
https://id.koumbit.net/anarcat
https://www.google.com/accounts/o8/id?id=AItOawkftzaCvV7EDKVDfJhsQZ3E1Vn-0db516w