| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
| |
When chunk=0, always try the unchunked key first. This avoids the overhead
of needing to read the git-annex branch to find the chunkcount.
However, if the unchunked key is not present, go on and try the chunks.
Also, when removing a chunked key, update the chunkcounts even when
chunk=0.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
No need to process each L.ByteString chunk, instead ask it to split.
Doesn't seem to have really sped things up much, but it also made the code
simpler.
Note that this does (and already did) buffer in memory. It seems that only
the directory special remote could take advantage of streaming chunks to
files w/o buffering, so probably won't add an interface to allow for that.
|
| |
|
|
|
|
|
| |
This will allow things like WebDAV to opean a single persistent connection
and reuse it for all the chunked data.
The crazy types allow for some nice code reuse.
|
| |
|
|
|
|
|
|
| |
Push it down from needing to be done in every Storer,
to being checked once inside ChunkedEncryptable.
Also, catch exceptions from PrepareStorer and PrepareRetriever,
just in case..
|
| | |
|
| |
|
|
|
| |
The best thing to do in this case is return Left, so that anything that
tries to access it will fail.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
And clean up legacy chunking code, which is in its own module now.
So much cleaner!
This commit was sponsored by Henrik Ahlgren
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I'd have liked to keep these two concepts entirely separate,
but that are entagled: Storing a key in an encrypted and chunked remote
need to generate chunk keys, encrypt the keys, chunk the data, encrypt the
chunks, and send them to the remote. Similar for retrieval, etc.
So, here's an implemnetation of all of that.
The total win here is that every remote was implementing encrypted storage
and retrival, and now it can move into this single place. I expect this
to result in several hundred lines of code being removed from git-annex
eventually!
This commit was sponsored by Henrik Ahlgren.
|
| |
|
|
|
|
|
| |
Chunk retrieval and reassembly, removal, and checking if all necessary
chunks are present.
This commit was sponsored by Damien Raude-Morvan.
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
Encyptable types of remotes that were not actually encrypted still had
the encryptedRemoteCostAdj applied to their configured cost, which was a
bug.
|
| | |
| |
| |
| |
| |
| |
| | |
Only when storing non-encrypted so far, not retrieving or checking if a key
is present or removing.
This commit was sponsored by Renaud Casenave-Péré.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Not yet used by any special remotes, but should not be too hard to add it
to most of them.
storeChunks is the hairy bit! It's loosely based on
Remote.Directory.storeLegacyChunked. The object is read in using a lazy
bytestring, which is streamed though, creating chunks as needed, without
ever buffering more than 1 chunk in memory.
Getting the progress meter update to work right was also fun, since
progress meter values are absolute. Finessed by constructing an offset
meter.
This commit was sponsored by Richard Collins.
|
| | |
|
| |
|
|
|
|
|
| |
Slightly tricky as they are not normal UUIDBased logs, but are instead maps
from (uuid, chunksize) to chunkcount.
This commit was sponsored by Frank Thomas.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Moved old legacy chunking code, and cleaned up the directory and webdav
remotes use of it, so when no chunking is configured, that code is not
used.
The config for new style chunking will be chunk=1M instead of chunksize=1M.
There should be no behavior changes from this commit.
This commit was sponsored by Andreas Laas.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
avoid unncessary passphrase prompts.
This is a security/usability tradeoff. To avoid exposing the gpg key ids
who can decrypt the repository, users can unset
gcrypt-publish-participants.
The gcrypt-publish-participants option is available in my fork of
git-remote-gcrypt.
This commit was sponsored by Christopher Kernahan.
|
| |
|
|
|
| |
The exception handling resulted in git config --list being run twice
for unavailable repos. This dials it back down to running it only once.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
annex.uuid set.
Catch an exception when ensureInitialized is run in a non-initted
repository. In this case, just read the git config, so that the Git.Repo
object is not LocalUnknown, which is what is used to represent remotes
on eg, drives that are not connected.
The assistant already got this right, and like with the assistant, this
causes an implicit git-annex init of the local remote on the second sync,
once the git-annex branch has been pushed to it.
See this comment for more analysis:
http://git-annex.branchable.com/todo/Recovering_from_a_bad_sync/#comment-64e469a2c1969829ee149cbb41b1c138
This commit was sponsored by jscit.
|
| |
|
|
| |
the location of a bucket, but only reading and writing content to it.
|
| | |
|
| |
|
|
| |
update code to avoid cwd and env redefinition warnings
|
| | |
|
| |
|
|
|
|
|
|
|
| |
I don't know if this was a bug, but I don't know if it was not a bug
either.
See also,
http://git-annex.branchable.com/bugs/Truncated_file_transferred_via_S3/
where the file is not truncated, but mangled..
|
| |
|
|
|
| |
initremote stores the location of an already existing git remote, and
enableremote setups up a remote using its stored location.
|
| | |
|
| |
|
|
|
|
|
|
| |
It is useful to be able to specify an alternative git-annex-shell
program to execute on the remote, e.g., to run a version not on the
PATH. Use remote.<name>.annex-shell if specified, instead of the
default "git-annex-shell" i.e., first so-named executable on the
PATH.
|
| |
|
|
| |
the user fix their problem
|
| | |
|
| | |
|
| |
|
|
| |
file system. This is a better approach to fix #700282 while not unncessarily losing file permissions on non-crippled systems.
|
| |
|
|
| |
implementation
|
| |
|
|
|
|
|
|
|
| |
Version 5.20140227 broke creation of glacier repositories, not including
the datacenter and vault in their configuration. This bug is fixed, but
glacier repositories set up with the broken version of git-annex need to
have the datacenter and vault set in order to be usable. This can be done
using git annex enableremote to add the missing settings. For details, see
http://git-annex.branchable.com/bugs/problems_with_glacier/
|
| | |
|
| |
|
|
|
|
|
| |
Motivation: Hook scripts for nautilus or other file managers
need to provide the user with feedback that a file is being downloaded.
This commit was sponsored by THM Schoemaker.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Benchmarking this with 1000 small files being copied, the time reduced from
15.98s to 14.64s -- an 8% improvement in the non-data-transfer overhead of
git-annex copy.
|
| |
|
|
| |
response timeout, which was only 5 seconds.
|
| |
|
|
|
| |
I suppose this is not necessary when it has a local cache, so I didn't
notice it was missing.
|
Joey Hess
Fraser Tweedale
Robie Basak
Alberto Berti