| Commit message (Collapse) | Author | Age |
|---|
| ... | |
| |
|
|
|
|
|
|
| |
Push it down from needing to be done in every Storer,
to being checked once inside ChunkedEncryptable.
Also, catch exceptions from PrepareStorer and PrepareRetriever,
just in case..
|
| | |
|
| |
|
|
|
| |
The best thing to do in this case is return Left, so that anything that
tries to access it will fail.
|
| |
|
|
|
|
|
|
| |
And clean up legacy chunking code, which is in its own module now.
So much cleaner!
This commit was sponsored by Henrik Ahlgren
|
| |
|
|
|
|
|
| |
Only when storing non-encrypted so far, not retrieving or checking if a key
is present or removing.
This commit was sponsored by Renaud Casenave-Péré.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Moved old legacy chunking code, and cleaned up the directory and webdav
remotes use of it, so when no chunking is configured, that code is not
used.
The config for new style chunking will be chunk=1M instead of chunksize=1M.
There should be no behavior changes from this commit.
This commit was sponsored by Andreas Laas.
|
| |
|
|
|
| |
Avoids abusing setting environment variables, which was always a hack
and won't work on windows.
|
| |
|
|
|
| |
And some reworking of types, and added an annex-availability git config
setting.
|
| |
|
|
|
|
|
|
|
| |
encryption=pubkey.
I think both of these are all that's affected, but I went ahead and fixed
all the remotes that set their config to M.empty to instead store the
actual config. Who knows what will expect it to be actually present in
future, the Remote instance of getGpgEncParams came to..
|
| |
|
|
|
|
|
|
| |
(eg, on removable drives)
gcrypt remotes are not yet handled.
This commit was sponsored by Sören Brunk.
|
| |
|
|
| |
chunkcount file to not be written. Work around repositories without such a file, so files can still be retreived from them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently only implemented for local git remotes. May try to add support
to git-annex-shell for ssh remotes later. Could concevably also be
supported by some special remote, although that seems unlikely.
Cronner user this when available, and when not falls back to
fsck --fast --from remote
git annex fsck --from does not itself use this interface.
To do so, I would need to pass --fast and all other options that influence
fsck on to the git annex fsck that it runs inside the remote. And that
seems like a lot of work for a result that would be no better than
cd remote; git annex fsck
This may need to be revisited if git-annex-shell gets support, since it
may be the case that the user cannot ssh to the server to run git-annex
fsck there, but can run git-annex-shell there.
This commit was sponsored by Damien Diederen.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
To support this, a core.gcrypt-id is stored by git-annex inside the git
config of a local gcrypt repository, when setting it up.
That is compared with the remote's cached gcrypt-id. When different, a
drive has been changed. git-annex then looks up the remote config for
the uuid mapped from the core.gcrypt-id, and tweaks the configuration
appropriately. When there is no known config for the uuid, it will refuse to
use the remote.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a git-remote-gcrypt encrypted special remote. Only sending files
in to the remote works, and only for local repositories.
Most of the work so far has involved making initremote work. A particular
problem is that remote setup in this case needs to generate its own uuid,
derivied from the gcrypt-id. That required some larger changes in the code
to support.
For ssh remotes, this will probably just reuse Remote.Rsync's code, so
should be easy enough. And for downloading from a web remote, I will need
to factor out the part of Remote.Git that does that.
One particular thing that will need work is supporting hot-swapping a local
gcrypt remote. I think it needs to store the gcrypt-id in the git config of the
local remote, so that it can check it every time, and compare with the
cached annex-uuid for the remote. If there is a mismatch, it can change
both the cached annex-uuid and the gcrypt-id. That should work, and I laid
some groundwork for it by already reading the remote's config when it's
local. (Also needed for other reasons.)
This commit was sponsored by Daniel Callahan.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
With the initremote parameters "encryption=pubkey keyid=788A3F4C".
/!\ Adding or removing a key has NO effect on files that have already
been copied to the remote. Hence using keyid+= and keyid-= with such
remotes should be used with care, and make little sense unless the point
is to replace a (sub-)key by another. /!\
Also, a test case has been added to ensure that the cipher and file
contents are encrypted as specified by the chosen encryption scheme.
|
| |
|
|
|
|
| |
directory special remote.
Directory special remotes now fully usable.
|
| |
|
|
|
| |
The latter is harder for me to remember, but avoids build failures in code
used by the configure program.
|
| |
|
|
| |
hold an object, was broken when using encryption.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
is made absolute.
Using a relative path would work, until the user changed to some other
directory in the repo and tried to access the remote from there..
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Most remotes have meters in their implementations of retrieveKeyFile
already. Simply hooking these up to the transfer log makes that information
available. Easy peasy.
This is particularly valuable information for encrypted remotes, which
otherwise bypass the assistant's polling of temp files, and so don't have
good progress bars yet.
Still some work to do here (see progressbars.mdwn changes), but this
is entirely an improvement from the lack of progress bars for encrypted
downloads.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
There was confusion in different parts of the progress bar code about
whether an update contained the total number of bytes transferred, or the
number of bytes transferred since the last update. One way this bug
showed up was progress bars that seemed to stick at zero for a long time.
In order to fix it comprehensively, I add a new BytesProcessed data type,
that is explicitly a total quantity of bytes, not a delta.
Note that this doesn't necessarily fix every problem with progress bars.
Particularly, buffering can now cause progress bars to seem to run ahead
of transfers, reaching 100% when data is still being uploaded.
|
| | |
|
| |
|
|
|
| |
Added a function to insert a new cost into a list, which could be used to
asjust costs after a drag and drop.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
git annex init probes for crippled filesystems, and sets direct mode, as
well as `annex.crippledfilesystem`.
Avoid manipulating permissions of files on crippled filesystems.
That would likely cause an exception to be thrown.
Very basic support in Command.Add for cripped filesystems; avoids the lock
down entirely since doing it needs both permissions and hard links.
Will make this better soon.
|
| |
|
|
| |
the transfer, which can happen in direct mode.
|
| | |
|
| |
|
|
|
| |
Still a couple of places that use git config ad-hoc, but this is most of it
done.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
too small chunksize was configured.
Ensure that each file has something written to it, even if the bytestring
chunk size is greater than the configured chunksize.
This means we may write a bit larger than the configured value, but only
when the configured value is very small; ie, < 8 kb.
|
| |
|
|
|
|
|
|
|
| |
Files are now written to a tmp directory in the remote, and once all
chunks are written, etc, it's moved into the final place atomically.
For now, checkpresent still checks every single chunk of a file, because
the old method could leave partially transferred files with some chunks
present and others not.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Both the directory and webdav special remotes used to have to buffer
the whole file contents before it could be decrypted, as they read
from chunks. Now the chunks are streamed through gpg with no buffering.
|
| |
|
|
|
| |
Note that receiving encrypted chunked content currently involves buffering.
(So does doing so with the directory special remote.)
|
| |
|
|
|
| |
However, directory still uses its optimzed chunked file writer, as it uses
less memory than the generic one in the helper.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
Transfer info files are updated when the callback is called, updating
the number of bytes transferred.
Left unused p variables at every place the callback should be used.
Which is rather a lot..
|
| |
|
|
|
|
|
| |
Currently only the web special remote is readonly, but it'd be possible to
also have readonly drives, or other remotes. These are handled in the
assistant by only downloading from them, and never trying to upload to
them.
|
| | |
|
| |
|
|
|
| |
Also broke out some helper functions around constructing remotes,
to be used later.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to record a semi-useful filename associated with the key,
this required plumbing the filename all the way through to the remotes'
storeKey and retrieveKeyFile.
Note that there is potential for deadlock here, narrowly avoided.
Suppose the repos are A and B. A sends file foo to B, and at the same
time, B gets file foo from A. So, A locks its upload transfer info file,
and then locks B's download transfer info file. At the same time,
B is taking the two locks in the opposite order. This is only not a
deadlock because the lock code does not wait, and aborts. So one of A or
B's transfers will be aborted and the other transfer will continue.
Whew!
|
Joey Hess
guilhem
Joey Hess