summaryrefslogtreecommitdiff
path: root/Remote
Commit message (Collapse)AuthorAge
* S3: Allow configuring with requeststyle=path to use path-style bucket access ↵Gravatar Joey Hess2016-02-09
| | | | | | instead of the default DNS-style access. untested
* WebDAV: Remove a bogus trailing slash from the end of the url to the ↵Gravatar Joey Hess2016-02-09
| | | | | | | | temporary store location for a key. Thanks, wzhd. That trailing slash is needed for legacy chunked mode, because it puts the chunks in a subdir under the key. But, outside legacy chunked mode, it's BS and it's amazing it worked at all with some webdav servers.
* WebDAV: Set depth 1 in PROPFIND request, for better compatability with some ↵Gravatar Joey Hess2016-02-09
| | | | servers. Thanks, wzhd.
* remove 3 build flagsGravatar Joey Hess2016-01-26
| | | | | | | | | | | | | * Removed the webapp-secure build flag, rolling it into the webapp build flag. * Removed the quvi and tahoe build flags, which only adds aeson to the core dependencies. * Removed the feed build flag, which only adds feed to the core dependencies. Build flags have cost in both code complexity and also make Setup configure have to work harder to find a usable set of build flags when some dependencies are missing.
* remove 163 lines of code without changing anything except importsGravatar Joey Hess2016-01-20
|
* avoid hard linking object from other repository when annex.thin is setGravatar Joey Hess2016-01-13
| | | | | This is simpler and less expensive than checking if the src file has a link count >= 2, and also is unlocked.
* remove reundant isDirect checkGravatar Joey Hess2016-01-13
| | | | Already checked in wantHardLink
* typoGravatar Joey Hess2015-12-26
|
* deal with unlocked files when calling rsyncParamsRemoteGravatar Joey Hess2015-12-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In copyFromRemote, it used to check isDirect, but that was not needed; the remote is sending the file, so it doesn't matter if the local, receiving repository is in direct mode or not. And, since the content is not present, yet, it's certianly not unlocked. Note that, the remote may indeed be sending an unlocked file, but sendkey uses sendAnnex, which will detect if the file is modified before or during transfer, and will exit nonzero, aborting the upload. So, the receiver doesn't need any checks. In copyToRemote, it forces recvkey to verify content whenever it's being sent from a v6 repository. recvkey is almost always going to verify content anyway, unless annex.verify is not set. So, this doesn't make it any more expensive, except for in that unusual configuration. The alternative would be to change the recvkey interface, so that the sender checks afterwards if what it was sending changed, and the receiver then throws out the bad transfer. That would be less expensive for the reciever, as it would not need to do a checksum verification. But, it would mean another network round trip, and since rsync closes the connection, it would need to open another ssh connection to do this. Even with connction caching, that would add latency to uploads. It would also complicate the interface, especially because an older git-annex-shell would not have the new interface available. For these reasons, I prefer punting on that at this time, and instead someone might set annex.verify=false and be unhappy that it still verifies.. (One other gotcha not dealt with is that a v5 repo could be upgraded to v6 while an upload is in progress, and a file unlocked and modified.) (Also, I double-checked Remote.GCrypt's calls to rsyncParamsRemote, and they're fine. When a file is being uploaded to gcrypt, or any other special repository, it is mediated by sendAnnex, so changes will be detected at that level and the special remote implementation doesn't need to worry about them.)
* add unlocked flag for git-annex-shell recvkeyGravatar Joey Hess2015-12-26
| | | | | | The direct flag is also set when sending unlocked content, to support old versions of git-annex-shell. At some point, the direct flag will be removed, and only the unlocked flag will be used.
* Merge branch 'master' into smudgeGravatar Joey Hess2015-12-11
|\
| * webdav: When testing the WebDAV server, send a file with content. The empty ↵Gravatar Joey Hess2015-12-11
| | | | | | | | file it was sending tickled bugs in some php WebDAV server.
* | check inode cache in prepSendAnnexGravatar Joey Hess2015-12-10
|/ | | | | This does mean one query of the database every time an object is sent. May impact performance.
* tahoe: Include tahoe capabilities in whereis display.Gravatar Joey Hess2015-11-30
|
* improve error message when special remote program cannot be runGravatar Joey Hess2015-11-18
|
* Display progress meter in -J mode when downloading from the web.Gravatar Joey Hess2015-11-16
| | | | | Including in addurl, and get --from web, but also in S3 and External special remotes when a web url is known for content in those remotes.
* refactorGravatar Joey Hess2015-11-16
|
* Display progress meter in -J mode when copying from a local git repo, to a ↵Gravatar Joey Hess2015-11-16
| | | | | | | | | | | | local git repo, and from a remote git repo. Had everything available, just didn't combine the progress meter with the other places progress is sent to update it. (And to a remote repo already did show progress.) Most special remotes should already display progress meters with -J, same as without it. One exception to this is the web, since it relies on wget/curl progress display without -J. Still todo..
* convert from Utility.LockPool to Annex.LockPool everywhereGravatar Joey Hess2015-11-12
|
* concurrent-output, first passGravatar Joey Hess2015-11-04
| | | | | | Output without -Jn should be unchanged from before. With -Jn, concurrent-output is used for messages, but regions are not used yet, so it's a mess.
* Fix failure to build with aws-0.13.0 and finish nearline support.Gravatar Joey Hess2015-11-02
| | | | | | * Fix failure to build with aws-0.13.0. * When built with aws-0.13.0, the S3 special remote can be used to create google nearline buckets, by setting storageclass=NEARLINE.
* Avoid displaying network transport warning when a ssh remote does not yet ↵Gravatar Joey Hess2015-10-15
| | | | | | | | | | | have an annex.uuid set. Instead, only display transport error if the configlist output doesn't include an annex.uuid line, even an empty one. A recent change made git-annex init try to get all the remote uuids, and so the transport error would be displayed by it. It was also displayed when eg, copying files to a remote that had no uuid yet.
* S3: Fix support for using https.Gravatar Joey Hess2015-10-15
| | | | Was using the http-only Manager before, not the tls-capable one.
* fix various build warnings, mostly on WindowsGravatar Joey Hess2015-10-13
| | | | And some when S3 is disabled
* use action, not sideActionGravatar Joey Hess2015-10-11
| | | | | | | | sideAction is for things not generally related to the current action being performed. And, it adds a newline after the side action. This was not the right thing to use for stuff like "checksum", where doing a checksum is part of the git annex get process, and indeed we want it to display "(checksum...) ok"
* add inAnnex check to local lockKeyGravatar Joey Hess2015-10-09
|
* improve display when lockcontent failsGravatar Joey Hess2015-10-09
| | | | | | | | /dev/null stderr; ssh is still able to display a password prompt despite this Show some messages so the user knows it's locking a remote, and knows if that locking failed.
* implement lockContent for ssh remotesGravatar Joey Hess2015-10-09
|
* fix local dropping to not require extra locking of copies, but only that the ↵Gravatar Joey Hess2015-10-09
| | | | local copy be locked for removal
* fix lockKey to run callback in original Annex monad, not local remote'sGravatar Joey Hess2015-10-09
|
* content locking during drop working for local git remotesGravatar Joey Hess2015-10-09
| | | | Only ssh remotes lack locking now
* add removeKey action to RemoteGravatar Joey Hess2015-10-08
| | | | | Not implemented for any remotes yet; probably the git remote is the only one that will ever implement it.
* add lockContentSharedGravatar Joey Hess2015-10-08
| | | | | | | | Also, rename lockContent to lockContentExclusive inAnnexSafe should perhaps be eliminated, and instead use `lockContentShared inAnnex`. However, I'm waiting on that, as there are only 2 call sites for inAnnexSafe and it's fiddly.
* other 80% of avoding verification when hard linking to objects in shared repoGravatar Joey Hess2015-10-02
| | | | | | | | | | | | | | | | | | | | In c3b38fb2a075b4250e867ebd910324c65712c747, it actually only handled uploading objects to a shared repository. To avoid verification when downloading objects from a shared repository, was a lot harder. On the plus side, if the process of downloading a file from a remote is able to verify its content on the side, the remote can indicate this now, and avoid the extra post-download verification. As of yet, I don't have any remotes (except Git) using this ability. Some more work would be needed to support it in special remotes. It would make sense for tahoe to implicitly verify things downloaded from it; as long as you trust your tahoe server (which typically runs locally), there's cryptographic integrity. OTOH, despite bup being based on shas, a bup repo under an attacker's control could have the git ref used for an object changed, and so a bup repo shouldn't implicitly verify. Indeed, tahoe seems unique in being trustworthy enough to implicitly verify.
* avoid verification when hard linking to objects in shared repositoryGravatar Joey Hess2015-10-02
| | | | Such a repository is implicitly trusted, so there's no point.
* Do verification of checksums of annex objects downloaded from remotes.Gravatar Joey Hess2015-10-01
| | | | | | | | | | | | | | | | * When annex objects are received into git repositories, their checksums are verified then too. * To get the old, faster, behavior of not verifying checksums, set annex.verify=false, or remote.<name>.annex-verify=false. * setkey, rekey: These commands also now verify that the provided file matches the key, unless annex.verify=false. * reinject: Already verified content; this can now be disabled by setting annex.verify=false. recvkey and reinject already did verification, so removed now duplicate code from them. fsck still does its own verification, which is ok since it does not use getViaTmp, so verification doesn't happen twice when using fsck --from.
* refactorGravatar Joey Hess2015-10-01
|
* avoid deprecation warnings when built with http-client >= 0.4.18Gravatar Joey Hess2015-10-01
| | | | | | | | | | Since I want git-annex to keep building on debian stable, I need to still support the old http-client, which required explicit calls to closeManager, or use of withManager to get Managers to close at appropriate times. This is not needed in the new version, and so they added a deprecation warning. IMHO much too early, because look at the mess I had to go through to avoid that deprecation warning while supporting both versions..
* avoid hard dependency on new version of awsGravatar Joey Hess2015-09-22
|
* S3 storage classes expansionGravatar Joey Hess2015-09-17
| | | | | | | | | | Added support for storageclass=STANDARD_IA to use Amazon's new Infrequently Accessed storage. Also allows using storageclass=NEARLINE to use Google's NearLine storage. The necessary changes to aws to support this are in https://github.com/aristidb/aws/pull/176
* annex.hardlink extended to also try to use hard links when copying from the ↵Gravatar Joey Hess2015-09-14
| | | | | | | repository to a remote. Also, it used to only check that one of the repos was not in direct mode; now when either repo is direct mode, annex.hardlink won't have an effect.
* support gpg.programGravatar Joey Hess2015-09-09
| | | | | | When gpg.program is configured, it's used to get the command to run for gpg. Useful on systems that have only a gpg2 command or want to use it instead of the gpg command.
* disable whereisKey for encrypted or chunked remotesGravatar Joey Hess2015-08-19
| | | | | | | This only makes sense for public repos, that are not chunked, so that there's a 1:1 from Key in the git-annex repo to file on the remote. Rather than making every remote implementation deal with that, just disable whereisKey when it doesn't make sense.
* make whereis show urls when web remote does not have contentGravatar Joey Hess2015-08-17
| | | | This is needed when external special remotes register an url for a key.
* External special remotes can now be built that can be used in readonly mode, ↵Gravatar Joey Hess2015-08-17
| | | | | | | | | | | | | | | | | where git-annex downloads content from the remote using regular http. Note that, if an url is added to the web log for such a remote, it's not distinguishable from another url that might be added for the web remote. (Because the web log doesn't distinguish which remote owns a plain url. Urls with a downloader set are distinguishable, but we're not using them here.) This seems ok-ish.. In such a case, both remotes will try to use both urls, and both remotes should be able to. The only issue I see is that dropping a file from the web remote will remove both urls in this case. This is not often done, and could even be considered a feature, I suppose.
* export some always failing methods for readonly remotesGravatar Joey Hess2015-08-17
|
* refactorGravatar Joey Hess2015-08-17
|
* refactorGravatar Joey Hess2015-08-17
|
* Added WHEREIS to external special remote protocol.Gravatar Joey Hess2015-08-13
|
* add some debugs to get timingsGravatar Joey Hess2015-08-13
| | | | | | | Note that I had one in Annex.Action.startup too, but it resulted in a weird message printed by ssh, "channel 2: bad ext data". I don't know why, but it only happened when transferinfo was run, so I wonder if 1d71ad072e13c8ed1cb8b34367b57d59e651f0a9 introduced a fragility somehow.