summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* fix compaction of export.logGravatar Joey Hess2017-09-12
| | | | | | | | It was not getting old lines removed, because the tree graft confused the updater, so it union merged from the previous git-annex branch, which still contained the old lines. Fixed by carefully using setIndexSha. This commit was supported by the NSF-funded DataLad project.
* devblogGravatar Joey Hess2017-09-12
|
* change export.log format to support multiple export remotesGravatar Joey Hess2017-09-12
| | | | | | | This breaks backwards compatibility, but only with unreleased versions of git-annex, which I think is acceptable. This commit was supported by the NSF-funded DataLad project.
* remove debug printGravatar Joey Hess2017-09-12
|
* bugGravatar Joey Hess2017-09-12
|
* export: cache connections for S3 and webdavGravatar Joey Hess2017-09-12
|
* more box.com strangenessGravatar Joey Hess2017-09-12
|
* webdav: Fix lack of url-escaping of filenames.Gravatar Joey Hess2017-09-12
| | | | | | | | | | | | | inDAVLocation does not url-escape, and so exporting a filename with spaces to box.com at least resulted in a error 400. It might also have affected storing keys on a webdav remote, if the key contained a space or other problem character. Pretty unlikely. I emailed Clint about the inDAVLocation gotcha, but seems best to fix it here. This commit was supported by the NSF-funded DataLad project.
* document box.com rename problemGravatar Joey Hess2017-09-12
|
* deal with box.com horrible infinite redirect behaviorGravatar Joey Hess2017-09-12
| | | | | | | | | | | | | | | | | | | webdav: Checking if a non-existent file is present on Box.com triggered a bug in its webdav support that generates an infinite series of redirects. It seems to redirect foo to foo/ to foo/index.php to foo/index.php/index.php ... Why a webdav endpoint would behave this way who knows. Deal with such problems by assuming such behavior means the file is not present. Can't simply disable following redirects, because the webdav endpoint could legitimately be redirected to a new endpoint. So, when this happens 10 redirects have to be followed, before it gives up and assumes this means the file does not exist. This commit was supported by the NSF-funded DataLad project.
* leave export logged as incomplete if initial renames failGravatar Joey Hess2017-09-12
| | | | | | | | | | This way, the temp files that might be left due to failure will be cleaned up next time. Also, nub the list of incomplete exports to avoid repeatedly adding the same tree to it when running export repeatedly when it's failing. This commit was supported by the NSF-funded DataLad project.
* export to webdavGravatar Joey Hess2017-09-12
| | | | | | | | | | | This basically works, but there's a bug when renaming a file that leaves a .git-annex-temp-content-key file in the webdav store, that never gets cleaned up. Also, exporting files with spaces to box.com seems to fail; perhaps it does not support it? This commit was supported by the NSF-funded DataLad project.
* update copyright yearGravatar Joey Hess2017-09-12
|
* propigate exception in checkPresentExportS3Gravatar Joey Hess2017-09-12
| | | | checkPresentExport is supposed to throw exceptions
* stop warning about removals from IAGravatar Joey Hess2017-09-12
| | | | | | | In a test, I uploaded a pdf, and several files were derived from it. After removing the pdf, the derived files went away after approximatly half an hour. This window does not seem worth warning about every time. Documented it in the tip.
* avoid showing error when copy failsGravatar Joey Hess2017-09-12
| | | | | | | | | | | | | Since renameExport is allowed to fail for any reason, and its failure is always recovered from by doing a new upload and deleting the old content, this avoids unnecessary noise. Copying a file on the IA failed, apparently something wrong with their emulation of S3: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "InvalidArgument", s3ErrorMessage = "Invalid Argument", s3ErrorResource = Just "x-(amz|archive)-copy-source header is bad: 'joeyh-public-test2/foo'", s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing, s3ErrorBucket = Nothing, s3ErrorEndpointRaw = Nothing, s3ErrorEndpoint = Nothing} This commit was sponsored by Jake Vosloo on Patreon.
* S3: Allow removing files from IA, but warn about derived versions ↵Gravatar Joey Hess2017-09-12
| | | | | | | | | | | | | potentially still existing there. Removal works, only derives are a potential issue, so allow removing with a warning. This way, unexporting a file works, and behavior is consistent with IA remotes whether or not exporttree=yes. Also tested exporting filenames containing unicode, spaces, underscores. All worked, despite the IA's faq saying it doesn't. This commit was sponsored by Trenton Cronholm on Patreon.
* Added a comment: Issue also affects Samsung devices, git unaffectedGravatar https://christian.amsuess.com/chrysn2017-09-11
|
* Added a comment: The initremote command appears to hang due to low entropyGravatar NathanCollins2017-09-10
|
* (no commit message)Gravatar ilovezfs2017-09-09
|
* closeGravatar Joey Hess2017-09-09
|
* (no commit message)Gravatar ilovezfs2017-09-09
|
* very minor typoGravatar yarikoptic2017-09-08
|
* very delayed response now that feature is addedGravatar Joey Hess2017-09-08
|
* don't show key urls in whereis for S3 with public=yes and exporttree=yesGravatar Joey Hess2017-09-08
|
* consistencyGravatar Joey Hess2017-09-08
|
* devblogGravatar Joey Hess2017-09-08
|
* Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-08
|\
* | S3 export finalizationGravatar Joey Hess2017-09-08
| | | | | | | | Fixed ACL issue, and updated some documentation.
* | S3 export (untested)Gravatar Joey Hess2017-09-08
| | | | | | | | | | | | | | | | | | | | | | It opens a http connection per file exported, but then so does git annex copy --to s3. Decided not to munge exported filenames for IA. Too large a chance of the munging having confusing results. Instead, export of files not supported by IA, eg with spaces in their name, will fail. This commit was supported by the NSF-funded DataLad project.
* | External special remote protocol extended to support export.Gravatar Joey Hess2017-09-08
| | | | | | | | | | | | Also updated example.sh to support export. This commit was supported by the NSF-funded DataLad project.
| * Added a commentGravatar karel-de-macil2017-09-08
|/
* devblogGravatar Joey Hess2017-09-07
|
* mention git-annex exportGravatar Joey Hess2017-09-07
|
* updateGravatar Joey Hess2017-09-07
|
* correctionGravatar Joey Hess2017-09-07
|
* commentGravatar Joey Hess2017-09-07
|
* Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-07
|\
* \ Merge branch 'export'Gravatar Joey Hess2017-09-07
|\ \
| * | interrupted export recovery bugfixesGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When an export was interrupted, the sqlite database won't have been committed necessarily. Also, the interrupted export might have been run in an entirely different repository. There's not a significant speed benefit in checking getExportLocation in this case anyway, so avoid it. Also, remove the old filename from the export database. Recovery from interrupted exports is now tested working. This commit was supported by the NSF-funded DataLad project.
| * | avoid renaming to temp files before deletingGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Only rename when actually ncessary. The diff gets buffered in memory. Probably git has to buffer a diff in memory when generating it as well, so this memory usage should not be a problem, even when the diff is very large. I hope. This commit was supported by the NSF-funded DataLad project.
| * | prevent exporttree=yes on remotes that don't support exportsGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | Don't allow "exporttree=yes" to be set when the special remote does not support exports. That would be confusing since the user would set up a special remote for exports, but `git annex export` to it would later fail. This commit was supported by the NSF-funded DataLad project.
| * | document new stuff for external special remotesGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | Got rid of RENAMEEXPORT-UNSUPPORTED, no reason not to use RENAMEEXPORT-FAILURE for that. This commit was supported by the NSF-funded DataLad project.
| | * (no commit message)Gravatar Horus2017-09-07
| | |
| | * Added a commentGravatar Horus2017-09-07
| | |
| | * Added a commentGravatar anthony@ad39673d230d75cbfd19d2757d754030049c76732017-09-06
| |/ |/|
* | devblogGravatar Joey Hess2017-09-06
| |
| * fix consistency bug reading from export databaseGravatar Joey Hess2017-09-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The export database has writes made to it and then expects to read back the same data immediately. But, the way that Database.Handle does writes, in order to support multiple writers, makes that not work, due to caching issues. This resulted in export re-uploading files it had already successfully renamed into place. Fixed by allowing databases to be opened in MultiWriter or SingleWriter mode. The export database only needs to support a single writer; it does not make sense for multiple exports to run at the same time to the same special remote. All other databases still use MultiWriter mode. And by inspection, nothing else in git-annex seems to be relying on being able to immediately query for changes that were just written to the database. This commit was supported by the NSF-funded DataLad project.
| * bugfixGravatar Joey Hess2017-09-06
| |
| * Merge branch 'master' into exportGravatar Joey Hess2017-09-06
| |\ | |/ |/|
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-01 18:46:43 +0000