aboutsummaryrefslogtreecommitdiff
path: root/doc
Commit message (Collapse)AuthorAge
* stop warning about removals from IAGravatar Joey Hess2017-09-12
| | | | | | | In a test, I uploaded a pdf, and several files were derived from it. After removing the pdf, the derived files went away after approximatly half an hour. This window does not seem worth warning about every time. Documented it in the tip.
* S3: Allow removing files from IA, but warn about derived versions ↵Gravatar Joey Hess2017-09-12
| | | | | | | | | | | | | potentially still existing there. Removal works, only derives are a potential issue, so allow removing with a warning. This way, unexporting a file works, and behavior is consistent with IA remotes whether or not exporttree=yes. Also tested exporting filenames containing unicode, spaces, underscores. All worked, despite the IA's faq saying it doesn't. This commit was sponsored by Trenton Cronholm on Patreon.
* Added a comment: Issue also affects Samsung devices, git unaffectedGravatar https://christian.amsuess.com/chrysn2017-09-11
|
* Added a comment: The initremote command appears to hang due to low entropyGravatar NathanCollins2017-09-10
|
* (no commit message)Gravatar ilovezfs2017-09-09
|
* closeGravatar Joey Hess2017-09-09
|
* (no commit message)Gravatar ilovezfs2017-09-09
|
* very minor typoGravatar yarikoptic2017-09-08
|
* very delayed response now that feature is addedGravatar Joey Hess2017-09-08
|
* consistencyGravatar Joey Hess2017-09-08
|
* devblogGravatar Joey Hess2017-09-08
|
* Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-08
|\
* | S3 export finalizationGravatar Joey Hess2017-09-08
| | | | | | | | Fixed ACL issue, and updated some documentation.
* | S3 export (untested)Gravatar Joey Hess2017-09-08
| | | | | | | | | | | | | | | | | | | | | | It opens a http connection per file exported, but then so does git annex copy --to s3. Decided not to munge exported filenames for IA. Too large a chance of the munging having confusing results. Instead, export of files not supported by IA, eg with spaces in their name, will fail. This commit was supported by the NSF-funded DataLad project.
* | External special remote protocol extended to support export.Gravatar Joey Hess2017-09-08
| | | | | | | | | | | | Also updated example.sh to support export. This commit was supported by the NSF-funded DataLad project.
| * Added a commentGravatar karel-de-macil2017-09-08
|/
* devblogGravatar Joey Hess2017-09-07
|
* mention git-annex exportGravatar Joey Hess2017-09-07
|
* updateGravatar Joey Hess2017-09-07
|
* correctionGravatar Joey Hess2017-09-07
|
* commentGravatar Joey Hess2017-09-07
|
* Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-07
|\
* \ Merge branch 'export'Gravatar Joey Hess2017-09-07
|\ \
| * | avoid renaming to temp files before deletingGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Only rename when actually ncessary. The diff gets buffered in memory. Probably git has to buffer a diff in memory when generating it as well, so this memory usage should not be a problem, even when the diff is very large. I hope. This commit was supported by the NSF-funded DataLad project.
| * | prevent exporttree=yes on remotes that don't support exportsGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | Don't allow "exporttree=yes" to be set when the special remote does not support exports. That would be confusing since the user would set up a special remote for exports, but `git annex export` to it would later fail. This commit was supported by the NSF-funded DataLad project.
| * | document new stuff for external special remotesGravatar Joey Hess2017-09-07
| | | | | | | | | | | | | | | | | | | | | Got rid of RENAMEEXPORT-UNSUPPORTED, no reason not to use RENAMEEXPORT-FAILURE for that. This commit was supported by the NSF-funded DataLad project.
| | * (no commit message)Gravatar Horus2017-09-07
| | |
| | * Added a commentGravatar Horus2017-09-07
| | |
| | * Added a commentGravatar anthony@ad39673d230d75cbfd19d2757d754030049c76732017-09-06
| |/ |/|
* | devblogGravatar Joey Hess2017-09-06
| |
| * fix consistency bug reading from export databaseGravatar Joey Hess2017-09-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The export database has writes made to it and then expects to read back the same data immediately. But, the way that Database.Handle does writes, in order to support multiple writers, makes that not work, due to caching issues. This resulted in export re-uploading files it had already successfully renamed into place. Fixed by allowing databases to be opened in MultiWriter or SingleWriter mode. The export database only needs to support a single writer; it does not make sense for multiple exports to run at the same time to the same special remote. All other databases still use MultiWriter mode. And by inspection, nothing else in git-annex seems to be relying on being able to immediately query for changes that were just written to the database. This commit was supported by the NSF-funded DataLad project.
| * Merge branch 'master' into exportGravatar Joey Hess2017-09-06
| |\ | |/ |/|
| * todoGravatar Joey Hess2017-09-06
| |
| * export file renamingGravatar Joey Hess2017-09-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is seriously super hairy. It has to handle interrupted exports, which may be resumed with the same or a different tree. It also has to recover from export conflicts, which could cause the wrong content to be renamed to a file. I think this works, or is close to working. See the update to the design for how it works. This is definitely not optimal, in that it does more renames than are necessary. It would probably be worth finding the keys that are really renamed and only renaming those. But let's get the "simple" approach to work first.. This commit was supported by the NSF-funded DataLad project.
| * record incomplete exports in export.logGravatar Joey Hess2017-09-06
| | | | | | | | | | | | | | | | | | | | Not yet used, but essential for resuming cleanly. Note that, in normmal operation, only one commit is made to export.log during an export; the incomplete version only gets to the journal and is then overwritten. This commit was supported by the NSF-funded DataLad project.
| * thoughts on handling renames efficientlyGravatar Joey Hess2017-09-06
| | | | | | | | | | | | This gets complicated, but I think this design will work! This commit was supported by the NSF-funded DataLad project.
* | Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-06
|\ \
* | | move line break to fix broken linkGravatar Edward Betts2017-09-06
| | |
* | | updateGravatar Joey Hess2017-09-06
| | |
| * | (no commit message)Gravatar karel-de-macil2017-09-06
|/ /
* | Added datalad "super-dataset".Gravatar yarikoptic2017-09-05
| |
* | removedGravatar EskildHustvedt2017-09-05
| |
* | Added a comment: Partial exportsGravatar EskildHustvedt2017-09-05
| |
* | Added a comment: Partial exportsGravatar EskildHustvedt2017-09-05
| |
* | (no commit message)Gravatar eacousineau2017-09-05
| |
* | Merge branch 'master' of ssh://git-annex.branchable.comGravatar Joey Hess2017-09-04
|\ \
* | | devblogGravatar Joey Hess2017-09-04
| | |
| | * git annex get from exportsGravatar Joey Hess2017-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Straightforward enough, except for the needed belt-and-suspenders sanity checks to avoid foot shooting due to exports not being key/value stores. * Even when annex.verify=false, always verify from exports. * Only get files from exports that use a backend that supports checksum verification. * Never trust exports, even if the user says to, because then `git annex drop` would drop content if the export seemed to contain a copy. This commit was supported by the NSF-funded DataLad project.
| | * use export db to correctly handle duplicate filesGravatar Joey Hess2017-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removed uncorrect UniqueKey key in db schema; a key can appear multiple times with different files. The database has to be flushed after each removal. But when adding files to the export, lots of changes are able to be queued up w/o flushing. So it's still fairly efficient. If large removals of files from exports are too slow, an alternative would be to make two passes over the diff, one pass queueing deletions from the database, then a flush and the a second pass updating the location log. But that would use more memory, and need to look up exportKey twice per removed file, so I've avoided such optimisation yet. This commit was supported by the NSF-funded DataLad project.
| | * implement exporttree=yes configurationGravatar Joey Hess2017-09-04
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Only export to remotes that were initialized to support it. * Prevent storing key/value on export remotes. * Prevent enabling exporttree=yes and encryption in the same remote. SetupStage Enable was changed to take the old RemoteConfig. This allowed only setting exporttree when initially setting up a remote, and not configuring it later after stuff might already be stored in the remote. Went with =yes rather than =true for consistency with other parts of git-annex. Changed docs accordingly. This commit was supported by the NSF-funded DataLad project.
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-19 23:16:24 +0000