diff options
author | 2017-09-04 14:33:09 -0400 | |
---|---|---|
committer | 2017-09-04 14:39:32 -0400 | |
commit | 25ed1e54abcc25f729fed016ec77a8cd049142fa (patch) | |
tree | f6130202fc2f23a6b641f8646705c900b757a012 /doc/design | |
parent | 3db6fd7621d3fa57bebfea216019e522f6d8d781 (diff) |
use export db to correctly handle duplicate files
Removed uncorrect UniqueKey key in db schema; a key can appear multiple
times with different files.
The database has to be flushed after each removal. But when adding files
to the export, lots of changes are able to be queued up w/o flushing.
So it's still fairly efficient.
If large removals of files from exports are too slow, an alternative
would be to make two passes over the diff, one pass queueing deletions
from the database, then a flush and the a second pass updating the
location log. But that would use more memory, and need to look up
exportKey twice per removed file, so I've avoided such optimisation yet.
This commit was supported by the NSF-funded DataLad project.
Diffstat (limited to 'doc/design')
-rw-r--r-- | doc/design/exporting_trees_to_special_remotes.mdwn | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/doc/design/exporting_trees_to_special_remotes.mdwn b/doc/design/exporting_trees_to_special_remotes.mdwn index 118f12978..7ff1df870 100644 --- a/doc/design/exporting_trees_to_special_remotes.mdwn +++ b/doc/design/exporting_trees_to_special_remotes.mdwn @@ -147,7 +147,8 @@ remotes, don't let it be turned off. The same file contents may be in a treeish multiple times under different filenames. That complicates using location tracking. One file may have been exported and the other not, and location tracking says that the content -is present in the export. +is present in the export. A sqlite database is needed to keep track of +this. ## recording exported filenames in git-annex branch |