[[!comment format=mdwn username="joey" subject="""comment 4""" date="2017-07-10T17:30:23Z" content=""" Yoh suggested storing a treeish associated with the export to a special remote. Pointed out that the diff from that treeish to the next one could be used to update the export. (That does seem very close to [[todo/dumb, unsafe, human-readable_backend]].) Would need to somehow deal with partial uploads. There are two ways an upload can be partial: * Some of the files in the treeish have been uploaded; others have not. * A file has been partially uploaded. These two cases need to be disentangled somehow in order to handle them. It could use the location log, so once a key gets uploaded to the special remote, its content is marked present. However, using the location log does not seem sufficient to handle all cases (eg two files swapping names between two treeishes, where one of the files has been uploaded only partially to the special remote). It seems promosing to keep track of two separate treeishes: 1. The treeish that is the current goal to have exported to the special remote. 2. The treeish for the current state of the special remote. Note that after even an interrupted transfer, this treeish needs to be updated to contain the current state of the special remote, which would mean constructing a new treeish. (Perhaps a per-remote index file could be used.) Having these two treeishes allows: * Detecting renames of exported files, which some special remotes can do more efficiently. * Determining the key that a given file on the special remote is storing the content of. * Resuming an interrupted export, without re-uploading all the files. * Detecting a partially uploaded file, because the current state treeish for the remote should not contain that file. * Knowing what key was in the process of being sent to a partially uploaded file, and so resuming that upload. Look at the goal treeish and see what key it has for the file; as long as the goal treeish is the same goal that was used for the interrupted export, that's the key. (This needs a way to track if the goal has changed.) * Optionally, making `git annex sync` and the assistant upload as needed to satisfy goal treeishes for special remotes. """]]