diff options
| -rw-r--r-- | Assistant/Threads/Transferrer.hs | 44 | ||||
| -rw-r--r-- | debian/changelog | 1 | ||||
| -rw-r--r-- | doc/bugs/smarter_flood_filling.mdwn | 2 |
3 files changed, 35 insertions, 12 deletions
diff --git a/Assistant/Threads/Transferrer.hs b/Assistant/Threads/Transferrer.hs index e75b5acae..63306831f 100644 --- a/Assistant/Threads/Transferrer.hs +++ b/Assistant/Threads/Transferrer.hs @@ -21,6 +21,7 @@ import qualified Remote import Types.Key import Locations.UserConfig import Assistant.Threads.TransferWatcher +import Annex.Wanted import System.Process (create_group) @@ -103,21 +104,40 @@ startTransfer program t info = case (transferRemote info, associatedFile info) o , File file ] -{- Checks if the file to download is already present, or the remote - - being uploaded to isn't known to have the file. -} +{- Called right before a transfer begins, this is a last chance to avoid + - unnecessary transfers. + - + - For downloads, we obviously don't need to download if the already + - have the object. + - + - Smilarly, for uploads, check if the remote is known to already have + - the object. + - + - Also, uploads get queued to all remotes, in order of cost. + - This may mean, for example, that an object is uploaded over the LAN + - to a locally paired client, and once that upload is done, a more + - expensive transfer remote no longer wants the object. (Since + - all the clients have it already.) So do one last check if this is still + - preferred content. + - + - We'll also do one last preferred content check for downloads. An + - example of a case where this could be needed is if a download is queued + - for a file that gets moved out of an archive directory -- but before + - that download can happen, the file is put back in the archive. + -} shouldTransfer :: Transfer -> TransferInfo -> Annex Bool shouldTransfer t info | transferDirection t == Download = - not <$> inAnnex key - | transferDirection t == Upload = - {- Trust the location log to check if the - - remote already has the key. This avoids - - a roundtrip to the remote. -} - case transferRemote info of - Nothing -> return False - Just remote -> - notElem (Remote.uuid remote) - <$> loggedLocations key + (not <$> inAnnex key) <&&> wantGet True file + | transferDirection t == Upload = case transferRemote info of + Nothing -> return False + Just r -> notinremote r + <&&> wantSend True file (Remote.uuid r) | otherwise = return False where key = transferKey t + file = associatedFile info + + {- Trust the location log to check if the remote already has + - the key. This avoids a roundtrip to the remote. -} + notinremote r = notElem (Remote.uuid r) <$> loggedLocations key diff --git a/debian/changelog b/debian/changelog index 4bd239b4c..3b2723478 100644 --- a/debian/changelog +++ b/debian/changelog @@ -48,6 +48,7 @@ git-annex (4.20130228) UNRELEASED; urgency=low combines all their changes into a single commit. * assistant: Fix ~/.ssh/git-annex-shell wrapper to work when the ssh key does not force a command. + * assistant: Be smarter about avoiding unncessary transfers. * webapp: DTRT when told to create a git repo that already exists. * webapp: Work around bug in Warp's slowloris attack prevention code, diff --git a/doc/bugs/smarter_flood_filling.mdwn b/doc/bugs/smarter_flood_filling.mdwn index cfff3698f..a2a4f2ce5 100644 --- a/doc/bugs/smarter_flood_filling.mdwn +++ b/doc/bugs/smarter_flood_filling.mdwn @@ -12,7 +12,9 @@ this. 1. Ensure that locally paired computers have a lower cost than network transfer remotes. 2. When queuing uploads, queue transfers to the lowest cost remotes first. + (already done) 3. Just before starting a transfer, re-check if the transfer is still wanted. + (done) Now, unnecessary transfers to tranfer repos are avoided if it can send the file locally instead. |