From 049bea4659e108967977852b5ca0cc00b74a8831 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 20 Jan 2014 17:34:58 -0400 Subject: Add and use numcopiesneeded preferred content expression. * Add numcopiesneeded preferred content expression. * Client, transfer, incremental backup, and archive repositories now want to get content that does not yet have enough copies. This means the asssistant will make copies of files that don't yet meet the configured numcopies, even to places that would not normally want the file. For example, if numcopies is 4, and there are 2 client repos and 2 transfer repos, and 2 removable backup drives, the file will be sent to both transfer repos in order to make 4 copies. Once a removable drive get a copy of the file, it will be dropped from one transfer repo or the other (but not both). Another example, numcopies is 3 and there is a client that has a backup removable drive and two small archive repos. Normally once one of the small archives has a file, it will not be put into the other one. But, to satisfy numcopies, the assistant will duplicate it into the other small archive too, if the backup repo is not available to receive the file. I notice that these examples are fairly unlikely setups .. the old behavior was not too bad, but it's nice to finally have it really correct. .. Almost. I have skipped checking the annex.numcopies .gitattributes out of fear it will be too slow. This commit was sponsored by Florian Schlegel. --- Limit.hs | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) (limited to 'Limit.hs') diff --git a/Limit.hs b/Limit.hs index fa6fa1f41..c0d32c68e 100644 --- a/Limit.hs +++ b/Limit.hs @@ -1,6 +1,6 @@ {- user-specified limits on files to act on - - - Copyright 2011-2013 Joey Hess + - Copyright 2011-2014 Joey Hess - - Licensed under the GNU GPL version 3 or higher. -} @@ -23,6 +23,7 @@ import qualified Backend import Annex.Content import Annex.UUID import Logs.Trust +import Logs.NumCopies import Types.TrustLevel import Types.Key import Types.Group @@ -177,6 +178,30 @@ limitCopies want = case split ":" want of | "+" `isSuffixOf` s = (>=) <$> readTrustLevel (beginning s) | otherwise = (==) <$> readTrustLevel s +{- Adds a limit to match files that need more copies made. + - + - Does not look at annex.numcopies .gitattributes, because that + - would require querying git check-attr every time a preferred content + - expression is checked, which would probably be quite slow. + -} +addNumCopiesNeeded :: String -> Annex () +addNumCopiesNeeded = addLimit . limitNumCopiesNeeded + +limitNumCopiesNeeded :: MkLimit +limitNumCopiesNeeded want = case readish want of + Just needed -> Right $ \notpresent -> checkKey $ + handle needed notpresent + Nothing -> Left "bad value for numcopiesneeded" + where + handle needed notpresent key = do + gv <- getGlobalNumCopies + case gv of + Nothing -> return False + Just numcopies -> do + us <- filter (`S.notMember` notpresent) + <$> (trustExclude UnTrusted =<< Remote.keyLocations key) + return $ numcopies - length us >= needed + {- Adds a limit to skip files not believed to be present in all - repositories in the specified group. -} addInAllGroup :: String -> Annex () -- cgit v1.2.3