diff options
author | Joey Hess <joeyh@joeyh.name> | 2017-09-20 14:37:20 -0400 |
---|---|---|
committer | Joey Hess <joeyh@joeyh.name> | 2017-09-20 15:29:13 -0400 |
commit | 3826ef1923a35ef8794f0e3beb0f94f0f40fd9be (patch) | |
tree | 0a0af9d00654786c9196f8f13c973e597cfb65fd /Assistant/Sync.hs | |
parent | d20933a25956a3a07247f66fe3a554761d616173 (diff) |
add exporter thread to assistant
This is similar to the pusher thread, but a separate thread because git
pushes can be done in parallel with exports, and updating a big export
should not prevent other git pushes going out in the meantime.
The exportThread only runs at most every 30 seconds, since updating an
export is more expensive than pushing. This may need to be tuned.
Added a separate channel for export commits; the committer records a
commit in that channel.
Also, reconnectRemotes records a dummy commit, to make the exporter
thread wake up and make sure all exports are up-to-date. So,
connecting a drive with a directory special remote export will
immediately update it, and getting online will automatically
update S3 and WebDAV exports.
The transfer queue is not involved in exports. Instead, failed
exports are retried much like failed pushes.
This commit was sponsored by Ewen McNeill.
Diffstat (limited to 'Assistant/Sync.hs')
-rw-r--r-- | Assistant/Sync.hs | 14 |
1 files changed, 9 insertions, 5 deletions
diff --git a/Assistant/Sync.hs b/Assistant/Sync.hs index aba90f64c..c6460e9ed 100644 --- a/Assistant/Sync.hs +++ b/Assistant/Sync.hs @@ -33,6 +33,7 @@ import Assistant.Threads.Watcher (watchThread, WatcherControl(..)) import Assistant.TransferSlots import Assistant.TransferQueue import Assistant.RepoProblem +import Assistant.Commits import Types.Transfer import Data.Time.Clock @@ -48,10 +49,10 @@ import Control.Concurrent - it's sufficient to requeue failed transfers. - - Also handles signaling any connectRemoteNotifiers, after the syncing is - - done. + - done, and records an export commit to make any exports be updated. -} reconnectRemotes :: [Remote] -> Assistant () -reconnectRemotes [] = noop +reconnectRemotes [] = recordExportCommit reconnectRemotes rs = void $ do rs' <- liftIO $ filterM (Remote.checkAvailable True) rs unless (null rs') $ do @@ -60,6 +61,7 @@ reconnectRemotes rs = void $ do whenM (liftIO $ Remote.checkAvailable False r) $ repoHasProblem (Remote.uuid r) (syncRemote r) mapM_ signal $ filter (`notElem` failedrs) rs' + recordExportCommit where gitremotes = filter (notspecialremote . Remote.repo) rs (_xmppremotes, nonxmppremotes) = partition Remote.isXMPPRemote rs @@ -143,9 +145,11 @@ pushToRemotes' now remotes = do then retry currbranch g u failed else fallback branch g u failed - updatemap succeeded failed = changeFailedPushMap $ \m -> - M.union (makemap failed) $ - M.difference m (makemap succeeded) + updatemap succeeded failed = do + v <- getAssistant failedPushMap + changeFailedPushMap v $ \m -> + M.union (makemap failed) $ + M.difference m (makemap succeeded) makemap l = M.fromList $ zip l (repeat now) retry currbranch g u rs = do |