aboutsummaryrefslogtreecommitdiff
path: root/Assistant
Commit message (Collapse)AuthorAge
* let's put type modules under the parent module, not in a Types directoryGravatar Joey Hess2013-03-10
|
* assistant: generate better commits for renamesGravatar Joey Hess2013-03-10
|
* tweakGravatar Joey Hess2013-03-10
|
* avoid ugly alert caused by trying to push to unavailable removable driveGravatar Joey Hess2013-03-10
|
* moved transfer queueing out of watcher and into committerGravatar Joey Hess2013-03-10
| | | | | | | | | | | | | This cleaned up the code quite a bit; now the committer just looks at the Change to see if it's a change that needs to have a transfer queued for it. If I later want to add dropping keys for files that were removed, or something like that, this should make it straightforward. This also fixes a bug. In direct mode, moving a file out of an archive directory failed to start a transfer to get its content. The problem was that the file had not been committed to git yet, and so the transfer code didn't want to touch it, since fileKey failed to get its key. Only starting transfers after a commit avoids this problem.
* assistant: Sync with all git remotes on startup.Gravatar Joey Hess2013-03-08
|
* logic errorGravatar Joey Hess2013-03-07
|
* avoid queuing transfers that are already queuedGravatar Joey Hess2013-03-07
| | | | | | | | I saw this happen in real life, when syncing to a newly added usb drive. I think it got scanned twice, and files were doubled in the queue. This could be optimised a little bit more, to only read from the mvar once, rather than twice.
* empty buddy list when client is connectingGravatar Joey Hess2013-03-07
| | | | | | | | This is not perfect, because on loss of connection, we do not currently immediately detect it and stop the client. It has to time out, and then the buddy list will clear. The NetWatcher should detect disconnects too..
* remove xaGravatar Joey Hess2013-03-07
| | | | | | I have a theory that some google xmpp servers don't send prsense for xa clients, while others do. Seeing some weird lack of presence messages sometimes there.
* add alerts while performing xmpp git push/pullGravatar Joey Hess2013-03-07
|
* assistant: Fix syncing after XMPP pairing.Gravatar Joey Hess2013-03-06
| | | | | A missing uuid broke this, and also caused a UI glitch in the webapp's remote list.
* show when not connected to xmpp serverGravatar Joey Hess2013-03-06
|
* assistant: XMPP git pull and push requests are cached and sent when presence ↵Gravatar Joey Hess2013-03-06
| | | | | | | | | | | | | | | of a new client is detected. Noticed that, At startup or network reconnect, git push messages were sent, often before presence info has been gathered, so were not sent to any buddies. To fix this, keep track of which buddies have seen such messages, and when new presence is received from a buddy that has not yet seen it, resend. This is done only for push initiation messages, so very little data needs to be stored.
* todoGravatar Joey Hess2013-03-06
|
* better xmpp debuggingGravatar Joey Hess2013-03-06
|
* avoid false alert about syncing with xmpp remoteGravatar Joey Hess2013-03-06
|
* avoid too early scanning for xmppremotesGravatar Joey Hess2013-03-06
|
* tag xmpp pushes with jidGravatar Joey Hess2013-03-06
| | | | | | | | | | This fixes the issue mentioned in the last commit. Turns out just collecting UUID of clients behind a XMPP remote is insufficient (although I should probably still do it for other reasons), because a single remote repo might be connected via both XMPP and local pairing. So a way is needed to know when a push was received from any client using a given XMPP remote over XMPP, as opposed to via ssh.
* assistant: Get back in sync with XMPP remotes after network reconnection, ↵Gravatar Joey Hess2013-03-06
| | | | | | | | | | | | | | | | | | | | | and on startup. Make manualPull send push requests over XMPP. When reconnecting with remotes, those that are XMPP remotes cannot immediately be pulled from and scanned, so instead maintain a set of (probably) desynced remotes, and put XMPP remotes on it. (This set could be used in other ways later, if we can detect we're out of sync with other types of remotes.) The merger handles detecting when a XMPP push is received from a desynced remote, and triggers a scan then, if they have in fact diverged. This has one known bug: A single XMPP remote can have multiple clients behind it. When this happens, only the UUID of one client is recorded as the UUID of the XMPP remote. Pushes from the other XMPP clients will not trigger a scan. If the client whose UUID is expected responds to the push request, it'll work, but when that client is offline, we're SOL.
* fix pair complete alert combiningGravatar Joey Hess2013-03-05
| | | | | The pairing complete alert had been conbining with some other alert, fixed this and now it's displayed once xmpp pairing is complete on both sides.
* two types of byNameGravatar Joey Hess2013-03-05
| | | | | | | | Clean up from 5123a1a83aa3b954fe67629508bab5ccea0e4148. In some cases, looking up a remote by name even though it has no UUID is desirable. This includes git annex sync, which can operate on remotes without an annex, and XMPP pairing, which runs addRemote (with calls byName) before the UUID of the XMPP remote has been configured in git.
* better liftAnnex, avoid using runAnnex undefinedGravatar Joey Hess2013-03-04
|
* fix another potential race with the watcher and direct modeGravatar Joey Hess2013-03-04
| | | | | | | | | Watcher wants to rewrite symlink to fix it. But in direct mode, the symlink could be replaced at any time with file content that has finished being transferred by some other process. So, just don't touch it. FWIW, I audited the rest of the assistant for places where it removes files, and the rest is ok. I have not audited the rest of git-annex.
* fixed the race breaking moving files from archive in direct modeGravatar Joey Hess2013-03-04
| | | | | | | | | | | | | | | | | | | | | | assistant: Fix bug in direct mode that could occur when a symlink is moved out of an archive directory, and resulted in the file not being set to direct mode when it was transferred. The bug was that the direct mode mapping was not up-to-date when the transferrer finished. So, finding no direct mode place to store the object, it was put into .git/annex in indirect mode. To fix this, just make the watcher update the direct mode mapping to include the new file before it starts the transfer. (Seems we don't need to update it to remove the old file if the link was moved, because the direct mode code will notice it's not present and the mapping gets updated for its removal later.) The reason this was a race, and was probably not seen often is because the committer came along and updated the direct mode mapping as part of adding the moved symlink. But when the file was sufficiently small or the remote sufficiently fast, this could happen after the transfer finished.
* webapp: New preferences page allows enabling/disabling debug logging at ↵Gravatar Joey Hess2013-03-03
| | | | runtime, as well as configuring numcopies and diskreserve.
* fixGravatar Joey Hess2013-03-03
|
* better prevention of auto repackGravatar Joey Hess2013-03-03
| | | | | | | | | | | | | Looking through the git sources (documentation is unclear), it seems commit doesn't ever trigger git-gc, mostly fetching and merging seems to. I cannot easily override the setting in all those places, so instead set gc.auto in git config when initializing a repository with the assistant. This does mean that the user cannot set gc.auto=0 and completely avoid repacks, as the assistant does it daily. But, it only does it after there are 100x the default number of loose objects, so this is probably not going to be too annoying.
* assistant: Prevent automatic commits from causing git-gc runs, as that can ↵Gravatar Joey Hess2013-03-03
| | | | make things quite slow. Instead, git-gc --auto is run once a day. (This can be disabled by the usual gc.auto=0 setting.)
* git subcommand cleanupGravatar Joey Hess2013-03-03
| | | | | | Pass subcommand as a regular param, which allows passing git parameters like -c before it. This was already done in the pipeing set of functions, but not the command running set.
* typoGravatar Joey Hess2013-03-01
|
* remove excess log rotation; openLog rotatesGravatar Joey Hess2013-03-01
|
* assistant: Fix dropping content when a file is moved to an archive directory.Gravatar Joey Hess2013-03-01
| | | | | | | A transfer is queued, but if the file has already been transferred to the remote before, the transfer is skipped. In this case, it needs to perform any actions it would normally take after finishing the transfer, like dropping the local object.
* assistant: Avoid noise in logs from git commit about typechanged files in ↵Gravatar Joey Hess2013-03-01
| | | | direct mode repositories.
* add additional debug info about reasons for dropsGravatar Joey Hess2013-03-01
|
* add additional debug info about reasons for transfersGravatar Joey Hess2013-03-01
|
* assistant: Logs are rotated to avoid them using too much disk space.Gravatar Joey Hess2013-03-01
| | | | | | This cannot completely guard against a runaway log event, and only runs every hour anyway, but it should avoid most problems with very long-running, active assistants using up too much space.
* no template haskell here any longerGravatar Joey Hess2013-02-28
|
* Makefile now builds using cabal, taking advantage of cabal's automatic ↵Gravatar Joey Hess2013-02-27
| | | | | | | | | detection of appropriate build flags. The only thing lost is ./ghci Speed: make fast used to take 20 seconds here, when rebuilding from touching Command/Unused.hs. With cabal, it's 29 seconds.
* webapp: Display any error message from git init if it fails to create a ↵Gravatar Joey Hess2013-02-26
| | | | repository.
* fix inverted logicGravatar Joey Hess2013-02-22
|
* remove debug printsGravatar Joey Hess2013-02-19
|
* fully support core.symlinks=false in all relevant symlink handling codeGravatar Joey Hess2013-02-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored annex link code into nice clean new library. Audited and dealt with calls to createSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ createSymbolicLink linktarget file only when core.symlinks=true Assistant/WebApp/Configurators/Local.hs: createSymbolicLink link link test if symlinks can be made Command/Fix.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/FromKey.hs: liftIO $ createSymbolicLink link file command only works in indirect mode Command/Indirect.hs: liftIO $ createSymbolicLink l f refuses to run if core.symlinks=false Init.hs: createSymbolicLink f f2 test if symlinks can be made Remote/Directory.hs: go [file] = catchBoolIO $ createSymbolicLink file f >> return True fast key linking; catches failure to make symlink and falls back to copy Remote/Git.hs: liftIO $ catchBoolIO $ createSymbolicLink loc file >> return True ditto Upgrade/V1.hs: liftIO $ createSymbolicLink link f v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to readSymbolicLink. Remaining calls are all safe, because: Annex/Link.hs: ( liftIO $ catchMaybeIO $ readSymbolicLink file only when core.symlinks=true Assistant/Threads/Watcher.hs: ifM ((==) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) code that fixes real symlinks when inotify sees them It's ok to not fix psdueo-symlinks. Assistant/Threads/Watcher.hs: mlink <- liftIO (catchMaybeIO $ readSymbolicLink file) ditto Command/Fix.hs: stopUnless ((/=) (Just link) <$> liftIO (catchMaybeIO $ readSymbolicLink file)) $ do command only works in indirect mode Upgrade/V1.hs: getsymlink = takeFileName <$> readSymbolicLink file v1 repos could not be on a filesystem w/o symlinks Audited and dealt with calls to isSymbolicLink. (Typically used with getSymbolicLinkStatus, but that is just used because getFileStatus is not as robust; it also works on pseudolinks.) Remaining calls are all safe, because: Assistant/Threads/SanityChecker.hs: | isSymbolicLink s -> addsymlink file ms only handles staging of symlinks that were somehow not staged (might need to be updated to support pseudolinks, but this is only a belt-and-suspenders check anyway, and I've never seen the code run) Command/Add.hs: if isSymbolicLink s || not (isRegularFile s) avoids adding symlinks to the annex, so not relevant Command/Indirect.hs: | isSymbolicLink s -> void $ flip whenAnnexed f $ only allowed on systems that support symlinks Command/Indirect.hs: whenM (liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f) $ do ditto Seek.hs:notSymlink f = liftIO $ not . isSymbolicLink <$> getSymbolicLinkStatus f used to find unlocked files, only relevant in indirect mode Utility/FSEvents.hs: | Files.isSymbolicLink s = runhook addSymlinkHook $ Just s Utility/FSEvents.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: | Files.isSymbolicLink s -> Utility/INotify.hs: checkfiletype Files.isSymbolicLink addSymlinkHook f Utility/Kqueue.hs: | Files.isSymbolicLink s = callhook addSymlinkHook (Just s) change all above are lower-level, not relevant Audited and dealt with calls to isSymLink. Remaining calls are all safe, because: Annex/Direct.hs: | isSymLink (getmode item) = This is looking at git diff-tree objects, not files on disk Command/Unused.hs: | isSymLink (LsTree.mode l) = do This is looking at git ls-tree, not file on disk Utility/FileMode.hs:isSymLink :: FileMode -> Bool Utility/FileMode.hs:isSymLink = checkMode symbolicLinkMode low-level Done!!
* fix assistant's use of lsof in crippled filesystem modeGravatar Joey Hess2013-02-15
|
* crippled filesystem support, probing and initial supportGravatar Joey Hess2013-02-14
| | | | | | | | | | | | git annex init probes for crippled filesystems, and sets direct mode, as well as `annex.crippledfilesystem`. Avoid manipulating permissions of files on crippled filesystems. That would likely cause an exception to be thrown. Very basic support in Command.Add for cripped filesystems; avoids the lock down entirely since doing it needs both permissions and hard links. Will make this better soon.
* deal with Android's nonstandard shell locationGravatar Joey Hess2013-02-13
| | | | | | This is so gratutious and pointless. It's a shame that everything we learned about Unix portability and the importance of standards has been thrown out the window by these guys.
* support Android's crippled lsofGravatar Joey Hess2013-02-11
|
* can now build Android targeted binaryGravatar Joey Hess2013-02-10
| | | | | | | | | | | | | | | | Various things that don't work on Android are just ifdefed out. * the webapp (needs template haskell for arm) * --include and --exclude globbing (needs libpcre, which is not ported; probably I'll make it use the pure haskell glob library instead) * annex.diskreserve checking (missing sys/statvfs.h) * timestamp preservation support (yawn) * S3 * WebDAV * XMPP The resulting 17mb binary has been tested on Android, and it is able to, at least, print its usage message.
* avoid a warningGravatar Joey Hess2013-02-06
|
* fix build with webapp disabledGravatar Joey Hess2013-02-06
| | | | Broken by recent thread manager and restarting improvements.