slow, stupid, and safe index updating

Always merge the git-annex branch into .git/annex/index before making a
commit from the index.

This ensures that, when the branch has been changed in any way
(by a push being received, or changes pulled directly into it, or
even by the user checking it out, and committing a change), the index
reflects those changes.

This is much too slow; it needs to be optimised to only update the
index when the branch has really changed, not every time.

Also, there is an unhandled race, when a change is made to the branch
right after the index gets updated. I left it in for now because it's
unlikely and I didn't want to complicate things with additional locking
yet.

4 files changed, 34 insertions, 25 deletions

diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index 44ee69e62..be19a9be0 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -45,14 +45,26 @@ originname = Git.Ref $ "origin/" ++ show name
 
 {- Populates the branch's index file with the current branch contents.
  - 
- - Usually, this is only done when the index doesn't yet exist, and
- - the index is used to build up changes to be commited to the branch,
- - and merge in changes from other branches.
+ - This is only done when the index doesn't yet exist, and the index 
+ - is used to build up changes to be commited to the branch, and merge
+ - in changes from other branches.
  -}
 genIndex :: Git.Repo -> IO ()
 genIndex g = Git.UnionMerge.stream_update_index g
 	[Git.UnionMerge.ls_tree fullname g]
 
+{- Merges the specified branches into the index.
+ - Any changes staged in the index will be preserved. -}
+mergeIndex :: [Git.Ref] -> Annex ()
+mergeIndex branches = do
+	h <- catFileHandle
+	inRepo $ \g -> Git.UnionMerge.merge_index h g branches
+
+{- Updates the branch's index to reflect the current contents of the branch.
+ - Any changes staged in the index will be preserved. -}
+updateIndex :: Annex ()
+updateIndex = withIndex $ mergeIndex [fullname]
+
 {- Runs an action using the branch's index file. -}
 withIndex :: Annex a -> Annex a
 withIndex = withIndex' False
@@ -66,6 +78,8 @@ withIndex' bootstrapping a = do
 			unless bootstrapping $ inRepo genIndex
 		a
 
+{- Runs an action using the branch's index file, first making sure that
+ - the branch and index are up-to-date. -}
 withIndexUpdate :: Annex a -> Annex a
 withIndexUpdate a = update >> withIndex a
 
@@ -106,11 +120,12 @@ create = unlessM hasBranch $ do
 {- Stages the journal, and commits staged changes to the branch. -}
 commit :: String -> Annex ()
 commit message = whenM journalDirty $ lockJournal $ do
+	updateIndex
 	stageJournalFiles
 	withIndex $ inRepo $ Git.commit message fullname [fullname]
 
-{- Ensures that the branch is up-to-date; should be called before data is
- - read from it. Runs only once per git-annex run.
+{- Ensures that the branch and index are is up-to-date; should be
+ - called before data is read from it. Runs only once per git-annex run.
  -
  - Before refs are merged into the index, it's important to first stage the
  - journal into the index. Otherwise, any changes in the journal would
@@ -126,8 +141,9 @@ commit message = whenM journalDirty $ lockJournal $ do
  -}
 update :: Annex ()
 update = onceonly $ do
-	-- ensure branch exists
+	-- ensure branch exists, and index is up-to-date
 	create
+	updateIndex
 	-- check what needs updating before taking the lock
 	dirty <- journalDirty
 	c <- filterM (changedBranch name . snd) =<< siblingBranches
@@ -141,14 +157,7 @@ update = onceonly $ do
 				" into " ++ show name
 		unless (null branches) $ do
 			showSideAction merge_desc
-			{- Note: This merges the branches into the index.
-			 - Any unstaged changes in the git-annex branch
-			 - (if it's checked out) will be removed. So,
-			 - documentation advises users not to directly
-			 - modify the branch.
-			 -}
-			h <- catFileHandle
-			inRepo $ \g -> Git.UnionMerge.merge_index h g branches
+			mergeIndex branches
 		ff <- if dirty then return False else tryFastForwardTo refs
 		unless ff $ inRepo $
 			Git.commit merge_desc fullname (nub $ fullname:refs)
diff --git a/debian/changelog b/debian/changelog
index 5c68fde2d..405e98b74 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -13,6 +13,11 @@ git-annex (3.20111204) UNRELEASED; urgency=low
     remote, by running git commit, pull, and push for you.
   * Version monad-control dependency in cabal file.
   * Fix bug in last version in getting contents from bare repositories.
+  * Ensure that git-annex branch changes are merged into git-annex's index,
+    which fixes a bug that could cause changes that were pushed to the
+    git-annex branch to get reverted. As a side effect, it's now safe
+    for users to check out and commit changes directly to the git-annex
+    branch.
 
  -- Joey Hess <joeyh@debian.org>  Sun, 04 Dec 2011 12:22:37 -0400
 
diff --git a/doc/bugs/git-annex_branch_corruption.mdwn b/doc/bugs/git-annex_branch_corruption.mdwn
index 63633fb08..5249e63d8 100644
--- a/doc/bugs/git-annex_branch_corruption.mdwn
+++ b/doc/bugs/git-annex_branch_corruption.mdwn
@@ -39,6 +39,9 @@ corresponds to its index, and if the branch is at a different ref,
 merge it into the index. I am still considering how to do that atomically;
 what if a push comes in while git-annex is updating its index?
 
+> Now git-annex always updates the index with the git-annex branch, which
+> is a slow, but safe way to avoid this problem. [[done]] --[[Joey]]
+
 ---
 
 ## Workaround
diff --git a/doc/internals.mdwn b/doc/internals.mdwn
index d84b3c489..68cc7c3cd 100644
--- a/doc/internals.mdwn
+++ b/doc/internals.mdwn
@@ -22,17 +22,9 @@ deleting or changing the file contents.
 This branch is managed by git-annex, with the contents listed below.
 
 The file `.git/annex/index` is a separate git index file it uses
-to accumulate changes for the git-annex. Also, `.git/annex/journal/` is used
-to record changes before they are added to git.
-
-Note that for speed reasons, git-annex assumes only it will modify this
-branch. If you go in and make changes directly, it will probably revert
-your changes in its next commit to the branch.
-
-The best way to make changes to the git-annex branch is instead
-to create a branch of it, with a name like "my/git-annex", and then
-use "git annex merge" to automerge your branch into the main git-annex
-branch.
+to accumulate changes for the git-annex branch.
+Also, `.git/annex/journal/` is used to record changes before they
+are added to git.
 
 ### `uuid.log`