Merge branch 'adjustedbranch'

6 files changed, 189 insertions, 94 deletions

diff --git a/doc/design/adjusted_branches.mdwn b/doc/design/adjusted_branches.mdwn
index f79a7f1ab..a4fd40650 100644
--- a/doc/design/adjusted_branches.mdwn
+++ b/doc/design/adjusted_branches.mdwn
@@ -62,12 +62,9 @@ it, so C does not remain in the adjusted branch history either. This will
 make other checkouts that are in the same adjusted branch end up with the
 same B' commit when they pull B.
 
-It might be useful to have a post-commit hook that generates B and B'
-and updates the branches. And/or `git-annex sync` could do it.
-
 There may be multiple commits made to the adjusted branch before any get
 applied back to the original branch. This is handled by reverse filtering
-one at a time and rebasing the others on top.
+commits one at a time and rebasing the others on top.
 
 	master           adjusted/master
 	A
@@ -112,10 +109,10 @@ beginning the merge. There may be staged changes, or changes in the work tree.
 
 First filter the new commit:
 
-	origin/master    adjusted/master
-	A
-	|--------------->A'
-	|                |
+	origin/master    adjusted/master      master
+	A                                     A
+	|--------------->A'                   |
+	|                |                    |
 	|                |
 	B
 	|                
@@ -123,10 +120,10 @@ First filter the new commit:
 
 Then, merge that into adjusted/master:
 
-	origin/master    adjusted/master
-	A
-	|--------------->A'
-	|                |
+	origin/master    adjusted/master     master
+	A                                    A
+	|--------------->A'                  |
+	|                |                   |
 	|                |
 	B                |
 	|                |
@@ -139,20 +136,30 @@ conflict should only affect the work tree/index, so can be resolved without
 making a commit, but B'' may end up being made to resolve a merge
 conflict.)
 
-Once the merge is done, we have a commit B'' on adjusted/master. To finish,
-adjust that commit so it does not have adjusted/master as its parent.
+Once the merge is done, we have a merge commit B'' on adjusted/master.
+To finish, redo that commit so it does not have A' as its parent.
 
-	origin/master    adjusted/master
-	A
-	|--------------->A'
-	|                |
+	origin/master    adjusted/master     master
+	A                                    A
+	|--------------->A'                  |
+	|                |                   |
 	|                |
 	B
 	|                
 	|--------------->B''
 	|                |
 
-Finally, update master to point to B''.
+Finally, update master, by reverse filtering B''.
+	
+	origin/master    adjusted/master    master
+	A                                   A
+	|--------------->A'                 |
+	|                |                  |
+	|                |                  |
+	B                                   |
+	|                                   |
+	|--------------->B'' - - - - - - -> B
+	|                |
 
 Notice how similar this is to the commit graph. So, "fast-forward" 
 merging the same B commit from origin/master will lead to an identical
@@ -172,48 +179,90 @@ between the adjusted work tree and pulled changes. A post-merge hook would
 be needed to re-adjust the work tree, and there would be a window where eg,
 not present files would appear in the work tree.]
 
+## another merge scenario
+
+Another merge scenario is when there's a new commit C on adjusted/master,
+and also a new commit B on origin/master.
+
+Start by adjusting B':
+
+	origin/master    adjusted/master      master
+	A                                     A
+	|--------------->A'                   |
+	|                |                    |
+	|                C'
+	B
+	|                
+	|---------->B'
+
+Then, merge B' into adjusted/master:
+
+	origin/master    adjusted/master     master
+	A                                    A
+	|--------------->A'                  |
+	|                |                   |
+	|                C'
+	B                |
+	|                |
+	|----------->B'->M'
+
+Here M' is the correct tree, but it has A' as its grandparent,
+which is the adjusted branch commit, so needs to be dropped in order to 
+get a commit that can be put on master.
+
+We don't want to lose commit C', but it's an adjusted
+commit, so needs to be de-adjusted.
+
+	origin/master    adjusted/master     master
+	A                                    A
+	|--------------->A'                  |
+	|                |                   |
+	|                C'- - - - - - - - > C
+	B                |
+	|                |
+	|----------->B'->M'
+	|               
+
+Now, we generate a merge commit, between B and C, with known result M'
+(so no actual merging done here).
+
+	origin/master    adjusted/master     master
+	A                                    A
+	|--------------->A'                  |
+	|                |                   |
+	|                C'- - - - - - - - > C
+	B                                    |
+	|                                    |
+	|--------------->M'<-----------------|
+	|               
+
+Finally, update master, by reverse filtering M'. The resulting commit
+on master will also be a merge between B and C.
+
 ## annex object add/remove
 
 When objects are added/removed from the annex, the associated file has to
 be looked up, and the filter applied to it. So, dropping a file with the
 missing file filter would cause it to be removed from the adjusted branch,
 and receiving a file's content would cause it to appear in the adjusted
-branch.
+branch. TODO
 
 These changes would need to be committed to the adjusted branch, otherwise
 `git diff` would show them.
 
 [WORKTREE: Simply adjust the work tree (and index) per the filter.]
 
-## reverse filtering
-
-Reversing filter #1 would mean only converting pointer files to
-symlinks when the file was originally a symlink. This is problimatic when a
-file is renamed. Would it be ok, if foo is renamed to bar and bar is
-committed, for it to be committed as an unlocked file, even if foo was
-originally locked? Probably.
+## reverse filtering commits
 
-Reversing filter #2 would mean not deleting removed files whose content was
-not present. When the commit includes deletion of files that were removed
-due to their content not being present, those deletions are not propigated.
-When the user deletes an unlocked file, the content is still
-present in annex, so reversing the filter should propigate the file
-deletion. 
+A user's commits on the adjusted branch have to be reverse filtered
+to get changes to apply to the master branch.
 
-What if an object was sent to the annex (or removed from the annex)
-after the commit and before the reverse filtering? This would cause the
-reverse filter to draw the wrong conclusion. Maybe look at a list of what
-objects were not present when applying the filter, and use that to decide
-which to not delete when reversing it?
+This reversal of one filter can be done as just another filter.
+Since only files touched by the commit will be reverse filtered, it doesn't
+need to reverse all changes made by the original filter.
 
-So, a reverse filter may need some state that was collected when running
-the filter forwards, in order to decide what to do.
-
-Alternatively, instead of reverse filtering the whole adjusted tree,
-look at just the new commit that's being propigated back from the
-adjusted to master branch. Get the diff from it to the previous
-commit; the changes that were made. Then de-adjust those changes,
-and apply the changes to the master branch.
+For example, reversing the unlock filter might lock the file. Or, it might
+do nothing, which would make all committed files remain unlocked.
 
 ## push
 
@@ -254,8 +303,15 @@ index in that case.
 
 Using `git checkout` when in an adjusted branch is problimatic, because a
 non-adjusted branch would then be checked out. But, we can just say, if
-you want to get into an adjusted branch, you have to run some command.
-Or, could make a post-checkout hook.
+you want to get into an adjusted branch, you have to run git annex adjust
+Or, could make a post-checkout hook. This is would mostly be confusing when
+git-annex init switched into the adjusted branch due to lack of symlink
+support.
+
+After a commit to an adjusted branch, `git push` won't do anything. The
+user has to know to git-annex sync. (Even if a pre-commit hook propigated
+the commit back to the master branch, `git push` wouldn't push it with the
+default "matching" push strategy.)
 
 Tags are bit of a problem. If the user tags an ajusted branch, the tag
 includes the local adjustments.  
@@ -282,47 +338,23 @@ adjusting filter, albeit an extreme one. This might improve view branches.
 For example, it's not currently possible to update a view branch with
 changes fetched from a remote, and this could get us there.
 
-This would need the reverse filter to be able to change metadata.
+This would need the reverse filter to be able to change metadata,
+so that a commit that moved files in the view updates their metadata.
 
 [WORKTREE: Wouldn't be able to integrate, unless view branches are changed
 into adjusted view worktrees.]
 
-## filter interface
-
-Distilling all of the above, the filter interface needs to be something
-like this, at its most simple:
-
-	data Filter = UnlockFilter | HideMissingFilter | UnlockHideMissingFilter
-
-	getFilter :: Annex Filter
-
-	setFilter :: Filter -> Annex ()
-
-	data FilterAction
-		= UnchangedFile FilePath
-		| UnlockFile FilePath
-		| HideFile FilePath
-
-	data FileInfo = FileInfo
-		{ originalBranchFile :: FileStatus
-		, isContentPresent :: Bool
-		}
-
-	data FileStatus = IsAnnexSymlink | IsAnnexPointer
-		deriving (Eq)
-
-	filterAction :: Filter -> FilePath -> FileInfo -> FilterAction
-	filterAction UnlockFilter f fi
-		| originalBranchFile fi == IsAnnexSymlink = UnlockFile f
-	filterAction HideMissingFilter f fi
-		| not (isContentPresent fi) = HideFile f
-	filterAction UnlockHideMissingFilter f fi
-		| not (isContentPresent fi) = HideFile f
-		| otherwise = filterAction UnlockFilter f fi
-	filterAction _ f _ = UnchangedFile f
-
-	filteredCommit :: Filter -> Git.Commit -> Git.Commit
-
-	-- Generate a version of the commit made on the filter branch
-	-- with the filtering of modified files reversed.
-	unfilteredCommit :: Filter -> Git.Commit -> Git.Commit
+## TODOs
+
+* Interface in webapp to enable adjustments.
+* Upgrade from direct mode to v6 in unlocked branch.
+* Honor annex.thin when entering an adjusted branch.
+* Cloning a repo that has an adjusted branch checked out gets into an ugly
+  state.
+* There are potentially races in code that assumes a branch like
+  master is not being changed by someone else. In particular,
+  propigateAdjustedCommits rebases the adjusted branch on top of master.
+  That is called by sync. The assumption is that any changes in master
+  have already been handled by updateAdjustedBranch. But, if another remote
+  pushed a new master at just the right time, the adjusted branch could be
+  rebased on top of a master that it doesn't incorporate, which is wrong.
diff --git a/doc/git-annex-adjust.mdwn b/doc/git-annex-adjust.mdwn
new file mode 100644
index 000000000..551eabe01
--- /dev/null
+++ b/doc/git-annex-adjust.mdwn
@@ -0,0 +1,50 @@
+# NAME
+
+git-annex adjust - enter an adjusted branch
+
+# SYNOPSIS
+
+git annex adjust --unlock`
+
+# DESCRIPTION
+
+Enters an adjusted form of the current branch. The annexed files will
+be treated differently. For example with --unlock all annexed files will
+be unlocked.
+
+The adjusted branch will have a name like "adjusted/master(unlocked)".
+Since it's a regular git branch, you can use `git checkout` to switch
+back to the original branch at any time.
+
+While in the adjusted branch, you can use git-annex and git commands as
+usual. Any commits that you make will initially only be made to the
+adjusted branch. 
+
+To propigate changes from the adjusted branch back to the original branch,
+and to other repositories, as well as to merge in changes from other
+repositories, use `git annex sync`.
+
+This command can only be used in a v6 git-annex repository.
+
+# OPTIONS
+
+* `--unlock`
+
+  Unlock all annexed files in the adjusted branch. This allows
+  annexed files to be modified.
+
+# SEE ALSO
+
+[[git-annex]](1)
+
+[[git-annex-unlock]](1)
+
+[[git-annex-upgrade]](1)
+
+[[git-annex-sync]](1)
+
+# AUTHOR
+
+Joey Hess <id@joeyh.name>
+
+Warning: Automatically converted into a man page by mdwn2man. Edit with care.
diff --git a/doc/git-annex-direct.mdwn b/doc/git-annex-direct.mdwn
index 3cade1a8c..c3d7dfadc 100644
--- a/doc/git-annex-direct.mdwn
+++ b/doc/git-annex-direct.mdwn
@@ -20,6 +20,8 @@ commands.
 Note that the direct mode/indirect mode distinction is removed in v6
 git-annex repositories. In such a repository, you can
 use [[git-annex-unlock]](1) to make a file's content be directly present.
+You can also use [[git-annex-adjust]](1) to enter a branch where all
+annexed files are unlocked, which is similar to the old direct mode.
 
 # SEE ALSO
 
@@ -29,6 +31,8 @@ use [[git-annex-unlock]](1) to make a file's content be directly present.
 
 [[git-annex-unlock]](1)
 
+[[git-annex-adjust]](1)
+
 # AUTHOR
 
 Joey Hess <id@joeyh.name>
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 6830f741f..e9698c169 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -295,6 +295,13 @@ subdirectories).
   
   See [[git-annex-indirect]](1) for details.
 
+* `adjust`
+
+  Switches a repository to use an adjusted branch, which can automatically
+  unlock all files, etc.
+  
+  See [[git-annex-adjust]](1) for details.
+
 # REPOSITORY MAINTENANCE COMMANDS
 
 * `fsck [path ...]`
diff --git a/doc/tips/unlocked_files.mdwn b/doc/tips/unlocked_files.mdwn
index fd103940e..cc9972f9e 100644
--- a/doc/tips/unlocked_files.mdwn
+++ b/doc/tips/unlocked_files.mdwn
@@ -95,6 +95,8 @@ mode is used. To make them always use unlocked mode, run:
 `git config annex.addunlocked true`
 """]]
 
+## mixing locked and unlocked files
+
 A v6 repository can contain both locked and unlocked files. You can switch 
 a file back and forth using the `git annex lock` and `git annex unlock`
 commands. This changes what's stored in git between a git-annex symlink
@@ -102,6 +104,12 @@ commands. This changes what's stored in git between a git-annex symlink
 the repository in locked mode, use `git annex add`; to add a file in
 unlocked mode, use `git add`.
 
+If you want to mostly keep files locked, but be able to locally switch
+to having them all unlocked, you can do so using `git annex adjust
+--unlock`. See [[git-annex-adjust]] for details. This is particularly
+useful when using filesystems like FAT, and OS's like Windows that don't
+support symlinks.
+
 ## using less disk space
 
 Unlocked files are handy, but they have one significant disadvantage
diff --git a/doc/todo/smudge.mdwn b/doc/todo/smudge.mdwn
index c615f8f14..a80869dc0 100644
--- a/doc/todo/smudge.mdwn
+++ b/doc/todo/smudge.mdwn
@@ -23,12 +23,6 @@ git-annex should use smudge/clean filters.
   (May need to use libgit2 to do this efficiently, cannot find
   any plumbing except git-update-index, which is very inneficient for
   smudged files.)
-* Crippled filesystem should cause all files to be transparently unlocked.
-  Note that this presents problems when dealing with merge conflicts and
-  when pushing changes committed in such a repo. Ideally, should avoid
-  committing implicit unlocks, or should prevent such commits leaking out
-  in pushes. See [[design/adjusted_branches]].
-
 * Eventually (but not yet), make v6 the default for new repositories.
   Note that the assistant forces repos into direct mode; that will need to
   be changed then, and it should enable annex.thin instead.