views: add automatically constructed file location metadata

When constructing views, metadata is available about the location of the
file in the view's reference branch. Allows incorporating parts of the
directory hierarchy in a view.

For example `git annex view tag=* podcasts/=*` makes a view in the form
tag/showname.

Performance impact: I benchmarked git annex view tag=* in the conference
proceedings repo to take 6.459s before this change, and 6.544s after.

FWIW, I considered making the syntax for this be podcasts/*, which might
be easier for the user to learn. However, I think it's not as good:

* The user has to then juggle two different syntaxes, and podcasts/* will
  be expanded by the shell so they also need to quote it, while podcasts/=*
  is unlikely to be expanded by the shell.
* It would allow for things like podcasts/*/* and *.mp3 which do not
  map well into views.

This commit was sponsored by Aurélien Pinceaux.

3 files changed, 49 insertions, 16 deletions

diff --git a/doc/design/metadata.mdwn b/doc/design/metadata.mdwn
index db0d51c5c..c700e3674 100644
--- a/doc/design/metadata.mdwn
+++ b/doc/design/metadata.mdwn
@@ -70,7 +70,7 @@ metadata is derived, at least year=yyyy and probably also month, etc.
 
 ### directory hierarchy metadata
 
-TODO From the original filename used in the master branch, when
+From the original filename used in the master branch, when
 constructing a view, generate fields. For example foo/bar/baz.mp3
 would get /=foo, foo/=bar, foo/bar/=baz, and .=mp3.
 
@@ -82,11 +82,12 @@ This allows using whatever directory hierarchy exists to inform the view,
 without locking the view into using it. 
 
 Complication: When refining a view, it only looks at the filenames in
-the view, so it would need to map from
+the view, so it has to map from
 those filenames to derive the same metadata, unless there is persistent
 storage. Luckily, the filenames used in the views currently include the
-subdirs (although not quite in a parseable format, would need some small
-changes).
+subdirs.
+
+**done**!
 
 # other uses for metadata
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index f3bbe0502..3c233f378 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -715,20 +715,29 @@ subdirectories).
 
         	git annex metadata annexscreencast.ogv -t video -t screencast -s author+=Alice
 
-* `view [field=value ...] [tag ...]`
+* `view [tag ...] [field=value ...] [location/=value]`
 
   Uses metadata to build a view branch of the files in the current branch,
   and checks out the view branch. Only files in the current branch whose
   metadata matches all the specified field values and tags will be
   shown in the view.
+
+  Once within a view, you can make additional directories, and
+  copy or move files into them. When you commit, the metadata will
+  be updated to correspond to your changes.
   
   Multiple values for a metadata field can be specified, either by using
   a glob (`field="*"`) or by listing each wanted value. The resulting view
   will put files in subdirectories according to the value of their fields.
 
-  Once within a view, you can make additional directories, and
-  copy or move files into them. When you commit, the metadata will
-  be updated to correspond to your changes.
+  There are fields corresponding to the path to the file. So a file
+  "foo/bar/baz/file" has fields "/=foo", "foo/=bar", and "foo/bar/=baz".
+  These location fields can be used the same as other metadata to construct
+  the view.
+
+  For example, `/=podcasts` will only include files from the podcasts
+  directory in the view, while `podcasts/=*` will preserve the
+  subdirectories of the podcasts directory in the view.
 
 * `vpop [N]`
 
@@ -737,12 +746,12 @@ subdirectories).
 
   The optional number tells how many views to pop.
 
-* `vfilter [field=value ...] [tag ...]`
+* `vfilter [tag ...] [field=value ...] [location/=value]`
 
   Filters the current view to only the files that have the
-  specified values and tags.
+  specified field values, tags, and locations.
 
-* `vadd [field=glob ...]`
+* `vadd [field=glob ...] [location/=glob]`
 
   Changes the current view, adding an additional level of directories
   to categorize the files.
diff --git a/doc/tips/metadata_driven_views.mdwn b/doc/tips/metadata_driven_views.mdwn
index 7b46ca974..e24bf29ae 100644
--- a/doc/tips/metadata_driven_views.mdwn
+++ b/doc/tips/metadata_driven_views.mdwn
@@ -24,8 +24,8 @@ metadata:
 	# git annex metadata --tag done videos/old
 	# git annex metadata --tag new videos/lotsofcats.ogv
 	# git annex metadata --tag sound podcasts
-	# git annex metadata --tag done podcasts/old
-	# git annex metadata --tag new podcasts/recent
+	# git annex metadata --tag done podcasts/*/old
+	# git annex metadata --tag new podcasts/*/recent
 
 So, you had a bunch of different kinds of files sorted into a directory
 structure. But that didn't really reflect how you approach the files.
@@ -81,9 +81,11 @@ all the way out of all views, you'll be back on the regular git branch you
 originally started from. You can also use `git checkout` to switch between
 views and other branches.
 
-Beyond simple tags, you can add whatever kinds of metadata you like, and
-use that metadata in more elaborate views. For example, let's add a year
-field.
+## fields
+
+Beyond simple tags and directories, you can add whatever kinds of metadata
+you like, and use that metadata in more elaborate views. For example, let's
+add a year field.
 
 	# git checkout master
 	# git annex metadata --set year=2014 work/2014
@@ -118,4 +120,25 @@ Oh, did you want it the other way around? Easy!
 	  |-- 2014
 	  `-- 2013
 
+## location fields
+
+Let's switch to a view containing only new podcasts. And since the
+podcasts are organized into one subdirectory per show, let's
+include those subdirectories in the view.
+
+	# git checkout master
+	# git annex view tag=new podcasts/=*
+	# tree -d
+	This_Developers_Life
+	Escape_Pod
+	GitMinutes
+	The_Haskell_Cast
+	StarShipSofa
+
+That's an example of using part of the directory layout of the original
+branch to inform the view. Every file gets fields automatically set up
+corresponding to the directory it's in. So a file"foo/bar/baz/file" has
+fields "/=foo", "foo/=bar", and "foo/bar/=baz". These location fields
+can be used the same as other metadata to construct the view.
+
 This has probably only scratched the surface of what you can do with views.