test: rearrange the test corpus into subfolders, fix tests

We will need this for improved folder search tests, but having some
folders should exercise our code paths better anyway.

Modify the relevant test accordingly to make it pass.

This reorganization triggers a bug in the test suite, namely that it
expects the output of --output=files to be in a certain order. So we
add the fix for that into the same commit.

This mainly involves sorting, although the case --duplicate=$n
requires more subtlety.

3 files changed, 180 insertions, 0 deletions

diff --git a/test/corpus/foo/new/03:2, b/test/corpus/foo/new/03:2,
new file mode 100644
index 00000000..c154ac5e
--- /dev/null
+++ b/test/corpus/foo/new/03:2,
@@ -0,0 +1,93 @@
+Date: Tue, 17 Nov 2009 14:00:54 -0500
+From: Lars Kellogg-Stedman <lars@seas.harvard.edu>
+To: notmuch@notmuchmail.org
+Message-ID: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+MIME-Version: 1.0
+User-Agent: Mutt/1.5.19 (2009-01-05)
+Subject: [notmuch] Working with Maildir storage?
+X-BeenThere: notmuch@notmuchmail.org
+X-Mailman-Version: 2.1.12
+Precedence: list
+List-Id: "Use and development of the notmuch mail system."
+	<notmuch.notmuchmail.org>
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>
+List-Post: <mailto:notmuch@notmuchmail.org>
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
+Content-Type: multipart/mixed; boundary="===============1685355122=="
+Sender: notmuch-bounces@notmuchmail.org
+Errors-To: notmuch-bounces@notmuchmail.org
+
+
+--===============1685355122==
+Content-Type: multipart/signed; micalg=pgp-sha256;
+	protocol="application/pgp-signature"; boundary="5Dr6Wqe9hdyl7LAI"
+Content-Disposition: inline
+
+
+--5Dr6Wqe9hdyl7LAI
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+Content-Transfer-Encoding: quoted-printable
+
+I saw the LWN article and decided to take a look at notmuch.  I'm
+currently using mutt and mairix to index and read a collection of
+Maildir mail folders (around 40,000 messages total).
+
+notmuch indexed the messages without complaint, but my attempt at
+searching bombed out. Running, for example:
+
+  notmuch search storage
+
+Resulted in 4604 lines of errors along the lines of:
+
+  Error opening
+  /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=
+=3D3026:2,S:
+  Too many open files
+
+I'm curious if this is expected behavior (i.e., notmuch does not work
+with Maildir) or if something else is going on.
+
+Cheers,
+
+--=20
+Lars Kellogg-Stedman <lars@seas.harvard.edu>
+Senior Technologist, Computing and Information Technology
+Harvard University School of Engineering and Applied Sciences
+
+
+--5Dr6Wqe9hdyl7LAI
+Content-Type: application/pgp-signature
+Content-Disposition: inline
+
+-----BEGIN PGP SIGNATURE-----
+Version: GnuPG v1.4.9 (GNU/Linux)
+
+iQEcBAEBCAAGBQJLAvLmAAoJENdGlQYxQazYRtcH/0usClQ1Z+EoTsA+URwIK6hD
+FsZUxFxRjMuOQRn2idZ/zhhg5jJj11ZaHjqxSkDvi2ywkTKUf1vX9LLzVy5hSR9M
+E6XQUd5QWAQXo1VsTeKkukIL0YqsPjdgrT8+Yt+OS2NvhEncql23oxnL2/pHkIFq
+r0NdTmVV5Jcar7w9J6X1Mi9m229a/9jV5FImsWISkIhIWznXU5SiU6zIw8xhP4E0
+xhvVSNJnFryjVHtva870aSQduhHfeLPzpYhqbkMPvlq+bcz6Q/Q2SwxJcGLNMPHa
+os9s9FGhCvFKUhVzezHWPgXNCcNT8qK89rcUldb5Oq4jaJb8RCZCYABplfoyaFs=
+=vO4s
+-----END PGP SIGNATURE-----
+
+--5Dr6Wqe9hdyl7LAI--
+
+--===============1685355122==
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+notmuch mailing list
+notmuch@notmuchmail.org
+http://notmuchmail.org/mailman/listinfo/notmuch
+
+--===============1685355122==--
+
diff --git a/test/corpus/foo/new/09:2, b/test/corpus/foo/new/09:2,
new file mode 100644
index 00000000..26b51b1f
--- /dev/null
+++ b/test/corpus/foo/new/09:2,
@@ -0,0 +1,33 @@
+From: "Mikhail Gusarov" <dottedmag@dottedmag.net>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:50:48 +0600
+Subject: [notmuch] Working with Maildir storage?
+In-Reply-To: <20091117203301.GV3165@dottiness.seas.harvard.edu> (Lars
+	Kellogg-Stedman's message of "Tue, 17 Nov 2009 15:33:01 -0500")
+References: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+	<87iqd9rn3l.fsf@vertex.dottedmag>
+	<20091117203301.GV3165@dottiness.seas.harvard.edu>
+Message-ID: <87fx8can9z.fsf@vertex.dottedmag>
+
+
+Twas brillig at 15:33:01 17.11.2009 UTC-05 when lars at seas.harvard.edu did gyre and gimble:
+
+ LK> Is the list archived anywhere?  The obvious archives
+ LK> (http://notmuchmail.org/pipermail/notmuch/) aren't available, and I
+ LK> think I subscribed too late to get the patch (I only just saw the
+ LK> discussion about it).
+
+ LK> It doesn't look like the patch is in git yet.
+
+Just has been pushed
+
+-- 
+  http://fossarchy.blogspot.com/
+-------------- next part --------------
+A non-text attachment was scrubbed...
+Name: not available
+Type: application/pgp-signature
+Size: 834 bytes
+Desc: not available
+URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20091118/0e33d964/attachment.pgp>
+
diff --git a/test/corpus/foo/new/10:2, b/test/corpus/foo/new/10:2,
new file mode 100644
index 00000000..4211d734
--- /dev/null
+++ b/test/corpus/foo/new/10:2,
@@ -0,0 +1,54 @@
+From: "Mikhail Gusarov" <dottedmag@dottedmag.net>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:51:18 +0600
+Subject: [notmuch] [PATCH] Handle rename of message file
+Message-ID: <1258491078-29658-1-git-send-email-dottedmag@dottedmag.net>
+
+If message file has been renamed, just update filename in the DB.
+
+Signed-off-by: Mikhail Gusarov <dottedmag at dottedmag.net>
+---
+ lib/database.cc |   21 ++++++++++++---------
+ 1 files changed, 12 insertions(+), 9 deletions(-)
+
+diff --git a/lib/database.cc b/lib/database.cc
+index 3c8d626..c4eb8b6 100644
+--- a/lib/database.cc
++++ b/lib/database.cc
+@@ -925,20 +925,23 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
+ 	if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
+ 	    _notmuch_message_set_filename (message, filename);
+ 	    _notmuch_message_add_term (message, "type", "mail");
++
++	    ret = _notmuch_database_link_message (notmuch, message, message_file);
++	    if (ret)
++		goto DONE;
++
++	    date = notmuch_message_file_get_header (message_file, "date");
++	    _notmuch_message_set_date (message, date);
++
++	    _notmuch_message_index_file (message, filename);
++	} else if (strcmp(notmuch_message_get_filename(message), filename)) {
++	    /* Message file has been moved/renamed */
++	    _notmuch_message_set_filename (message, filename);
+ 	} else {
+ 	    ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
+ 	    goto DONE;
+ 	}
+ 
+-	ret = _notmuch_database_link_message (notmuch, message, message_file);
+-	if (ret)
+-	    goto DONE;
+-
+-	date = notmuch_message_file_get_header (message_file, "date");
+-	_notmuch_message_set_date (message, date);
+-
+-	_notmuch_message_index_file (message, filename);
+-
+ 	_notmuch_message_sync (message);
+     } catch (const Xapian::Error &error) {
+ 	fprintf (stderr, "A Xapian exception occurred: %s.\n",
+-- 
+1.6.3.3
+
+