test: Abort driver if a test script aborts

Previously, if a test script aborted (e.g., because it passed too few
arguments to a test function), the test driver loop would simply
continue on to the next test script and the final results would
declare that everything passed (except that the test count would look
suspiciously low, but maybe you just misremembered how many tests
there were).

Now, if a test script exits with a non-zero status and did not produce
a final results file, we propagate that failure out of the driver loop
immediately.

To keep this simple, this patch removes the PID from the test-results
file name.  This PID was inherited from the git test system and seems
unnecessary, since the file name already includes the name of the test
script and the test-results directory is created anew for each run.

3 files changed, 8 insertions, 2 deletions

diff --git a/test/basic b/test/basic
index 1b842d2b..b7feb07e 100755
--- a/test/basic
+++ b/test/basic
@@ -79,7 +79,7 @@ test_begin_subtest "Ensure that -v does not suppress test output"
 output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
-rm $TEST_DIRECTORY/test-results/test-verbose-*
+rm $TEST_DIRECTORY/test-results/test-verbose
 rm -r $TEST_DIRECTORY/tmp.test-verbose
 test_expect_equal "$output" "$expected"
 
diff --git a/test/notmuch-test b/test/notmuch-test
index 9a1b3758..f2754398 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -81,6 +81,12 @@ trap 'e=$?; kill $!; exit $e' HUP INT TERM
 for test in $TESTS; do
     $TEST_TIMEOUT_CMD ./$test "$@" &
     wait $!
+    # If the test failed without producing results, then it aborted,
+    # so we should abort, too.
+    RES=$?
+    if [[ $RES != 0 && ! -e "test-results/${test%.sh}" ]]; then
+        exit $RES
+    fi
 done
 trap - HUP INT TERM
 
diff --git a/test/test-lib.sh b/test/test-lib.sh
index e0922315..77063a4d 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -920,7 +920,7 @@ test_done () {
 	GIT_EXIT_OK=t
 	test_results_dir="$TEST_DIRECTORY/test-results"
 	mkdir -p "$test_results_dir"
-	test_results_path="$test_results_dir/${0%.sh}-$$"
+	test_results_path="$test_results_dir/${0%.sh}"
 
 	echo "total $test_count" >> $test_results_path
 	echo "success $test_success" >> $test_results_path