From a5344a3940f292d8d4f8907f92a54bf726b87e44 Mon Sep 17 00:00:00 2001 From: patrick brisbin Date: Fri, 23 Aug 2013 14:31:32 -0400 Subject: An integration test suite This test suite uses cram to run integration tests through `/bin/sh`. The tests are all high-level acceptance tests; they should work regardless of the implemention code. To run them, you must first install cram: sudo pip install cram Then the `check` target will run them: make check Failure output should be printed clearly to stdout, but in general: full test output is in `test/test-suite.log` and output specific to a test named `foo.t` is in `foo.t.log`. Tests are now encouraged in `CONTRIBUTING.md` as part of the normal pull request process. This is a TAP-enabled test suite. --- CONTRIBUTING.md | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) (limited to 'CONTRIBUTING.md') diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fcf9da1..478dd56 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,18 +4,15 @@ Contributing Overview -------- -- We do not currently have a test suite. - Update `NEWS.md.in`. - Update `.mailmap`. +- Write a test covering your feature or fix. +- Ensure existing and new tests are passing. - Submit a pull request on GitHub. Explanation ----------- -Since we do not yet have a test suite, you should make sure to install -and try your patch in a various scenarios. We will also entertain a -contribution with a test case. - Consider updating `NEWS.md.in`. The topmost section is for the upcoming release. Bugfixes should be marked with `BUGFIX`. Small things (typos, code style) should be grouped but with multiple authors (`Documentation @@ -25,7 +22,37 @@ We use your name and email address as produced by `git-shortlog(1)`. You can change how this is formatted by modifying `.mailmap`. More details on that file can be found in the git [Documentation/mailmap.txt][mailmap]. +Our test suite is new, and therefore it is not yet mandatory to include +tests with pull requests. However, you must ensure that the existing +test suite passes with any changes you make. Also, any attempts to add +or extend tests will increase the chances of your pull request being +merged. + Submit a pull request using GitHub. If there is a relevant bug, mention it in the commit message (`Fixes #42.`). [mailmap]: https://github.com/git/git/blob/6a907786af835ac15962be53f1492f2 + +Testing +----- + +The test suite uses [cram][]. It is an integration suite, meaning the +programs are exercised from the outside and assertions are made only on +their output or effects. + +All tests can be run like so: + + $ make check + +Individual tests can be run like so: + + $ env TESTS=lsrc-dotfiles-dirs.t make -e check + +If you intend to write a new test: + +1. Add your test at `test/subcommand-something-meaningful.t`. +2. Add the relative name to the `TESTS` variable in `test/Makefile.am`. +3. Source `test/helper.sh` as the first line of your test. +4. When in doubt, use existing tests as a guide. + +[cram]: https://bitheap.org/cram/ -- cgit v1.2.3