aboutsummaryrefslogtreecommitdiffhomepage
path: root/test/test-lib.sh
Commit message (Collapse)AuthorAge
...
* test: do not hide test_emacs errorsGravatar Dmitry Kurochkin2011-11-13
| | | | | | | | Do not redirect test_emacs stderr to /dev/null. Test_emacs uses emacsclient(1) now and it does not print unwanted messages (like those from `message') to stderr. But it does print useful errors, e.g. when emacs server connection fails, given expression is not valid or undefined function is called.
* exec emacs at the end of run_emacs scriptGravatar Tomi Ollila2011-11-12
| | | | | In the last line of run_emacs, exec the emacs process. With one fork less the process list is (also) neater.
* test: use dtach(1) instead of screen(1) in emacs testsGravatar Tomi Ollila2011-11-11
| | | | | dtach is simpler than screen and is not setuid/setgid program so TMPDIR does not get cleared by dynamic loader when executed
* test: avoid using screen(1) configuration filesGravatar Dmitry Kurochkin2011-11-05
| | | | | | Set SCREENRC and SYSSCREENRC environment variables to "/dev/null" as suggested by Jim Paris to avoid potential problems with screen(1) configuration files.
* test: run emacs inside screenGravatar Dmitry Kurochkin2011-11-05
| | | | | | | | | | | | | Before the change, emacs run in daemon mode without any visible buffers. Turns out that this affects emacs behavior in some cases. In particular, `window-end' function returns `point-max' instead of the last visible position. That makes it hard or impossible to implement some tests. The patch runs emacs in a detached screen(1) session. So that it works exactly as if it has a visible window. Note: screen terminates when emacs exits. So the patch does not introduce new "running processes left behind" issues.
* test/test-lib.sh: update commentsGravatar David Bremner2011-10-22
| | | | | | | - explain test_expect_equal_file - remove mention of test_expect_failure, since that function was removed. Based on id:"1317317811-29540-1-git-send-email-thomas@schwinge.name"
* test: Report test failures from test_expect_*Gravatar Austin Clements2011-09-12
| | | | | | This makes test_expect_* return non-zero if the test fails, so the caller can make decisions based on this, such as setting test prerequisites.
* test: Fix message when skipping test_expect_equal* testsGravatar Austin Clements2011-09-12
| | | | | | For these types of tests, the test name is previously recorded in a variable, not passed to the test function, so pass this variable to test_skip.
* test: reset test_subtest_known_broken_ after each success/failure.Gravatar David Bremner2011-09-12
| | | | | | | | | | | This means that test_subtest_known_broken needs to be called before every known broken subtest, which is no different than what is documented for the test_begin_subtest case. The assumption is that every test ends up calling either skipping, calling test_ok_ or test_failure_ and and the latter in turn delegate to the known_broken versions in the case where test_subtest_known_broken_ is set.
* test: improve known broken tests supportGravatar Dmitry Kurochkin2011-09-10
| | | | | | | | | | | | | | | | | There is existing support for broken tests. But it is not convenient to use. The primary issue is that we have to maintain a set of test_expect_*_failure functions which are equivalent to the normal test_expect_* counterparts except for what functions are called for result reporting. The patch adds test_subtest_known_broken function which marks a subset as broken, making the normal test_expect_* functions behave as test_expect_*_failure. All test_expect_*_failure functions are removed. Test_known_broken_failure_ is changed to format details the same way as test_failure_ does. Another benefit of this change is that the diff when a broken test is fixed would be small and nice. Documentation is updated accordingly.
* test: Nix increment_mtime.Gravatar Austin Clements2011-06-29
| | | | | With the fix for the mtime race, this workaround is no longer necessary.
* test: use emacsclient(1) for Emacs testsGravatar Dmitry Kurochkin2011-06-28
| | | | | | | | | | | | | | | | | Before the change, every Emacs test ran in a separate Emacs instance. Starting Emacs many times wastes considerable time and it gets worse as the test suite grows. The patch solves this by using a single Emacs server and emacsclient(1) to run multiple tests. Emacs server is started on the first test_emacs call and stopped when test_done is called. We take care not to leave orphan Emacs processes behind when test is terminated by whatever reason: Emacs server runs a watchdog that periodically checks that the test is still running. Some tests need to provide user input. Before the change, this was done using echo(1) to Emacs stdin. This no longer works and instead `standard-input' variable is set accordingly to make `read' return the appropriate string.
* test: Increment mtime when delivering a message with emacs_deliver_messageGravatar Carl Worth2011-06-28
| | | | | | Without this, mail messages delivered by emacs_deliver_message might not be seen by the next invocation of "notmuch new", (which can lead to test-suite failures if emacs_deliver_message is fast enough).
* test:Improve test behaviors when --root is usedGravatar Mark Anderson2011-06-28
| | | | | | | | | | | Change add_email_corpus, emacs_deliver_message and tests to use $TEST_DIRECTORY instead of '..'. This improves the behavior of the usage of --root=<dir>, as the assumption of what '..' means will usually be incorrect. Document -root option in README and update valgrind to work with -root.
* test: generate run_emacs script once on test startupGravatar Dmitry Kurochkin2011-06-28
| | | | | | | Instead of generating auxiliary run_emacs script every time test_emacs is run, do it once in the beginning of the test. Also, use absolute paths in the script to make it more robust.
* test: set variables using `let' instead of `setq' in Emacs testsGravatar Dmitry Kurochkin2011-06-28
| | | | | | | | | Using `setq' for setting variables in Emacs tests affect other tests that may run in the same Emacs environment. Currently it works because each test is run in a separate Emacs instance. But in the future multiple tests will run in a single Emacs instance. The patch changes all variables to use `let', so the scope of the change is limited to a single test.
* test: wrap and indent test_emacs callsGravatar Dmitry Kurochkin2011-06-28
| | | | | | | Most test_emacs calls have long arguments that consist of many expressions. Putting them on a single line makes it hard to read and produces poor diff when they are changed. The patch puts every expression in test_emacs calls on a separate line.
* test: cleanup test_emacsGravatar Dmitry Kurochkin2011-06-28
| | | | | Move auxiliary function definition and configuration from command line to test-lib.el.
* test: do not set `message-signature' in test_emacsGravatar Dmitry Kurochkin2011-06-28
| | | | | It is no longer needed since tests are run in a temporary home directory instead of the user's one.
* fix sum moar typos [comments in source code]Gravatar Pieter Praet2011-06-23
| | | | | | | | | | Various typo fixes in comments within the source code. Signed-off-by: Pieter Praet <pieter@praet.org> Edited-by: Carl Worth <cworth@cworth.org> Restricted to just source-code comments, (and fixed fix of "descriptios" to "descriptors" rather than "descriptions").
* test: Create and set temporary home directoryGravatar Florian Friesdorf2011-06-01
| | | | | | | | | | | | | | In the master branch in test/emacs two tests access the build users home directory, so does emacs_deliver_message in the crypto branch. The tests should not touch the build user's home directory. The patch creates a directory in the temporary test directory and sets home accordingly. In case of a non-existent home directory, the tests are failing without this patch. Signed-off-by: Jameson Graef Rollins <jrollins@finestructure.net>
* test: copy files in test_expect_equal_file instead of moving themGravatar Dmitry Kurochkin2011-05-31
| | | | | | | | | | | Before the change, test_expect_equal_file moved files it compared in case of failure. The patch changes it to copy the files instead. This allows testing non-temporary files which are stored in git. Note: the change should not result in new temporary files left after the tests. Test_expect_equal_file used to move files only on failure, so callers had to cleanup them anyway.
* test: add crypto tests for signature verification and decryptionGravatar Jameson Graef Rollins2011-05-27
| | | | | | | | | | | | | | | | | | | This adds a new "crypto" test script to the test suite to test PGP/MIME signature verification and message decryption. Included here is a test GNUPGHOME with a test secret key (passwordless), and test for: * signing/verification * signing/verification with full owner trust * verification with signer key unavailable * encryption/decryption * decryption failure with missing key * encryption/decryption + signing/verfifying * reply to encrypted message * verification of signature from revoked key These tests are not expected to pass now, but will as crypto functionality is included.
* test: add notmuch_show_sanitize_all function that is a little more aggressive.Gravatar Jameson Graef Rollins2011-05-27
| | | | | | The old notmuch_show_sanitize function only scrubed part of the filename. This one scrubs the full filename, as well as the message id.
* test: new test-lib function to test for equality between filesGravatar Jameson Graef Rollins2011-05-27
| | | | | | | | | | We need to be able to test for the presence of a newline at the end of output. There's no good way to capture trailing newlines in bash, so redirecting output to a file is the next best thing. This new function should be used when testing for output that is expected to have trailing newlines. The next commit will demonstrate the use of this.
* test: change "#!/bin/bash" to "#!/usr/bin/env bash" enhances portabilityGravatar Joel Borggrén-Franck2011-05-27
| | | | | | Change #!/bin/bash at start of tests to "#!/usr/bin/env bash". That way systems running on bash < 4 can prepend bash >= 4 to path before running the tests.
* test: add test-lib.el file with `visible-buffer-string' functionGravatar Dmitry Kurochkin2011-05-26
| | | | | | | | | | | | | The patch adds test-lib.el file for Emacs tests auxiliary stuff. Currently, it implements two functions: `visible-buffer-string' and `visible-buffer-substring'. These are similar to standard counterparts without "visible-" prefix but exclude invisible text. The functions are not used anywhere at the moment but should be useful for testing hiding/showing in the Emacs interface. Edited-by: Carl Worth <cworth@cworth.org> Fixed "basic" test to ignore new test-lib.el file.
* Clarify usage of `additional_headers' in test/test-lib.sh:generate_message.Gravatar Thomas Schwinge2011-01-28
| | | | Signed-off-by: Thomas Schwinge <thomas@schwinge.name>
* Fix misspelling in a comment.Gravatar Carl Worth2011-01-15
| | | | Just a simple change I noticed in passing.
* test: Leave tmp.<testname> for broken testsGravatar Carl Worth2010-12-07
| | | | | | Previously, this directory was only preserved for failing tests. But it's important to be able to easily debug known-broken tests, so preserve the actual vs. expected output for those as well.
* Fix handling of broken tests.Gravatar Austin Clements2010-12-07
| | | | | Make sure to close the subtest for test_expect_equal_failure, just like in test_expect_equal.
* test: Detect unfinished subsetsGravatar Michal Sojka2010-11-16
| | | | | | When test_begin_subtest is not followed by corresponding test_expect_equal, the output of the rest of the test script is errornously suppressed. Add code to detect these bugs in test scripts.
* test: Add trailing newline to error messagesGravatar Michal Sojka2010-11-16
| | | | | | The newline was removed from say_color in commit 222926ab to allow printing test status in the beginning of the line. Error messages are never followed by other text so we add the newline to error function.
* test: Better handling of stdout and stderrGravatar Michal Sojka2010-11-16
| | | | | | | | | | | | | Git-style tests (test_expect_success etc.) suppress stdout and stderr unless -v is given. Notmuch-style tests (created by test_begin_subtest and test_expect_equal) do not have this behavior so implement it the same. Additionally, for both test styles, the test-lib.sh is changed so that the content of suppressed stdout and stderr is shown in case of failed test. Finally a test for this functionality is added to basic tests.
* Tests for maildir synchronizationGravatar Michal Sojka2010-11-10
| | | | Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
* test: Don't strip portions of test name after '-' for temporary files.Gravatar Carl Worth2010-11-10
| | | | | | | We have test names like maildir-sync now, so it's cleaner if the temporary files created are named things like maildir-sync-10.out rather than maildir-10.out. Presumably the extra stripping here came from naming conventions in git's test suite.
* test: Don't mess with user's HOME directoryGravatar Carl Worth2010-11-05
| | | | | | | | | | This was too rude of a thing to do and could easily introduce problems, (as reported by Rob Browning whose environment required some HOME-specific things for shell startup). Instead, implement more focused changes to ensure that particular file in $HOME don't cause problems. Specifically, we fix known problems with ~/.signature and ~/.mailrc here.
* test: Generate a little run_emacs script to help debug any failures.Gravatar Carl Worth2010-11-05
| | | | | | | | After any emacs test failure, the tmp.emacs directory will have this run_emacs script in it which the user can use to run emacs within the test suite environment, (pointing at the test suite's notmuch database, using the local notmuch command-line program, and the local notmuch emacs lisp code).
* test: Emit a friendly error message if run with bash < 4.0.Gravatar Carl Worth2010-10-28
| | | | | | | | | | The bash code in the test suite is using associative arrays which were only added to bash as of release 4.0. If the test suite is run with an older bash, we now immediately error out and explain the situation, (instead of emitting confusing error messages and failing dozens of tests, which is what happened before this change).
* test: Set alternate HOME during tests.Gravatar Carl Worth2010-10-27
| | | | | | We set the HOME environment variable to the test directory to avoid the tests relying on any configuration files from the test author's own home directory, (such as ${HOME}/.emacs or similar).
* test: Add simple tests for navigating notmuch-hello and notmuch-search viewsGravatar Carl Worth2010-10-22
| | | | | | | | | | | | | | | | | We simulate the act of selecting the "inbox" saved search from notmuch-hello and the act of selecting a desired thread from the notmuch-search results. The test for the navigation of notmuch-hello is currently marked as BROKEN since its output is in the opposite order compared to the '(notmuch-search "tag:inbox")' test. This question of ordering is a currently open issue on the notmuch mailing list, so we'll let the test suite reflect that for now. Finally, this commit also abstracts some common emacs lisp code, (waiting for the current buffer's process to complete), into a new notmuch-test-wait function that is made available to anything calling test_emacs.
* test: Add a new test_expect_equal_failureGravatar Carl Worth2010-10-22
| | | | | Which allows us to have a known-broken test that would otherwise use test_expect_equal.
* test: Add a new test_emacs function to test-lib.shGravatar Carl Worth2010-10-22
| | | | | | | | This should be quite handy for doing automated testing of the emacs-based functionality in notmuch. This function invokes emacs with the necessary command-line arguments, (to run in batch mode with no local initialization, to load the notmuch code from the source directory, and to ensure an 80-column width).
* test: Fix add_email_corpus function to be quiet.Gravatar Carl Worth2010-10-22
| | | | This simply avoids some unneeded noise in the "make test" output.
* test: Fix the search and dump-restore tests to operator on non-empty mail store.Gravatar Carl Worth2010-09-20
| | | | | We do this with a new add_email_corpus function that establishes a mail store with 50 messages from the notmuch mailing list.
* test: Remove useless NOTMUCH variable (in favor of simply "notmuch")Gravatar Carl Worth2010-09-20
| | | | | | | | | | When the NOTMUCH variable was originally invented it was used as an explicit path to the notmuch binary being tested. Today, the test suite sets the PATH variable instead, so the NOTMUCH variable always has a value of simply "notmuch". We simplifying that by using the constant value rather than the continual variable reference.
* test: Remove some dead code in test-lib.shGravatar Carl Worth2010-09-20
| | | | | These assignements weren't being used at all and were just confusing me, (the real assignments happen later on in the file).
* test: Print section names, and rename all test sectionsGravatar Carl Worth2010-09-20
| | | | | | | | Now that we can usefully pass section names via the NOTMUCH_SKIP_TESTS environment variable, it's useful to actually print those names out for the user. Then, since we're now printing these names, let's use nicer names, (not excessively long but also not using abbreviations like "msg").
* test: Make the --valgrind option useful, (and drop --verbose).Gravatar Carl Worth2010-09-20
| | | | | | | | | In order for --valgrind to be useful, we drop noisy additional output of all of the commands being executed in verbose mode. This makes --verbose alone quite useless, so we don't document it any more. Also, add a zlib valgrind suppression that was showing up frequently in the test suite.
* test: Rename GIT_SKIP_TESTS to NOTMUCH_SKIP_TESTSGravatar Carl Worth2010-09-20
| | | | | | | | | By scanning test-lib.sh for occurrences of "git" or "GIT", I found that most of those are internal things, (like the GIT_TEST_TEE_STARTED variable). But GIT_SKIP_TESTS is part of the user-interface to the test suite, so we rename it to reference notmuch rather than git. Also, the GIT_TRACE warning is git-specific, so we drop that as well.
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-20 04:33:46 +0000