diff options
Diffstat (limited to 'doc/misc/ert.texi')
| -rw-r--r-- | doc/misc/ert.texi | 168 |
1 files changed, 163 insertions, 5 deletions
diff --git a/doc/misc/ert.texi b/doc/misc/ert.texi index aeae6aef8c7..0d01efb0355 100644 --- a/doc/misc/ert.texi +++ b/doc/misc/ert.texi @@ -109,6 +109,7 @@ Appendix @end menu @end ifnottex + @node Introduction @chapter Introduction @cindex introduction to ERT @@ -123,7 +124,7 @@ commands to run them to verify whether the definitions that are currently loaded in Emacs pass the tests. Some Lisp files have comments like the following (adapted from the -package @code{pp.el}): +package @file{pp.el}): @lisp ;; (pp-to-string '(quote quote)) ; expected: "'quote" @@ -358,6 +359,7 @@ Prompt for a test and then show its documentation. @end table + @node Running Tests in Batch Mode @section Running Tests in Batch Mode @cindex running tests in batch mode @@ -375,7 +377,7 @@ emacs -batch -l ert -l my-tests.el -f ert-run-tests-batch-and-exit @end example This command will start up Emacs in batch mode, load ERT, load -@code{my-tests.el}, and run all tests defined in it. It will exit +@file{my-tests.el}, and run all tests defined in it. It will exit with a zero exit status if all tests passed, or nonzero if any tests failed or if anything else went wrong. It will also print progress messages and error diagnostics to standard output. @@ -390,12 +392,37 @@ summary as shown below: emacs -batch -l ert -f ert-summarize-tests-batch-and-exit output.log @end example +@vindex ert-batch-print-level +@vindex ert-batch-print-length +ERT attempts to limit the output size for failed tests by choosing +conservative values for @code{print-level} and @code{print-length} +when printing Lisp values. This can in some cases make it difficult +to see which portions of those values are incorrect. Use +@code{ert-batch-print-level} and @code{ert-batch-print-length} +to customize that: + +@example +emacs -batch -l ert -l my-tests.el \ + --eval "(let ((ert-batch-print-level 10) \ + (ert-batch-print-length 120)) \ + (ert-run-tests-batch-and-exit))" +@end example + +@vindex ert-batch-backtrace-line-length +Even modest settings for @code{print-level} and @code{print-length} can +produce extremely long lines in backtraces, however, with attendant +pauses in execution progress. Set +@code{ert-batch-backtrace-line-length} to t to use the value of +@code{backtrace-line-length}, @code{nil} to stop any limitations on backtrace +line lengths (that is, to get full backtraces), or a positive integer to +limit backtrace line length to that number. + @vindex ert-quiet By default, ERT in batch mode is quite verbose, printing a line with result after each test. This gives you progress information: how many tests have been executed and how many there are. However, in some cases this much output may be undesirable. In this case, set -@code{ert-quiet} variable to a non-nil value: +@code{ert-quiet} variable to a non-@code{nil} value: @example emacs -batch -l ert -l my-tests.el \ @@ -414,10 +441,21 @@ emacs -batch -l ert -l my-tests.el \ -eval '(ert-run-tests-batch-and-exit "to-match")' @end example +@vindex EMACS_TEST_VERBOSE@r{, environment variable} By default, ERT test failure summaries are quite brief in batch mode---only the names of the failed tests are listed. If the -EMACS_TEST_VERBOSE environment variable is set, the failure summaries -will also include the data from the failing test. +@env{EMACS_TEST_VERBOSE} environment variable is set, the failure +summaries will also include the data from the failing test. + +@vindex EMACS_TEST_JUNIT_REPORT@r{, environment variable} +ERT can produce JUnit test reports in batch mode. If the environment +variable @env{EMACS_TEST_JUNIT_REPORT} is set, ERT will produce for +every test package @file{my-tests.el} a corresponding JUnit test +report @file{my-tests.xml}. The function +@code{ert-summarize-tests-batch-and-exit} collects all these package +test reports into a new JUnit test report, with the respective name of +that environment variable. + @node Test Selectors @section Test Selectors @@ -486,8 +524,10 @@ to find where a test was defined if the test was loaded from a file. * Expected Failures:: Tests for known bugs. * Tests and Their Environment:: Don't depend on customizations; no side effects. * Useful Techniques:: Some examples. +* erts files:: Files containing many buffer tests. @end menu + @node The @code{should} Macro @section The @code{should} Macro @@ -768,6 +808,121 @@ for testing. Usually, this makes the interfaces easier to use as well. +@node erts files +@section erts files + +@findex ert-test-erts-file +Many relevant Emacs tests depend on comparing the contents of a buffer +before and after executing a particular function. These tests can be +written the normal way---making a temporary buffer, inserting the +``before'' text, running the function, and then comparing with the +expected ``after'' text. However, this often leads to test code +that's pretty difficult to read and write, especially when the text in +question is multi-line. + +So ert provides a function called @code{ert-test-erts-file} that takes +two parameters: The name of a specially-formatted @dfn{erts} file, and +(optionally) a function that performs the transform. + +@findex erts-mode +These erts files can be edited with the @code{erts-mode} major mode. + +An erts file is divided into sections by the (@samp{=-=}) separator. + +Here's an example file containing two tests: + +@example +Name: flet + +=-= +(cl-flet ((bla (x) +(* x x))) +(bla 42)) +=-= +(cl-flet ((bla (x) + (* x x))) + (bla 42)) +=-=-= + +Name: defun + +=-= +(defun x () + (print (quote ( thingy great + stuff)))) +=-=-= +@end example + +A test starts with a line containing just @samp{=-=} and ends with a +line containing just @samp{=-=-=}. The test may be preceded by +freeform text (for instance, comments), and also name/value pairs (see +below for a list of them). + +If there is a line with @samp{=-=} inside the test, that designates +the start of the ``after'' text. Otherwise, the ``before'' and +``after'' texts are assumed to be identical, which you typically see +when writing indentation tests. + +@code{ert-test-erts-file} puts the ``before'' section into a temporary +buffer, calls the transform function, and then compares with the +``after'' section. + +Here's an example usage: + +@lisp +(ert-test-erts-file "elisp.erts" + (lambda () + (emacs-lisp-mode) + (indent-region (point-min) (point-max)))) +@end lisp + +A list of the name/value specifications that can appear before a test +follows. The general syntax is @samp{Name: Value}, but continuation +lines can be used (along the same lines as in mail---subsequent lines +that start with a space are part of the value). + +@example +Name: foo +Code: (indent-region + (point-min) (point-max)) +@end example + +@table @samp +@item Name +All tests should have a name. This name will appear in ERT output if +the test fails, and helps to identify the failing test. + +@item Code +This is the code that will be run to do the transform. This can also +be passed in via the @code{ert-test-erts-file} call, but @samp{Code} +overrides that. It's used not only in the following test, but in all +subsequent tests in the file (until overridden by another @samp{Code} +specification). + +@item No-Before-Newline +@itemx No-After-Newline +These specifications say whether the ``before'' or ``after'' portions +have a newline at the end. (This would otherwise be impossible to +specify.) + +@item Point-Char +Sometimes it's useful to be able to put point at a specific place +before executing the transform function. @samp{Point-Char: |} will +make @code{ert-test-erts-file} place point where @samp{|} is in the +``before'' form (and remove that character), and will check that it's +where the @samp{|} character is in the ``after'' form (and issue a +test failure if that isn't the case). (This is used in all subsequent +tests, unless overridden by a new @samp{Point-Char} spec.) + +@item Skip +If this is present and value is a form that evaluates to a +non-@code{nil} value, the test will be skipped. +@end table + +If you need to use the literal line single line @samp{=-=} in a test +section, you can quote it with a @samp{\} character. + + @node How to Debug Tests @chapter How to Debug Tests @@ -969,6 +1124,7 @@ For information on mocks, stubs, fixtures, or test suites, see below. * Fixtures and Test Suites:: How ERT differs from tools for other languages. @end menu + @node Mocks and Stubs @section Other Tools for Emacs Lisp @cindex mocks and stubs @@ -1043,11 +1199,13 @@ e.g., to run quick tests during interactive development and slow tests less often. This can be achieved with the @code{:tag} argument to @code{ert-deftest} and @code{tag} test selectors. + @node Index @unnumbered Index @printindex cp + @node GNU Free Documentation License @appendix GNU Free Documentation License @include doclicense.texi |