From fbbb379fe08b051b40c071041108a2526533f417 Mon Sep 17 00:00:00 2001 From: Alexis Hildebrandt Date: Fri, 7 Feb 2014 00:21:38 +0100 Subject: Check examples in documentation when running tests The DocTests.py script will parse a given texinfo file for specially marked examples, run the ledger command from the example, and check the result against the example output from the documentation. --- doc/ledger3.texi | 75 +++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 52 insertions(+), 23 deletions(-) (limited to 'doc/ledger3.texi') diff --git a/doc/ledger3.texi b/doc/ledger3.texi index c85f8446..acf58698 100644 --- a/doc/ledger3.texi +++ b/doc/ledger3.texi @@ -346,13 +346,13 @@ for each. To find the balances of all of your accounts, run this command: -@smallexample +@smallexample @c command:1071890 $ ledger -f drewr3.dat balance @end smallexample Ledger will generate: -@smallexample +@smallexample @c output:1071890 $ -3,804.00 Assets $ 1,396.00 Checking $ 30.00 Business @@ -381,8 +381,11 @@ pare this down to show only the accounts you want. A more useful report is to show only your Assets and Liabilities: -@smallexample +@smallexample @c command:5BF4D8E $ ledger -f drewr3.dat balance Assets Liabilities +@end smallexample + +@smallexample @c output:5BF4D8E $ -3,804.00 Assets $ 1,396.00 Checking $ 30.00 Business @@ -402,16 +405,16 @@ $ ledger -f drewr3.dat balance Assets Liabilities To show all transactions and a running total: -@smallexample +@smallexample @c command:66E3A2C $ ledger -f drewr3.dat register @end smallexample @noindent Ledger will generate: -@smallexample +@smallexample @c output:66E3A2C 10-Dec-01 Checking balance Assets:Checking $ 1,000.00 $ 1,000.00 - Equity:Opening Balances $ -1,000.00 0 + Equit:Opening Balances $ -1,000.00 0 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 112.50 @@ -450,8 +453,11 @@ interested in seeing transactions for: @cindex accounts, limiting by @cindex limiting by accounts -@smallexample +@smallexample @c command:96B0EB3 $ ledger -f drewr3.dat register Groceries +@end smallexample + +@smallexample @c output:96B0EB3 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 112.50 @@ -465,8 +471,11 @@ $ ledger -f drewr3.dat register Groceries @noindent Which matches the balance reported for the @samp{Groceries} account: -@smallexample +@smallexample @c command:AECD64E $ ledger -f drewr3.dat balance Groceries +@end smallexample + +@smallexample @c output:AECD64E $ 334.00 Expenses:Food:Groceries @end smallexample @@ -474,8 +483,11 @@ $ ledger -f drewr3.dat balance Groceries If you would like to find transaction to only a certain payee use @samp{payee} or @samp{@@}: -@smallexample +@smallexample @c command:C6BC57E $ ledger -f drewr3.dat register payee "Organic" +@end smallexample + +@smallexample @c output:C10BC57E 10-Dec-20 Organic Co-op Expense:Food:Groceries $ 37.50 $ 37.50 Expense:Food:Groceries $ 37.50 $ 75.00 Expense:Food:Groceries $ 37.50 $ 112.50 @@ -497,8 +509,11 @@ a check to clear, but you should treat it as money spent. The @command{cleared} report will not format correctly for accounts that contain multiple commodities): -@smallexample +@smallexample @c command:B86F6A6 $ ledger -f drewr3.dat cleared +@end smallexample + +@smallexample @c output:B86F6A6 $ -3,804.00 $ 775.00 Assets $ 1,396.00 $ 775.00 10-Dec-20 Checking $ 30.00 0 Business @@ -517,8 +532,8 @@ $ ledger -f drewr3.dat cleared $ -20.00 0 MasterCard $ 200.00 0 Mortgage:Principal $ -243.60 0 Tithe ----------------- ---------------- --------- - $ -243.60 0 +---------------- ---------------- --------- + $ -243.60 0 @end smallexample @noindent @@ -3826,8 +3841,11 @@ note the implicit logical and between @samp{Auto} and If you want the entire contents of a branch of your account tree, use the highest common name in the branch: -@smallexample +@smallexample @c command:B0468E1 $ ledger balance -f drewr3.dat Income +@end smallexample + +@smallexample @c output:B0468E1 $ -2,030.00 Income $ -2,000.00 Salary $ -30.00 Sales @@ -3838,15 +3856,25 @@ $ ledger balance -f drewr3.dat Income You can use general regular expressions in nearly anyplace Ledger needs a string: -@smallexample +@smallexample @c command:EAE389F $ ledger balance -f drewr3.dat ^Bo +@end smallexample +@smallexample @c output:EAE389F +@end smallexample + +This first example looks for any account starting with @samp{Bo}, of +which there are none. + +@smallexample @c command:E2AF6AD $ ledger balance -f drewr3.dat Bo +@end smallexample + +@smallexample @c output:E2AF6AD $ 20.00 Expenses:Books @end smallexample -The first example looks for any account starting with @samp{Bo}, of -which there are none. The second looks for any account with @samp{Bo}, -which is @samp{Expenses:Books}. +This second example looks for any account with @samp{Bo}, which is +@samp{Expenses:Books}. @cindex limit by payees @findex --limit @var{EXPR} @@ -5596,22 +5624,23 @@ Format Codes}). @item --master-account @var{STR} Prepend all account names with the argument. -@smallexample -$ ledger -f test/input/drewr3.dat bal --master-account HUMBUG +@smallexample @c command:A76BB56 +$ ledger -f drewr3.dat bal --no-total --master-account HUMBUG +@end smallexample + +@smallexample @c output:A76BB56 0 HUMBUG $ -3,804.00 Assets $ 1,396.00 Checking $ 30.00 Business $ -5,200.00 Savings - $ 20.00 Books $ -1,000.00 Equity:Opening Balances - $ 6,634.00 Expenses - $ 11,000.00 Auto + $ 6,654.00 Expenses + $ 5,500.00 Auto $ 20.00 Books $ 300.00 Escrow $ 334.00 Food:Groceries $ 500.00 Interest:Mortgage - $ -5,520.00 Assets:Checking $ -2,030.00 Income $ -2,000.00 Salary $ -30.00 Sales -- cgit v1.2.3 From a1cc8ca15ae1761fbcb4c6ec9f8af82e8ab411ab Mon Sep 17 00:00:00 2001 From: Alexis Hildebrandt Date: Fri, 7 Feb 2014 18:38:43 +0100 Subject: Add support to check documentation examples with inline data --- doc/ledger3.texi | 58 ++++++++++++++++++++++++++++++++------------------------ test/DocTests.py | 45 ++++++++++++++++++++++++++++++++++--------- 2 files changed, 69 insertions(+), 34 deletions(-) (limited to 'doc/ledger3.texi') diff --git a/doc/ledger3.texi b/doc/ledger3.texi index acf58698..48fafc13 100644 --- a/doc/ledger3.texi +++ b/doc/ledger3.texi @@ -1503,7 +1503,7 @@ entry. For example, the following entries reflect transaction made for a business trip to Europe from the US: -@smallexample +@smallexample @c input:82150D9 2011/09/23 Cash in Munich Assets:Cash E50.00 Assets:Checking $-66.00 @@ -1519,8 +1519,11 @@ spent on Dinner in Munich. Running a ledger balance report shows: -@smallexample +@smallexample @c command:82150D9 $ ledger -f example.dat bal +@end smallexample + +@smallexample @c output:82150D9 $-66.00 E15.00 Assets E15.00 Cash @@ -3641,7 +3644,7 @@ the money to be evenly distributed over the next six months so that your monthly budgets gradually take a hit for the vegetables you'll pick up from the co-op, even though you've already paid for them. -@smallexample +@smallexample @c input:6453542 2008/10/16 * (2090) Bountiful Blessings Farm Expenses:Food:Groceries $ 37.50 ; [=2008/10/01] Expenses:Food:Groceries $ 37.50 ; [=2008/11/01] @@ -3659,15 +3662,17 @@ really knows that it debited $225 this month. And using @option{--effective} option, initial date will be overridden by effective dates. -@smallexample +@smallexample @c command:6453542 $ ledger --effective register Groceries +@end smallexample -08-Oct-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 37.50 -08-Nov-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 75.00 -08-Dec-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 112.50 -09-Jan-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 150.00 -09-Feb-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 187.50 -09-Mar-01 Bountiful Blessi.. Expe:Food:Groceries $ 37.50 $ 225.00 +@smallexample @c output:6453542 +08-Oct-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 37.50 +08-Nov-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 75.00 +08-Dec-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 112.50 +09-Jan-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 150.00 +09-Feb-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 187.50 +09-Mar-01 Bountiful Blessings.. Expense:Food:Groceries $ 37.50 $ 225.00 @end smallexample @node Periodic Transactions, Concrete Example of Automated Transactions, Effective Dates, Automated Transactions @@ -3792,14 +3797,14 @@ options. The balance report is the most commonly used report. The simplest invocation is: -@smallexample +@smallexample @c command:1D00D56 $ ledger balance -f drewr3.dat @end smallexample @noindent which will print the balances of every account in your journal. -@smallexample +@smallexample @c output:1D00D56 $ -3,804.00 Assets $ 1,396.00 Checking $ 30.00 Business @@ -3826,8 +3831,11 @@ Most times this is more than you want. Limiting the results to specific accounts is as easy as entering the names of the accounts after the command. -@smallexample +@smallexample @c command:06B2AD4 $ ledger balance -f drewr3.dat Auto MasterCard +@end smallexample + +@smallexample @c output:06B2AD4 $ 5,500.00 Expenses:Auto $ -20.00 Liabilities:MasterCard -------------------- @@ -4992,7 +5000,7 @@ earlier postings. Here's how it works: Say you currently have this posting in your ledger file: -@smallexample +@smallexample @c input:03ACB97 2004/03/15 * Viva Italiano Expenses:Food $12.45 Expenses:Tips $2.55 @@ -5003,17 +5011,17 @@ Now it's @samp{2004/4/9}, and you've just eating at @samp{Viva Italiano} again. The exact amounts are different, but the overall form is the same. With the @command{xact} command you can type: -@smallexample +@smallexample @c command:03ACB97 $ ledger xact 2004/4/9 viva food 11 tips 2.50 @end smallexample This produces the following output: -@smallexample +@smallexample @c output:03ACB97 2004/04/09 Viva Italiano - Expenses:Food $11.00 - Expenses:Tips $2.50 - Liabilities:MasterCard $-13.50 + Expenses:Food $11.00 + Expenses:Tips $2.50 + Liabilities:MasterCard @end smallexample It works by finding a past posting matching the regular expression @@ -6491,7 +6499,7 @@ In the balance report, it shows all the accounts affected by transactions having a related posting. For example, if a file had this transaction: -@smallexample +@smallexample @c input:94C5675 2004/03/20 Safeway Expenses:Food $65.00 Expenses:Cash $20.00 @@ -6500,16 +6508,16 @@ this transaction: And the register command was: -@smallexample -$ ledger -r register food +@smallexample @c command:94C5675 +$ ledger -f example.dat -r register food @end smallexample The following would be output, showing the postings related to the posting that matched: -@smallexample -2004/03/20 Safeway Expenses:Cash $-20.00 $-20.00 - Assets:Checking $85.00 $65.00 +@smallexample @c output:94C5675 +04-Mar-20 Safeway Expenses:Cash $20.00 $20.00 + Assets:Checking $-85.00 $-65.00 @end smallexample @item --budget diff --git a/test/DocTests.py b/test/DocTests.py index daac1db5..eb1a0205 100755 --- a/test/DocTests.py +++ b/test/DocTests.py @@ -26,6 +26,8 @@ class DocTests: self.examples = dict() self.testin_token = 'command' self.testout_token = 'output' + self.testdat_token = 'input' + self.test_files = list() def read_example(self): endexample = re.compile(r'^@end\s+smallexample\s*$') @@ -41,7 +43,8 @@ class DocTests: return hashlib.sha1(example.rstrip()).hexdigest()[0:7].upper() def find_examples(self): - startexample = re.compile(r'^@smallexample\s+@c\s+(%s|%s)(?::([\dA-Fa-f]+))?' % (self.testin_token, self.testout_token)) + startexample = re.compile(r'^@smallexample\s+@c\s+(%s|%s|%s)(?::([\dA-Fa-f]+))?' + % (self.testin_token, self.testout_token, self.testdat_token)) while True: line = self.file.readline() self.current_line += 1 @@ -87,11 +90,17 @@ class DocTests: command = example[self.testin_token][self.testin_token] except KeyError: command = None + try: output = example[self.testout_token][self.testout_token] except KeyError: output = None + try: + input = example[self.testdat_token][self.testdat_token] + except KeyError: + input = None + if command and output: command = command.rstrip().split() if command[0] == '$': command.remove('$') @@ -99,19 +108,37 @@ class DocTests: command[index] = self.ledger command.insert(index+1, '--init-file') command.insert(index+2, '/dev/null') - scriptpath = os.path.dirname(os.path.realpath(__file__)) - test_input_dir = scriptpath + '/../test/input/' - for i, arg in enumerate(command): - if '.dat' in arg or '.ledger' in arg: - if os.path.exists(test_input_dir + arg): - command[i] = test_input_dir + arg try: - verify = subprocess.check_output(command)#.decode('utf-8') + findex = command.index('-f') + except ValueError: + try: + findex = command.index('--file') + except ValueError: + findex = index+1 + command.insert(findex, '--file') + command.insert(findex+1, test_id + '.dat') + + if findex: + scriptpath = os.path.dirname(os.path.realpath(__file__)) + test_input_dir = scriptpath + '/../test/input/' + test_file = command[findex+1] + test_file_created = False + if not os.path.exists(test_file): + if input: + test_file_created = True + with open(test_file, 'w') as f: + f.write(input) + elif os.path.exists(test_input_dir + test_file): + command[findex+1] = test_input_dir + test_file + try: + verify = subprocess.check_output(command) except: verify = str() + if test_file_created: + os.remove(test_file) valid = (output == verify) if self.verbose: - print test_id, ':', u'Passed' if valid else u'FAILED' + print test_id, ':', 'Passed' if valid else 'FAILED' else: sys.stdout.write('.' if valid else 'E') -- cgit v1.2.3 From a2f86c85df7ac9f00facefbc9318e2a06e41b73b Mon Sep 17 00:00:00 2001 From: Alexis Hildebrandt Date: Sun, 9 Feb 2014 07:54:27 +0100 Subject: Add explanation on how to validate documentation examples by specially marking @smallexample, which will be used by DocTests.py --- doc/ledger3.texi | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) (limited to 'doc/ledger3.texi') diff --git a/doc/ledger3.texi b/doc/ledger3.texi index 48fafc13..425dd5bc 100644 --- a/doc/ledger3.texi +++ b/doc/ledger3.texi @@ -20,6 +20,67 @@ @c Restructuring manual ideas @c http://beyondgrep.com/documentation/ack-2.04-man.html +@c How to make documented ledger examples validate automatically. +@c +@c The test/DocTests.py script will be run along the with the other +@c tests when using ctest or acprep check. +@c The script parses the texinfo file and looks for three kinds of +@c specially marked @smallexamples, then it will run the ledger +@c command from the exmaple, and compare the results with the output +@c from the documentation. +@c +@c To specially mark a @smallexample append @c command:UUID, where +@c UUID is the first 7 digits from the commands sha1sum, e.g.: +@c +@c @smallexample @c command:CDE330A +@c $ ledger -f sample.dat reg expenses +@c @end smallexample +@c +@c Then DocTests.py will look for corresponding documented output, +@c which may appear anywhere in the file, and is marked with +@c @smallexample @c output:UUID where UUID is the UUID from the +@c corresponding ledger command example, e.g.: +@c +@c @smallexample @c output:CDE330A +@c 04-May-27 Book Store Expenses:Books $20.00 $20.00 +@c Expenses:Cards $40.00 $60.00 +@c Expenses:Docs $30.00 $90.0 +@c @end smallexample +@c +@c Now where does this data in sample.dat come from? +@c DocTests.py is a bit smart about ledger's file argument, since +@c it will check if the given filename exists in the test/input/ +@c directory. +@c +@c Sometimes the journal data for an example is specified within +@c the documentation itself, in that case the journal example data +@c needs to be specially marked as well using @smallexample @c input:UUID, +@c again with the UUID being the UUID of the corresponding ledger example +@c command, e.g.: +@c +@c @smallexample @c input:35CB2A3 +@c 2014/02/09 The Italian Place +@c Expenses:Food:Dining $ 36.84 +@c Assets:Cash +@c @end smallexample +@c +@c @smallexample @c command:35CB2A3 +@c $ ledger -f inline.dat accounts +@c @end smallexample +@c +@c @smallexample @c output:35CB2A3 +@c Assets:Cash +@c Expenses:Food:Dining +@c @end smallexample +@c +@c Additionally DocTests.py will pass --init-file /dev/null to ledger to +@c ignore any default arguments to ledger the user running the tests +@c has configured. +@c +@c To manually run the tests in this file run: +@c $ ./test/DocTests.py -vv --ledger ./ledger --file ./test/ledger3.texi + + @copying Copyright @copyright{} 2003–2014, John Wiegley. All rights reserved. -- cgit v1.2.3