From 86a23cd263a2e67351a3d748ffc69d65f4746184 Mon Sep 17 00:00:00 2001 From: Christoph Dittmann Date: Sun, 10 Jun 2018 13:03:53 +0100 Subject: Remove TOK_A_YEAR token This fixes #1626. The tokenizer eagerly classifies 4-digit integers as TOK_A_YEAR tokens. In some contexts such as "every 1000 years", this causes errors. I think the tokenizer does not have enough information available to distinguish between integers and years. After this patch, the tokenizer will always classify integers as TOK_INT tokens. The "has 4 digits" heuristic to determine if an integer is a year is moved to the place where it's actually needed (and it can be slightly more generic there, too). --- test/regress/BBFA1759.test | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'test/regress/BBFA1759.test') diff --git a/test/regress/BBFA1759.test b/test/regress/BBFA1759.test index 7a402d0c..5df7ecb2 100644 --- a/test/regress/BBFA1759.test +++ b/test/regress/BBFA1759.test @@ -2,7 +2,7 @@ test period june 2008 --- Period expression tokens --- TOK_A_MONTH: Jun -TOK_A_YEAR: 2008 +TOK_INT: 2008 END_REACHED: --- Before stabilization --- -- cgit v1.2.3