Remove TOK_A_YEAR token

This fixes #1626.

The tokenizer eagerly classifies 4-digit integers as TOK_A_YEAR
tokens.  In some contexts such as "every 1000 years", this causes
errors.

I think the tokenizer does not have enough information available to
distinguish between integers and years.

After this patch, the tokenizer will always classify integers as
TOK_INT tokens.  The "has 4 digits" heuristic to determine if an
integer is a year is moved to the place where it's actually
needed (and it can be slightly more generic there, too).

1 files changed, 28 insertions, 0 deletions

diff --git a/test/regress/1626.test b/test/regress/1626.test
new file mode 100644
index 00000000..89ae80f8
--- /dev/null
+++ b/test/regress/1626.test
@@ -0,0 +1,28 @@
+test period every 1000 years from 1 Sep 2011 to 30 May 2012 --now=2018-06-10
+--- Period expression tokens ---
+TOK_EVERY: every
+TOK_INT: 1000
+TOK_YEARS: years
+TOK_SINCE: since
+TOK_INT: 1
+TOK_A_MONTH: Sep
+TOK_INT: 2011
+TOK_UNTIL: until
+TOK_INT: 30
+TOK_A_MONTH: May
+TOK_INT: 2012
+END_REACHED: <EOF>
+
+--- Before stabilization ---
+   range: from day 1 to day 30
+duration: 1000 years
+
+--- After stabilization ---
+   range: from day 1 to day 30
+   start: 18-Jan-01
+  finish: 18-Jan-30
+duration: 1000 years
+
+--- Sample dates in range (max. 20) ---
+ 1: 18-Jan-01 -- 18-Jan-29
+end test