|
|
This fixes #1626.
The tokenizer eagerly classifies 4-digit integers as TOK_A_YEAR
tokens. In some contexts such as "every 1000 years", this causes
errors.
I think the tokenizer does not have enough information available to
distinguish between integers and years.
After this patch, the tokenizer will always classify integers as
TOK_INT tokens. The "has 4 digits" heuristic to determine if an
integer is a year is moved to the place where it's actually
needed (and it can be slightly more generic there, too).
|
