Fix type canonicalization bugs (#3761)

When canonical heap types were already present in the global store, for example
during the --roundtrip pass, type canonicalization was not working correctly.
The issue was that the GlobalCanonicalizer was replacing temporary HeapTypes
with their canonical equivalents one type at a time, but the act of replacing a
temporary HeapType use with a canonical HeapType use could change the shape of
later HeapTypes, preventing them from being correctly matched with their
canonical counterparts. This PR fixes that problem by computing all the
temporary-to-canonical heap type replacements before executing them.

To avoid a similar problem when canonicalizing Types, one solution would have
been to pre-calculate the replacements before executing them just like with the
HeapTypes, but that would have required either complex bookkeeping or moving
temporary Types into the global store when they are first canonicalized. That
would have been complicated because unlike for temporary HeapTypeInfos, the
unique_pointer to temporary TypeInfos is not readily available. This PR instead
switches back to using pointer-identity based equality and hashing for
TypeInfos, which works because we only ever canonicalize Types with canonical
children. This change should be a nice performance improvement as well.

Another bug this PR fixes is that shape hashing and equality considered
BasicKind HeapTypes to be different from their corresponding BasicHeapTypes,
which meant that canonicalization could produce different types for the same
type definition depending on whether the definition used a TypeBuilder or not.
The fix is to pre-canonicalize BasicHeapTypes (and Types that have them as
children) during shape hashing and equality. The same mechanism is also used to
simplify Store's canonicalization.

Fixes #3736.

0 files changed, 0 insertions, 0 deletions