Fix vacuum on rtts with depth (#4327)

Found by the fuzzer. Calling makeZero on an rtt with depth will
error because we try to create a zero Literal from it, and we can't
do that - we don't know a list of super types to give it. We could
work around it, but we don't want to: if the rtt has depth then we
can't make a nice zero for it, we'd need some rtt.subs anyhow,
so simply mark it as a type we can't make a zero for.

1 files changed, 9 insertions, 0 deletions

diff --git a/src/ir/literal-utils.h b/src/ir/literal-utils.h
index ce7da69cd..21ade6d84 100644
--- a/src/ir/literal-utils.h
+++ b/src/ir/literal-utils.h
@@ -35,6 +35,14 @@ inline bool canMakeZero(Type type) {
   if (type.isNonNullable()) {
     return false;
   }
+  if (type.isRtt() && type.getRtt().hasDepth()) {
+    // An rtt with depth cannot be constructed as a simple zero: we'd need to
+    // create not just a zero (an rtt.canon) but also some rtt.subs that add to
+    // the depth, so disallow that. Also, there is no practical way to create a
+    // zero Literal for such a type, as we'd need to supply the list of super
+    // types somehow, and creating a zero Literal is how makeZero works.
+    return false;
+  }
   if (type.isTuple()) {
     for (auto t : type) {
       if (!canMakeZero(t)) {
@@ -46,6 +54,7 @@ inline bool canMakeZero(Type type) {
 }
 
 inline Expression* makeZero(Type type, Module& wasm) {
+  assert(canMakeZero(type));
   // TODO: Remove this function once V8 supports v128.const
   // (https://bugs.chromium.org/p/v8/issues/detail?id=8460)
   Builder builder(wasm);