Disallow --nominal with GC (#4758)

Nominal types don't make much sense without GC, and in particular trying to emit
them with typed function references but not GC enabled can result in invalid
binaries because nominal types do not respect the type ordering constraints
required by the typed function references proposal. Making this change was
mostly straightforward, but required fixing the fuzzer to use --nominal only
when GC is enabled and required exiting early from nominal-only optimizations
when GC was not enabled.

Fixes #4756.

1 files changed, 3 insertions, 18 deletions

diff --git a/test/lit/nominal-no-gc.wast b/test/lit/nominal-no-gc.wast
index 7247caa5a..82f2698d9 100644
--- a/test/lit/nominal-no-gc.wast
+++ b/test/lit/nominal-no-gc.wast
@@ -1,20 +1,5 @@
-;; Write the module with --nominal but without GC
-;; RUN: wasm-opt %s --nominal --disable-gc -g -o %t.wasm
+;; Using --nominal without GC is not allowed.
 
-;; We should not get any recursion groups even though we used --nominal. We use
-;; --hybrid -all here to make sure that any rec groups from the binary will
-;; actually show up in the output and cause the test to fail.
-;; RUN: wasm-opt %t.wasm --hybrid -all -S -o - | filecheck %s
+;; RUN: not wasm-opt %s --nominal --disable-gc -g -o %t.wasm 2>&1 | filecheck %s
 
-;; Also check that we don't get a failure with the default configuration.
-;; RUN: wasm-opt %t.wasm
-
-;; CHECK-NOT: rec
-
-(module
-  (type $foo (func))
-  (type $bar (func))
-
-  (func $f1 (type $foo))
-  (func $f2 (type $bar))
-)
+;; CHECK: Nominal typing is only allowed when GC is enabled