Change the Literal class's operator== to be bitwise (#1661)

The change means that nan values will be compared bitwise when writing A == B, and so the float rule of a nan is different from itself would not apply.

I think this is a safer default. In particular this PR fixes a fuzz bug in the rse pass, which placed Literals in a hash table, and due to nan != nan, an infinite loop... Also, looks like we really want a bitwise comparison pretty much everywhere anyhow, as can be seen in the diff here. Really the single place we need a floaty comparison is in the intepreter where we implement f32.eq etc., and there the code was already using the proper code path anyhow.

1 files changed, 2 insertions, 14 deletions

diff --git a/src/wasm/literal.cpp b/src/wasm/literal.cpp
index 032b3f7f8..b5c575b1c 100644
--- a/src/wasm/literal.cpp
+++ b/src/wasm/literal.cpp
@@ -81,26 +81,14 @@ int64_t Literal::getBits() const {
 
 bool Literal::operator==(const Literal& other) const {
   if (type != other.type) return false;
-  switch (type) {
-    case Type::none: return true;
-    case Type::i32: return i32 == other.i32;
-    case Type::f32: return getf32() == other.getf32();
-    case Type::i64: return i64 == other.i64;
-    case Type::f64: return getf64() == other.getf64();
-    default: abort();
-  }
+  if (type == none) return true;
+  return getBits() == other.getBits();
 }
 
 bool Literal::operator!=(const Literal& other) const {
   return !(*this == other);
 }
 
-bool Literal::bitwiseEqual(const Literal& other) const {
-  if (type != other.type) return false;
-  if (type == none) return true;
-  return getBits() == other.getBits();
-}
-
 uint32_t Literal::NaNPayload(float f) {
   assert(std::isnan(f) && "expected a NaN");
   // SEEEEEEE EFFFFFFF FFFFFFFF FFFFFFFF