summaryrefslogtreecommitdiff
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* Remove typed-function-references feature (#5030)Thomas Lively2022-09-091-1/+0
| | | | | | | | | | | | | | | | In practice typed function references will not ship before GC and is not independently useful, so it's not necessary to have a separate feature for it. Roll the functionality previously enabled by --enable-typed-function-references into --enable-gc instead. This also avoids a problem with the ongoing implementation of the new GC bottom heap types. That change will make all ref.null instructions in Binaryen IR refer to one of the bottom heap types. But since those bottom types are introduced in GC, it's not valid to emit them in binaries unless unless GC is enabled. The fix if only reference types is enabled is to emit (ref.null func) instead of (ref.null nofunc), but that doesn't always work if typed function references are enabled because a function type more specific than func may be required. Getting rid of typed function references as a separate feature makes this a nonissue.
* [Wasm GC] Support non-nullable locals in the "1a" form (#4959)Alon Zakai2022-08-311-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An overview of this is in the README in the diff here (conveniently, it is near the top of the diff). Basically, we fix up nn locals after each pass, by default. This keeps things easy to reason about - what validates is what is valid wasm - but there are some minor nuances as mentioned there, in particular, we ignore nameless blocks (which are commonly added by various passes; ignoring them means we can keep more locals non-nullable). The key addition here is LocalStructuralDominance which checks which local indexes have the "structural dominance" property of 1a, that is, that each get has a set in its block or an outer block that precedes it. I optimized that function quite a lot to reduce the overhead of running that logic after each pass. The overhead is something like 2% on J2Wasm and 0% on Dart (0%, because in this mode we shrink code size, so there is less work actually, and it balances out). Since we run fixups after each pass, this PR removes logic to manually call the fixup code from various places we used to call it (like eh-utils and various passes). Various passes are now marked as requiresNonNullableLocalFixups => false. That lets us skip running the fixups after them, which we normally do automatically. This helps avoid overhead. Most passes still need the fixups, though - any pass that adds a local, or a named block, or moves code around, likely does. This removes a hack in SimplifyLocals that is no longer needed. Before we worked to avoid moving a set into a try, as it might not validate. Now, we just do it and let fixups happen automatically if they need to: in the common code they probably don't, so the extra complexity seems not worth it. Also removes a hack from StackIR. That hack tried to avoid roundtrip adding a nondefaultable local. But we have the logic to fix that up now, and opts will likely keep it non-nullable as well. Various tests end up updated here because now a local can be non-nullable - previous fixups are no longer needed. Note that this doesn't remove the gc-nn-locals feature. That has been useful for testing, and may still be useful in the future - it basically just allows nn locals in all positions (that can't read the null default value at the entry). We can consider removing it separately. Fixes #4824
* Implement `extern.externalize` and `extern.internalize` (#4975)Thomas Lively2022-08-292-1/+5
| | | | These new GC instructions infallibly convert between `extern` and `any` references now that those types are not in the same hierarchy.
* Simplify TrapsNeverHappen fuzzing (#4957)Alon Zakai2022-08-241-4/+5
| | | | Also fix a small logic error - call lines can be prefixes of each other, so use the full line (including newline) to differentiate.
* [Fuzzer] Fuzz TrapsNeverHappen mode (#4936)Alon Zakai2022-08-221-2/+85
| | | | | | | | | | | | | | | | This mode is tricky to fuzz because the mode is basically "assume traps never happen; if a trap does happen, that is undefined behavior". So if any trap occurs in the random fuzz testcase, we can't optimize with -tnh and assume the results stay to same. To avoid that, we ignore all functions from the first one that traps, that is, we only compare the code that ran without trapping. That often is a small subset of the total functions, sadly, but I do see that this ends up with some useful coverage despite the drawback. This also requires some fixes to comparing of references, specifically, funcrefs are printed with the function name/index, but that can change during opts, so ignore that. This wasn't noticed before because this new fuzzer mode does comparisons of --fuzz-exec-before output, instead of doing --fuzz-exec which runs it before and after and compares it internally in wasm-opt. Here we are comparing the output externally, which we didn't do before.
* Mutli-Memories Support in IR (#4811)Ashley Nelson2022-08-171-0/+8
| | | | | | | This PR removes the single memory restriction in IR, adding support for a single module to reference multiple memories. To support this change, a new memory name field was added to 13 memory instructions in order to identify the memory for the instruction. It is a goal of this PR to maintain backwards compatibility with existing text and binary wasm modules, so memory indexes remain optional for memory instructions. Similarly, the JS API makes assumptions about which memory is intended when only one memory is present in the module. Another goal of this PR is that existing tests behavior be unaffected. That said, tests must now explicitly define a memory before invoking memory instructions or exporting a memory, and memory names are now printed for each memory instruction in the text format. There remain quite a few places where a hardcoded reference to the first memory persist (memory flattening, for example, will return early if more than one memory is present in the module). Many of these call-sites, particularly within passes, will require us to rethink how the optimization works in a multi-memories world. Other call-sites may necessitate more invasive code restructuring to fully convert away from relying on a globally available, single memory pointer.
* Fix require() in wasm2js test suite for newer Node.js (#4913)Alon Zakai2022-08-171-0/+2
| | | | | | | Without this fix, newer node errors on: "node-esm-loader.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return. This adds that shortCircuit property.
* Fuzzer: When typed-function-references is disabled, disable GC too (#4908)Alon Zakai2022-08-161-1/+2
|
* Fix name of port_passes_tests_to_lit.py script. NFC (#4902)Sam Clegg2022-08-121-2/+2
| | | I was reading these tests and failing to find the names script.
* Port test/passes/legalize-js-interface* to lit (#4903)Sam Clegg2022-08-121-3/+12
| | | | | Also, add support for the `--binaryen-bin` flag to `scripts/port_passes_tests_to_lit.py`. This is needed for folks who don't do in-tree builds.
* Move `auto_update_tests.py` into scripts directory. NFC (#4879)Sam Clegg2022-08-112-2/+202
| | | | So it lives alongside `update_lit_checks.py` and `update_help_checks.py`
* Fuzzer: Emit absolute paths in commands (#4890)Alon Zakai2022-08-101-28/+31
| | | | | | | * better * fix * undo
* Fuzzer: Add some docs and error handling" (#4887)Alon Zakai2022-08-091-4/+19
|
* Remove metadata generation from wasm-emscripten-finalize (#4863)Sam Clegg2022-08-071-10/+0
| | | | This is no longer needed by emscripten as of: https://github.com/emscripten-core/emscripten/pull/16529
* Remove RTTs (#4848)Thomas Lively2022-08-052-18/+9
| | | | | | | RTTs were removed from the GC spec and if they are added back in in the future, they will be heap types rather than value types as in our implementation. Updating our implementation to have RTTs be heap types would have been more work than deleting them for questionable benefit since we don't know how long it will be before they are specced again.
* Bail out of fuzz_shell.js if instantiation fails (#4873)Thomas Lively2022-08-041-2/+9
| | | | | | | | | Sometimes the fuzzer produces valid modules that trap during instantiation. When that happens, the JS harness used to run the fuzzer output in d8 would previously throw an error, creating spurious fuzzer failures on valid modules. Update fuzz_shell.js to catch and supress errors during instantiation (but not validation) to avoid these spurious failures. Fixes #4865.
* Re-run scripts/test/generate_lld_tests.py. NFC (#4861)Sam Clegg2022-08-021-4/+6
|
* Grand Unified Flow Analysis (GUFA) (#4598)Alon Zakai2022-07-221-0/+2
| | | | | | | | | | | | | This tracks the possible contents in the entire program all at once using a single IR. That is in contrast to say DeadArgumentElimination of LocalRefining etc., all of whom look at one particular aspect of the program (function params and returns in DAE, locals in LocalRefining). The cost is to build up an entire new IR, which takes a lot of new code (mostly in the already-landed PossibleContents). Another cost is this new IR is very big and requires a lot of time and memory to process. The benefit is that this can find opportunities that are only obvious when looking at the entire program, and also it can track information that is more specialized than the normal type system in the IR - in particular, this can track an ExactType, which is the case where we know the value is of a particular type exactly and not a subtype.
* [Strings] GC variants for string.encode (#4817)Alon Zakai2022-07-211-0/+2
|
* [Strings] Add string.new GC variants (#4813)Alon Zakai2022-07-191-0/+2
|
* [Strings] stringview_wtf16.length (#4809)Alon Zakai2022-07-181-0/+1
| | | | This measures the length of a view, so it seems simplest to make it a sub-operation of the existing measure instruction.
* [Strings] stringview_*.slice (#4805)Alon Zakai2022-07-151-0/+3
| | | | | | | Unfortunately one slice is the same as python [start:end], using 2 params, and the other slice is one param, [CURR:CURR+num] (where CURR is implied by the current state in the iter). So we can't use a single class here. Perhaps a different name would be good, like slice vs substring (like JS does), but I picked names to match the current spec.
* [Strings] stringview access operations (#4798)Alon Zakai2022-07-131-0/+5
|
* [Strings] string.as (#4797)Alon Zakai2022-07-121-0/+3
|
* [Parser] Start to parse instructions (#4789)Thomas Lively2022-07-111-6/+22
| | | | | | | | | | | | | | | | | | | | | Update gen-s-parser.py to produce a second version of its parsing code that works with the new wat parser. The new version automatically replaces the `s` element argument in the existing parser with the `ctx` and `in` arguments used by the new parser, so adding new instructions will not require any additional work in gen-s-parser.py after this change. Also add stub `make***` functions to the new wat parser, with a few filled out, namely `makeNop`, `makeUnreachable`, `makeConst`, and `makeRefNull`. Update the `global` parser to parse global initializer instructions and update wat-kitchen-sink.wast to demonstrate that the instructions are parsed correctly. Adding new instruction classes will require adding a new `make***` function to wat-parser.cpp in additional to wasm-s-parser.{h,cpp} after this change, but adding a trivial failing implementation is good enough for the time being, so I don't expect this to appreciably increase our maintenance burden in the near term. The infrastructure for parsing folded instructions, instructions with operands, and control flow instructions will be implemented in future PRs.
* Parse `rec` in update_lit_checks.py (#4784)Thomas Lively2022-07-081-1/+1
|
* [Strings] string.is_usv_sequence (#4783)Alon Zakai2022-07-081-0/+1
| | | | | | | This implements it as a StringMeasure opcode. They do have the same number of operands, same trapping behavior, and same return type. They both get a string and do some inspection of it to return an i32. Perhaps the name could be StringInspect or something like that, rather than StringMeasure..? But I think for now this might be good enough, and the spec may change anyhow later.
* [Strings] string.eq (#4781)Alon Zakai2022-07-081-0/+1
|
* [Strings] string.concat (#4777)Alon Zakai2022-07-081-0/+1
|
* [Strings] string.encode (#4776)Alon Zakai2022-07-071-0/+2
|
* Fuzzer: Fix determinism fuzzing (#4767)Alon Zakai2022-07-071-6/+14
| | | | | | #4758 regressed the determinism fuzzer. First, FEATURE_OPTS is not a list but a string. Second, as a hack another location would add --strip-dwarf to that list, but that only works in wasm-opt. To fix that, remove the hack and instead just ignore DWARF-containing files.
* [Strings] string.measure (#4775)Alon Zakai2022-07-071-0/+2
|
* [Strings] Add string.const (#4768)Alon Zakai2022-07-061-0/+1
| | | | | This is more work than a typical instruction because it also adds a new section: all the (string.const "foo") strings are put in a new "strings" section in the binary, and the instructions refer to them by index.
* [Strings] Add string.new* instructions (#4761)Alon Zakai2022-06-291-0/+2
| | | | | | This is the first instruction from the Strings proposal. This includes everything but interpreter support.
* [Strings] Add string proposal types (#4755)Alon Zakai2022-06-291-1/+3
| | | | | | | | This starts to implement the Wasm Strings proposal https://github.com/WebAssembly/stringref/blob/main/proposals/stringref/Overview.md This just adds the types.
* Disallow --nominal with GC (#4758)Thomas Lively2022-06-281-12/+11
| | | | | | | | | | | Nominal types don't make much sense without GC, and in particular trying to emit them with typed function references but not GC enabled can result in invalid binaries because nominal types do not respect the type ordering constraints required by the typed function references proposal. Making this change was mostly straightforward, but required fixing the fuzzer to use --nominal only when GC is enabled and required exiting early from nominal-only optimizations when GC was not enabled. Fixes #4756.
* [Fuzzer] Pretty print of feature opts and passes (#4740)Max Graey2022-06-221-2/+2
|
* Disable Wasm2C in fuzz_opt for now (#4743)Max Graey2022-06-211-2/+4
| | | Relates to #4741
* Update relaxed SIMD instructionsThomas Lively2022-06-071-2/+0
| | | | | Update the opcodes for all relaxed SIMD instructions and remove the unsigned dot product instructions that are no longer in the proposal.
* Global Struct Inference pass: Infer two constants in struct.get (#4659)Alon Zakai2022-06-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | This optimizes constants in the megamorphic case of two: when we know two function references are possible, we could in theory emit this: (select (ref.func A) (ref.func B) (ref.eq (..ref value..) ;; globally, only 2 things are possible here, and one has ;; ref.func A as its value, and the other ref.func B (ref.func A)) That is, compare to one of the values, and emit the two possible values there. Other optimizations can then turn a call_ref on this select into an if over two direct calls, leading to devirtualization. We cannot compare a ref.func directly (since function references are not comparable), and so instead we look at immutable global structs. If we find a struct type that has only two possible values in some field, and the structs are in immutable globals (which happens in the vtable case in j2wasm for example), then we can compare the references of the struct to decide between the two values in the field.
* Fuzzer: Pass v8 flag for extended-const (#4543)Alon Zakai2022-05-271-1/+2
| | | | | This flag is needed now that we have testcases using extended-const in our test suite, as the fuzzer will grab random testcases, modify them, and run them through v8.
* [Fuzzer] Ignore relaxed-simd test for initial contents (#4683)Alon Zakai2022-05-241-0/+8
| | | | If we use it as initial contents, we will try to execute it, and hit the TODOs in the interpreter for unimplemented parts.
* Fuzzer: Show a clear error if a given wasm fails to run (#4672)Alon Zakai2022-05-171-1/+5
| | | | | | Without this the error from a bad given wasm file - either by the user, or during a reduction where smaller wasms are given - could be very confusing. A bad given wasm file during reduction, in particular, indicates a bug in the reducer most likely.
* Add ref.cast_nop_static (#4656)Thomas Lively2022-05-111-0/+1
| | | | | | This unsafe experimental instruction is semantically equivalent to ref.cast_static, but V8 will unsafely turn it into a nop. This is meant to help us measure cast overhead more precisely than we can by globally turning all casts into nops.
* [Fuzzer] Add typesystem flag to reduction script (#4651)Alon Zakai2022-05-061-2/+3
| | | | | Without this the reduction will fail on not being able to parse the input file, if the input file depends on nominal typing.
* [NominalFuzzing] Add typesystem flag to wasm-dis (#4645)Alon Zakai2022-05-051-2/+2
| | | | | wasm-dis does enable all features by default, so we don't need the feature flags, but we do need --nominal etc. since we emit such modules now.
* Remove externref (#4633)Thomas Lively2022-05-041-8/+0
| | | | | | Remove `Type::externref` and `HeapType::ext` and replace them with uses of anyref and any, respectively, now that we have unified these types in the GC proposal. For backwards compatibility, continue to parse `extern` and `externref` and maintain their relevant C API functions.
* [NominalFuzzing] Use feature flags in a missing place (#4628)Alon Zakai2022-05-021-1/+1
| | | | | | | Instead of a raw run command, use the helper function, which adds the feature flags. That adds --nominal which is needed more now after #4625 This fixes the fuzz failures mentioned in #4625 (comment)
* Lit tests: print failing module index (#4614)Alon Zakai2022-04-261-0/+5
| | | | When a .wat file has lots of modules inside it, printing the index of the module is helpful to find which is erroring.
* [NominalFuzzing] Switch to nominal fuzzing by default (#4610)Alon Zakai2022-04-221-3/+13
| | | Add a flag to make it easy to pick which typesystem to test.