summaryrefslogtreecommitdiff
path: root/scripts/test
Commit message (Collapse)AuthorAgeFilesLines
* [NFC] Move more logic about unfuzzable tests to a shared location (#7175)Alon Zakai2024-12-201-1/+19
| | | | | | | It turns out that #7165 was not enough because we had a second (!?) list of tests to ignore, and also a condition. Move all that to the shared location as well, continuing that PR. Also remove simd.wast from the list, as that issue has been fixed.
* Update flake8 and fix errors (#7172)Thomas Lively2024-12-202-4/+4
| | | | | | | | | The flake8 we were running on CI was too old and began giving spurious errors about the uninterpreted contents of f-strings. Update to the latest flake8 and fix all the new errors, including the previously incorrect comment syntax in the .flake8 file. Also remove scripts/storage.py, since it didn't seem to be used for anything we currently need.
* [NFC] Move code from fuzz_opt.py to a shared place (#7165)Alon Zakai2024-12-192-0/+92
| | | | | The list of tests, and which tests cannot be fuzzed, will be useful in a future PR that uses it for ClusterFuzz as well. This just moves things out so they are reusable.
* Enable upstream spec tests requiring block parameters (#7151)Thomas Lively2024-12-162-5/+2
| | | | Now that #7149 added support for parsing block parameters, we can run additional spec tests that previously failed.
* Error on missing wasm2js test expectations (#7119)Alon Zakai2024-11-261-1/+9
|
* [wasm2js] Run LLVM nontrapping fptoint lowering when running for emscripten ↵Derek Schuff2024-11-261-4/+5
| | | | | | (#7116) Lower away saturating fptoint operations when we know we are using emscripten.
* [tests] Fix wasm2js test command line generation (#7114)Derek Schuff2024-11-251-2/+2
| | | | | | | | | --emscripten is added to the command line if the filename contains the string 'emscripten', but the current logic includes the cwd, so it fails if the tests are run from a directory with 'emscripten' in the name. Fix it so the test only includes the basename.
* Ignore fp16 in the fuzzer (#6881)Alon Zakai2024-08-291-0/+1
| | | | Add the feature flag in V8 invocations, but also disable the feature as it isn't quite ready yet.
* Support more reference constants in wast scripts (#6865)Thomas Lively2024-08-261-3/+3
| | | | | | | | | | | | | | Spec tests use constants like `ref.array` and `ref.eq` to assert that exported function return references of the correct types. Support more such constants in the wast parser. Also fix a bug where the interpretation of `array.new_data` for arrays of packed fields was not properly truncating the packed data. Move the function for reading fields from memory from literal.cpp to wasm-interpreter.h, where the function for truncating packed data lives. Other bugs prevent us from enabling any more spec tests as a result of this change, but we can get farther through several of them before failing. Update the comments about the failures accordingly.
* Support `ref.extern n` in spec tests (#6858)Thomas Lively2024-08-212-12/+8
| | | | | | | | | | | | | | | | | Spec tests pass the value `ref.extern n`, where `n` is some integer, into exported functions that expect to receive externrefs and receive such values back out as return values. The payload serves to distinguish externrefs so the test can assert that the correct one was returned. Parse these values in wast scripts and represent them as externalized i31refs carrying the payload. We will need a different representation eventually, since some tests explicitly expect these externrefs to not be i31refs, but this suffices to get several new tests passing. To get the memory64 version of table_grow.wast passing, additionally fix the interpreter to handle growing 64-bit tables correctly. Delete the local versions of the upstream tests that can now be run successfully.
* [NFC] Triage spec test problems (#6857)Thomas Lively2024-08-211-81/+80
| | | | Add comments to the spec test skip list briefly explaining why each skipped spec test must be skipped.
* Fix encoding of heap type definitions (#6856)Thomas Lively2024-08-201-3/+2
| | | | | | | | The leading bytes that indicate what kind of heap type is being defined are bytes, but we were previously treating them as SLEB128-encoded values. Since we emit the smallest LEB encodings possible, we were writing the correct bytes in output files, but we were also improperly accepting binaries that used more than one byte to encode these values. This was caught by an upstream spec test.
* Add the upstream spec testsuite as a submodule (#6853)Thomas Lively2024-08-202-6/+99
| | | | | | Run the upstream tests by default, except for a large list of them that do not successfully run. Remove the local version of those that do successfully run where the local version is entirely subsumed by the upstream version.
* Testing: Add an env var to pick the V8 binary (#6836)Alon Zakai2024-08-161-2/+2
| | | | | Also we had a mix of os.environ.get and os.getenv. Prefer the former, as the default value does actual work, so it's a little more efficient to not run it unnecessarily. That is, os.getenv('X', work()) is less efficient than os.environ.get('X') or work().
* [threads] ref.i31_shared (#6735)Thomas Lively2024-07-121-1/+1
| | | | | | | Implement `ref.i31_shared` the new instruction for creating references to shared i31s. Implement binary and text parsing and emitting as well as interpretation. Copy the upstream spec test for i31 and modify it so that all the heap types are shared. Comment out some parts that we do not yet support.
* Re-enable spec tests requiring multivalue (#6684)Thomas Lively2024-06-201-2/+0
| | | And delete tests that no longer pass now that multivalue is standard.
* Validate memarg offsets (#6683)Thomas Lively2024-06-201-2/+1
| | | | | For 32-bit memories, the offset value must be in the u32 range. Update the address.wast spec test to assert that a module with an overlarge offset value is invalid rather than malformed.
* Validate that names are valid UTF-8 (#6682)Thomas Lively2024-06-191-4/+0
| | | | | | Add an `isUTF8` utility and use it in both the text and binary parsers. Add missing checks for overlong encodings and overlarge code points in our WTF8 reader, which the new utility uses. Re-enable the spec tests that test UTF-8 validation.
* Fix validation of unused LEB128 bits (#6680)Thomas Lively2024-06-191-1/+0
| | | | | The unused bits must be a sign extension of the significant value, but we were previously only validating that unsigned LEBs had their unused bytes set to zero. Re-enable the spec test that checks for proper validation.
* Check malformed mutability on imported globals (#6679)Thomas Lively2024-06-181-1/+0
| | | And re-enable the globals.wast spec test, which checks this.
* Re-enable binary.wast spec test (#6677)Thomas Lively2024-06-181-3/+0
| | | | | | Fix the wast parser to accept IDs on quoted modules, remove tests that are invalidated by the multimemory proposal, and add validation that the total number of variables in a function is less than 2^32 and that the code section is present if there is a non-empty function section.
* [Parser] Fix bug in unreachable fallback logic (#6676)Thomas Lively2024-06-181-3/+0
| | | | | | | | | When popping past an unreachable instruction would lead to popping from an empty stack or popping an incorrect type, we need to avoid popping and produce new Unreachable instructions instead to ensure we parse valid IR. The logic for this was flawed and made the synthetic Unreachable come before the popped unreachable child, which was not correct in the case that that popped unreachable was a branch or other non-trapping instruction. Fix and simplify the logic and re-enable the spec test that uncovered the bug.
* Reject invalid section IDs (#6675)Thomas Lively2024-06-181-1/+0
| | | | | | Rather than treating them as custom sections. Also fix UB where invalid `Section` enum values could be used as keys in a map. Use the raw `uint8_t` section IDs as keys instead. Re-enable a disabled spec test that was failing because of this bug and UB.
* Enable more spec tests (#6669)Thomas Lively2024-06-172-43/+30
| | | | | Re-triage all the disabled spec tests and re-enable many of them. Improve the module splitting logic to correctly handle (by skipping) quoted modules and their associated assertions.
* Fix running just a single spec test via `check.py spec --spec-test foo`. ↵Sam Clegg2024-05-101-5/+6
| | | | NFC (#6580)
* Do not write assertions to split.wast for spec tests (#6383)Thomas Lively2024-03-071-1/+1
| | | | | | | | | | As part of our running of spec tests, we split out each module in a test script into a separate text file for processing with wasm-opt. We previously included the test assertions corresponding to the module into that text file, where they were ignored by the legacy text parser. The new parser errors out due to the extra tokens after the module, though, so to avoid problems once we switch to the new parser, stop including the assertions in those text files. Also remove a nearby unused argument as a drive-by cleanup.
* Fuzzing: Remove some obsolete experimental V8 flags (#6347)Alon Zakai2024-02-281-4/+0
| | | | Those flags were removed in V8 as the features are no longer experimental. This PR removes some warnings from being logged (but V8 does not error on them).
* Fuzzer: Allow using initial content with V8 (#6327)Alon Zakai2024-02-221-0/+1
| | | | | | | | | | | | | | | One problem was that spec testcases had exports with names that are not valid to write as JS exports.name. For example an export with a - in the name would end up as exports.foo-bar etc. Since #6310 that is fixed as we do not emit such JS (we use the generic fuzz_shell.js script which iterates over the keys in exports with exports[name]). Also fix a few trivial fuzzer issues that initial content uncovered: - Ignore a wat file with invalid utf-8. - Print string literals in the same way from JS as from C++. - Enable the stringref flag in V8. - Remove tag imports (the same as we do for global and function and other imports).
* [test] Port tests in test/ to test/lit/basic/ (#6160)Heejin Ahn2023-12-132-63/+8
| | | | | | | | | | | | | | | | | | | | | This ports all tests from `test/` to `test/lit/basic/`. The set of commands and `CHECK` lines used are the same as the ones in #6159. Now we use `lit` to test these, this also deletes all `.wast`, `.wast.from-wast`, `.wast.fromBinary`, and `.wast.fromBinary.noDebugInfo` files from `test/` and all related test routines from the python scripts. All `CHECK` lines are generated by `update_lit_checks.py --all-items`. This also deletes these three multi-memory tests in `test/lit/`, because they seem to contain the same code with the ones in `test/`, which have been ported to `test/lit/basic/` along with other tests. - `test/lit/multi-memories-atomics64.wast` - `test/lit/multi-memories-basics.wast` - `test/lit/multi-memories-simd.wast` This also adds newlines between `(func`s in case there are none to make `CHECK` lines easy to view, and removes some extra existing newlines here and there.
* [test] Fix wasm2js stale check failure (#6168)Heejin Ahn2023-12-111-2/+2
| | | | | | | I tried to exclude wasm2js asserts tests from `check_for_stale_files` in #6164, but ended up doing it incorrectly. The file I checked for was `wasm2js.wast.asserts`, while the output I should have excluded was `wasm2js.asserts.js`. This fixes the code so we now check the prefix and not the filename.
* [test] Remove asm2wasm tests from scripts (#6162)Heejin Ahn2023-12-112-24/+0
| | | | We don't have `*.fromasm` files anymore. Also `BIN_DIR` and `WATERFALL_BUILD_DIR` variables don't seem to be used as well.
* [test] Remove / move *.js tests from test/ (#6163)Heejin Ahn2023-12-111-4/+4
| | | | | | | | | | | | | `wasm2js.asserts.js` and `wasm2js.traps.js` seem to be used in wasm2js asserts test: https://github.com/WebAssembly/binaryen/blob/1d615b38dd4152494d2f4d3520c8b1d917624a30/scripts/test/wasm2js.py#L28 https://github.com/WebAssembly/binaryen/blob/1d615b38dd4152494d2f4d3520c8b1d917624a30/scripts/test/wasm2js.py#L126-L127 But other `*.js` tests in `test/` don't seem to be used anywhere. Please let me know if they are actually being used. This moves `wasm2js.asserts.js` and `wasm2js.traps.js`, which are only used in wasmjs tests, to `test/wasm2js/`, and deletes all other `*.js` tests in `test/`.
* [test] Make get_tests return only files (#6164)Heejin Ahn2023-12-112-0/+3
| | | | | | | | | | | | | | | | | | | | Currently `get_tests` returns files and directories, especially when the extension is not given. This makes `get_tests` return a directory like `test/wasm2js/` as a test. `wasm2js.py`'s `check_for_stale_files` errors out when there are files within `test/wasm2js/` whose basenames don't match any files within any of `test/`, `test/spec/`, `test/wasm2js/`. https://github.com/WebAssembly/binaryen/blob/1d615b38dd4152494d2f4d3520c8b1d917624a30/scripts/test/wasm2js.py#L33-L46 `wasm2js.wast.asserts` is apparently a special case for asserts test: https://github.com/WebAssembly/binaryen/blob/1d615b38dd4152494d2f4d3520c8b1d917624a30/scripts/test/wasm2js.py#L28 and this doesn't seem to have the matching `wast` tests in the three test directories. But it just happened to not error out because `get_tests` returns directory names too and one of them was `wasm2js` (`test/wasm2js/` directory). This makes `get_tests` return only files, and make files in `assert_tests` not error out additionally.
* [wasm-emscripten-finalize] Remove --separate-data-segments (#6091)Sam Clegg2023-11-271-37/+12
| | | See #6088
* Remove various testing spam (#6109)Alon Zakai2023-11-143-6/+7
| | | | | Avoid some common warnings and stop printing various stdout/stderr stuff. Helps #6104
* Enable auto_initial_contents by default in fuzz_opt.py (#5943)Thomas Lively2023-09-141-2/+2
| | | | This setting is useful enough that there is basically no reason not to use it. Turn it on by default to save some typing when running the fuzzer.
* Add new V8 flag for final types and remove old removed ones (#5807)Alon Zakai2023-07-101-2/+1
|
* Add a CMake flag to enable Wasm exceptions in the BinaryenJS build (#5454)Derek Schuff2023-02-021-1/+1
|
* Unpin V8 (#5277)Thomas Lively2022-11-171-3/+1
| | | | | | | | | | | | | | * Do not compare reference values across executions Since we optimize assuming a closed world, optimizations can change the types and structure of GC data even in externally-visible ways. Because differences are expected, the fuzzer already did not compare reference-typed values from before and after optimizations when running with nominal typing. Update it to not compare these values under any type system. * Unpin V8 Our WasmGC output is no longer compatible with the previously pinned version and the issue that caused us to pin it in the first place has been resolved.
* Pin v8 10.8.104 (#5112)Thomas Lively2022-10-051-1/+4
| | | | | | More recent version of V8 include a change from `dataref` to `structref` that prevent WasmGC modules produced by the fuzzer from validating. Use the last compatible v8 version if it is in the path until we can update Binaryen to be compatible with newer v8.
* [Wasm GC] Support non-nullable locals in the "1a" form (#4959)Alon Zakai2022-08-311-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An overview of this is in the README in the diff here (conveniently, it is near the top of the diff). Basically, we fix up nn locals after each pass, by default. This keeps things easy to reason about - what validates is what is valid wasm - but there are some minor nuances as mentioned there, in particular, we ignore nameless blocks (which are commonly added by various passes; ignoring them means we can keep more locals non-nullable). The key addition here is LocalStructuralDominance which checks which local indexes have the "structural dominance" property of 1a, that is, that each get has a set in its block or an outer block that precedes it. I optimized that function quite a lot to reduce the overhead of running that logic after each pass. The overhead is something like 2% on J2Wasm and 0% on Dart (0%, because in this mode we shrink code size, so there is less work actually, and it balances out). Since we run fixups after each pass, this PR removes logic to manually call the fixup code from various places we used to call it (like eh-utils and various passes). Various passes are now marked as requiresNonNullableLocalFixups => false. That lets us skip running the fixups after them, which we normally do automatically. This helps avoid overhead. Most passes still need the fixups, though - any pass that adds a local, or a named block, or moves code around, likely does. This removes a hack in SimplifyLocals that is no longer needed. Before we worked to avoid moving a set into a try, as it might not validate. Now, we just do it and let fixups happen automatically if they need to: in the common code they probably don't, so the extra complexity seems not worth it. Also removes a hack from StackIR. That hack tried to avoid roundtrip adding a nondefaultable local. But we have the logic to fix that up now, and opts will likely keep it non-nullable as well. Various tests end up updated here because now a local can be non-nullable - previous fixups are no longer needed. Note that this doesn't remove the gc-nn-locals feature. That has been useful for testing, and may still be useful in the future - it basically just allows nn locals in all positions (that can't read the null default value at the entry). We can consider removing it separately. Fixes #4824
* Fix require() in wasm2js test suite for newer Node.js (#4913)Alon Zakai2022-08-171-0/+2
| | | | | | | Without this fix, newer node errors on: "node-esm-loader.mjs 'resolve'" did not call the next hook in its chain and did not explicitly signal a short circuit. If this is intentional, include `shortCircuit: true` in the hook's return. This adds that shortCircuit property.
* Move `auto_update_tests.py` into scripts directory. NFC (#4879)Sam Clegg2022-08-111-2/+2
| | | | So it lives alongside `update_lit_checks.py` and `update_help_checks.py`
* Remove metadata generation from wasm-emscripten-finalize (#4863)Sam Clegg2022-08-071-10/+0
| | | | This is no longer needed by emscripten as of: https://github.com/emscripten-core/emscripten/pull/16529
* Re-run scripts/test/generate_lld_tests.py. NFC (#4861)Sam Clegg2022-08-021-4/+6
|
* Fuzzer: Pass v8 flag for extended-const (#4543)Alon Zakai2022-05-271-1/+2
| | | | | This flag is needed now that we have testcases using extended-const in our test suite, as the fuzzer will grab random testcases, modify them, and run them through v8.
* Add --no-emit-metadata option to wasm-emscripten-finalize (#4450)Sam Clegg2022-01-191-1/+3
| | | | | | This is useful for the case where we might want to finalize without extracting metadata. See: https://github.com/emscripten-core/emscripten/pull/15918
* Switch binaryen.js/wasm to ESM (#4280)dcode2021-10-282-10/+20
|
* Update to C++17 and use std::optional for getSuperType (#4203)Derek Schuff2021-10-181-1/+1
| | | This sets the C++ standard variable in the build to C++17, and makes use of std::optional (a C++17 library feature) in one place, to test that it's working.
* Select initial contents automatically in fuzzer (#4173)Heejin Ahn2021-09-221-0/+8
| | | | | | | | | This is another attempt to address #4073. Instead of relying on the timestamp, this examines git log to gather the list of test files added or modified within some fixed number of days. The number of days is currently set to 30 (= 1 month) but can be changed. This will be enabled by `--auto-initial-contents`, which is now disabled by default. Hopefully fixes #4073.
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-09 14:34:48 +0000