summaryrefslogtreecommitdiff
path: root/check.py
Commit message (Collapse)AuthorAgeFilesLines
* Update flake8 and fix errors (#7172)Thomas Lively2024-12-201-2/+2
| | | | | | | | | The flake8 we were running on CI was too old and began giving spurious errors about the uninterpreted contents of f-strings. Update to the latest flake8 and fix all the new errors, including the previously incorrect comment syntax in the .flake8 file. Also remove scripts/storage.py, since it didn't seem to be used for anything we currently need.
* Enable more spec tests (#6669)Thomas Lively2024-06-171-28/+20
| | | | | Re-triage all the disabled spec tests and re-enable many of them. Improve the module splitting logic to correctly handle (by skipping) quoted modules and their associated assertions.
* Remove obsolete parser code (#6607)Thomas Lively2024-05-291-1/+1
| | | | | Remove `SExpressionParser`, `SExpressionWasmBuilder`, and `cashew::Parser`. Simplify gen-s-parser.py. Remove the --new-wat-parser and --deprecated-wat-parser flags.
* [Parser] Use the new parser in wasm-shell and wasm-as (#6529)Thomas Lively2024-04-241-2/+2
| | | | | | | | | | | | | | | | | | | Updating just one or the other of these tools would cause the tests spec/import-after-*.fail.wast to fail, since only the updated tool would correctly fail to parse its contents. To avoid this, update both tools at once. (The tests erroneously pass before this change because check.py does not ensure that .fail.wast tests fail, only that failing tests end in .fail.wast.) In wasm-shell, to minimize the diff, only use the new parser to parse modules and instructions. Continue using the legacy parsing based on s-expressions for the other wast commands. Updating the parsing of the other commands to use `Lexer` instead of `SExpressionParser` is left as future work. The boundary between the two parsing styles is somewhat hacky, but it is worth it to enable incremental development. Update the tests to fix incorrect wast rejected by the new parser. Many of the spec/old_* tests use non-standard forms from before Wasm MVP was standardized, so fixing them would have been onerous. All of these tests have non-old_* variants, so simply delete them.
* Do not write assertions to split.wast for spec tests (#6383)Thomas Lively2024-03-071-2/+2
| | | | | | | | | | As part of our running of spec tests, we split out each module in a test script into a separate text file for processing with wasm-opt. We previously included the test assertions corresponding to the module into that text file, where they were ignored by the legacy text parser. The new parser errors out due to the extra tokens after the module, though, so to avoid problems once we switch to the new parser, stop including the assertions in those text files. Also remove a nearby unused argument as a drive-by cleanup.
* [test] Port tests in test/ to test/lit/basic/ (#6160)Heejin Ahn2023-12-131-1/+1
| | | | | | | | | | | | | | | | | | | | | This ports all tests from `test/` to `test/lit/basic/`. The set of commands and `CHECK` lines used are the same as the ones in #6159. Now we use `lit` to test these, this also deletes all `.wast`, `.wast.from-wast`, `.wast.fromBinary`, and `.wast.fromBinary.noDebugInfo` files from `test/` and all related test routines from the python scripts. All `CHECK` lines are generated by `update_lit_checks.py --all-items`. This also deletes these three multi-memory tests in `test/lit/`, because they seem to contain the same code with the ones in `test/`, which have been ported to `test/lit/basic/` along with other tests. - `test/lit/multi-memories-atomics64.wast` - `test/lit/multi-memories-basics.wast` - `test/lit/multi-memories-simd.wast` This also adds newlines between `(func`s in case there are none to make `CHECK` lines easy to view, and removes some extra existing newlines here and there.
* Remove various testing spam (#6109)Alon Zakai2023-11-141-2/+2
| | | | | Avoid some common warnings and stop printing various stdout/stderr stuff. Helps #6104
* Add BUILD_TESTS CMake option (#4536)Thomas Lively2022-03-211-5/+8
| | | | Turning it off removes the build dependency on the third-party googletest library.
* Introduce gtest (#4466)Thomas Lively2022-01-201-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | Add gtest as a git submodule in third_party and integrate it into the build the same way WABT does. Adds a new executable, `binaryen-unittests`, to execute `gtest_main`. As a nontrivial example test, port one of the `TypeBuilder` tests from example/ to gtest/. Using gtest has a number of advantages over the current example tests: - Tests are compiled and linked at build time rather than runtime, surfacing errors earlier and speeding up test execution. - Tests are all built into a single binary, reducing overall link time and further reducing test overhead. - Tests are built from the same CMake project as the rest of Binaryen, so compiler settings (e.g. sanitizers) are applied uniformly rather than having to be separately set via the COMPILER_FLAGS environment variable. - Using the industry-standard gtest rather than our own script reduces our maintenance burden. Using gtest will lower the barrier to writing C++ tests and will hopefully lead to us having more proper unit tests.
* [ctor-eval] Add an option to keep some exports (#4441)Alon Zakai2022-01-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | By default wasm-ctor-eval removes exports that it manages to completely eval (if it just partially evals then the export remains, but points to a function with partially-evalled contents). However, in some cases we do want to keep the export around even so, for example during fuzzing (as the fuzzer wants to call the same exports before and after wasm-ctor-eval runs) and also if there is an ABI we need to preserve (like if we manage to eval all of main()), or if the function returns a value (which we don't support yet, but this is a PR to prepare for that). Specifically, there is now a new option: --kept-exports foo,bar That is a list of exports to keep around. Note that when we keep around an export after evalling the ctor we make the export point to a new function. That new function just contains a nop, so that nothing happens when it is called. But the original function is kept around as it may have other callers, who we do not want to modify.
* [ctor-eval] Add --ignore-external-input option (#4428)Alon Zakai2022-01-061-0/+2
| | | | | | | | | | | | This is meant to address one of the main limitations of wasm-ctor-eval in emscripten atm, that libc++ global ctors will read env vars, which means they call an import, which stops us from evalling, emscripten-core/emscripten#15403 (comment) To handle that, this adds an option to ignore external input. When set, we can assume that no env vars will be read, no reading from stdin, no arguments to main(), etc. Perhaps these could each be separate options, but I think keeping it simple for now might be good enough.
* Add DS_Store to non-executable filter in check.py (#3999)Max Graey2021-07-161-1/+1
|
* Lit tests for tool help messages (#3965)Thomas Lively2021-07-071-15/+3
| | | | | | Add list tests for the help messages of all tools, factoring out common options into shared tests. This is slightly brittle because the text wrapping depends on the length of the longest option, but that brittleness should be worth the benefit of being able to see the actual help text in the tests.
* Fix 'example' tests in check.py (#3920)Thomas Lively2021-06-071-5/+5
| | | | | | Rename the test suite from 'gcc' to 'example' to match the name in auto_update_tests.py. Also retrieve the tests to run using `shared.get_tests`, which properly applies the --filter argument, again to match auto_update_tests.py.
* Run lit tests with pass-debug as well, to check for more errors (#3836)Alon Zakai2021-04-221-9/+12
|
* Run spec test all at once after binary transform (#3817)Abbas Mashayekh2021-04-201-13/+15
| | | | | | | | | | | | | | | | | | #3792 added support for module linking and (register command to wasm-shell, but forgot about three problems: - Splitting spec tests prevents linking test modules together. - Registered modules may still be used in assertions or an invoke - Modules may re-export imported objects This PR appends transformed modules after binary checks to a spec.wast file, plus assertion tests and register commands. Then runs wasm-shell on the whole file. It also keeps both the module name and its registered name available in wasm-shell for use in shell commands and linked modules. Furthermore, it correctly finds the module where an object is defined even if it is imported and re-exported several times. The updated version of imports.wast spec test is enabled to verify the fixes.
* Update interpreter for new EH spec (#3498)Heejin Ahn2021-01-211-3/+0
| | | | | | | | | | | | | This updates the interpreter for the EH instructions (modulo `delegate`) to match the new spec. Before we had an `exnref` type so threw a `Literal` of `exnref` type which contained `ExceptionPackage`. But now that we don't have `exnref` anymore, so we add the contents of `ExceptionPackage` to `WasmException`, which is used only for the `ExpressionRunner` class hierarchy. `exnref` and `ExceptionPackage` will be removed in a followup CL. This allows nonzero depths for `rethrow` for now for testing; we disallowed that for safety measure, but given that there are no passes that modifies that field, I think the risk is low.
* wasm-reduce: Fix setting of feature flags after loading (#3493)Alon Zakai2021-01-151-2/+2
| | | | | We mistakenly did not set the flags to all, which meant that if the features section was not present, we'd not have the proper features set, leading to errors on writing.
* Basic EH instrucion support for the new spec (#3487)Heejin Ahn2021-01-151-0/+3
| | | | | | | | | | | | | | | | | | | | This updates `try`-`catch`-`catch_all` and `rethrow` instructions to match the new spec. `delegate` is not included. Now `Try` contains not a single `catchBody` expression but a vector of catch bodies and events. This updates most existing routines, optimizations, and tests modulo the interpreter and the CFG traversal. Because the interpreter has not been updated yet, the EH spec test is temporarily disabled in check.py. Also, because the CFG traversal for EH is not yet updated, several EH tests in `rse_all-features.wast`, which uses CFG traversal, are temporarily commented out. Also added a few more tests in existing EH test functions in test/passes. In the previous spec, `catch` was catching all exceptions so it was assumed that anything `try` body throws is caught by its `catch`, but now we can assume the same only if there is a `catch_all`. Newly added tests test cases when there is a `catch_all` and cases there are only `catch`es separately.
* Improve lit support (#3426)Sam Clegg2020-12-091-4/+4
| | | | | | | | This uses the same technique used in llvm-lit to enable running on in-tree tests with out-of-tree builds. So you can run something like this: ../binaryen-out/bin/binaryen-lit test/lit/
* Introduce lit/FileCheck tests (#3367)Thomas Lively2020-11-181-0/+13
| | | | | | | | | | | | | | | lit and FileCheck are the tools used to run the majority of tests in LLVM. Each lit test file contains the commands to be run for that test, so lit tests are much more flexible and can be more precise than our current ad hoc testing system. FileCheck reads expected test output from comments, so it allows test output to be written alongside and interspersed with test input, making tests more readable and precise than in our current system. This PR adds a new suite to check.py that runs lit tests in the test/lit directory. A few tests have been ported to demonstrate the features of the new test runner. This change is motivated by a need for greater flexibility in testing wasm-split. See #3359.
* Allow setting the path to libbinaryen.so shared library (#3025)rathann2020-11-101-1/+1
| | | | This makes it easier to install libbinaryen.so into an alternative locations. Fixes part of issue #2999 for me.
* Remove remaining bits of C API tracing, which has been removed (#3322)Alon Zakai2020-11-051-30/+17
| | | It was removed in #2841
* Remove remnants of spec interpreter, and excess mozjs warning (#3262)Alon Zakai2020-10-201-5/+0
| | | | | The spec interpreter is no longer used at all. Mozjs is still used optionally, but while it was crucial in the past for test coverage, it is entirely optional now and not run by default, so no need to warn.
* Improve testing on Windows (#3142)Wouter van Oortmerssen2020-09-171-4/+0
| | | | | | This PR contains: - Changes that enable/disable tests on Windows to allow for better local testing. - Also changes many abort() into Fatal() when it is really just exiting on error. This is because abort() generates a dialog window on Windows which is not great in automated scripts. - Improvements to CMake to better work with the project in IDEs (VS).
* Upgrade to C++17 (#3103)Daniel Wirtz2020-09-061-2/+2
|
* Remove asm2wasm (#3042)Alon Zakai2020-08-171-3/+0
| | | | | | | Now that fastcomp has been removed from Emscripten, there is no need for the asm2wasm tool which it used to compile fastcomp's asm.js output to wasm. See emscripten-core/emscripten#11860
* Skip tests that fail on windows and enable all the rest (#3035)Alon Zakai2020-08-111-6/+17
| | | | | | | | | | | | | | This lets us run most tests at least on that platform. Add a new function for skipping those tests, skip_if_on_windows, so that it's easy to find which tests are disabled on windows for later fixing efforts. This fixes a few minor issues for windows, like comparisons should ignore \r in some cases. Rename all passes tests that use --dwarfdump to contain "dwarf" in their name, which makes it easy to skip those (and is clearer anyhow).
* Remove unneeded old binary validation in test suite (#3034)Alon Zakai2020-08-101-2/+0
| | | | | | | | This was useful back when we didn't have many VMs to test in, and we weren't confident in our binaries being valid wasm. Today we have lots of testing on VMs, and in particular, this test tends to fail when we do things like reorder SIMD opcode constants. (This doesn't fail on CI as we don't have v8 installed there, so this path is never reached, but it does happen locally.)
* Put validator test outputs in out/test (#2882)Heejin Ahn2020-05-311-6/+6
| | | | | | | We now put outputs of all other tests in out/test in order not to pollute the test directory, but validator tests didn't have a output file specified so their output files were written in test/validator. This adds `-o a.wasm` to validator tests command lines, in the same way as other tests, to make them go into out/test directory.
* Fix issues with cleanup dSYM in scripts for MacOS (#2871)Max Graey2020-05-261-5/+0
| | | | Remove rmtree call from check.py & auto_update_tests.py
* Use --detect-features in wasm-reduce. Fixes #2813 (#2815)Alon Zakai2020-04-281-3/+3
|
* Fix binaryenjs testing (#2810)Sam Clegg2020-04-271-2/+14
| | | | | | These tests are now optional. However, if you run them and the build is not found they will now error out, in order to avoid silently failing.
* Convert CI from travis + appveyor to github actions (#2646)Sam Clegg2020-04-211-1/+1
| | | | | | The intention is to move away from travis and appveyor which have become very slow. See: #2356
* Emit tuples in the fuzzer (#2695)Thomas Lively2020-04-151-2/+2
| | | | | | | | Emit tuple.make, tuple.extract, and multivalue control flow, and tuple locals and globals when multivalue is enabled. Also slightly refactors the top-level `makeConcrete` function to be more selective about what it tries to make based on the requested type to reduce the number of trivial nodes created because the requested type is incompatible with the requested node.
* Speed up wasm-reduce test (#2756)Alon Zakai2020-04-131-2/+2
| | | | | | The test there just wants to see that the reducer can remove a significant amount of code. I changed it from a file of 3.6K to 200 bytes, which is enough to see the effect, and much faster.
* Use std::cout for interpreter trap logging (#2755)Alon Zakai2020-04-131-1/+5
| | | | | | | | | | | | | | We used std::cerr as a workaround for that this logging interfered with spec testing. But it's easy enough to filter out this stuff for the spec tests. The benefit to using std::cout is that as you can see in the test output here, this is relevant test output - it's not a side channel for debugging. If the rest of the interpreter output is in std::cout but only traps are in std::cerr then they might end up out of order etc., so best to keep them all together. This will allow easier additions of tests for fuzz testcases
* Factor out wasm_opt testing. NFC. (#2737)Sam Clegg2020-04-091-108/+2
| | | | Also, factor out auto-updating of binaryenjs testing so it lives alongside the actual test code.
* Update Precompute to handle tuples (#2687)Thomas Lively2020-03-101-1/+1
| | | | | | This involves replacing `Literal::makeZero` with `Literal::makeZeroes` and `Literal::makeSingleZero` and updating `isConstantExpression` to handle constant tuples as well. Also makes `Literals` its own struct and adds convenience methods on it.
* DWARF: Fix debug_range handling of invalid entries (#2662)Alon Zakai2020-02-181-1/+3
| | | | | | | | | | | | | | If an invalid entry appears - either it began as such, or became invalid after optimization - we should not emit (0, 0) which is an end marker. Instead, emit an invalid entry marker, something with (0, x) for x != 0. As a bonus, if a test/passes case has "noprint" in the name, don't print the wasm, which we do by default. In the testcase here for example we just care about the dwarf, and the printed module would be quite large. Thank you to @paolosevMSFT for identifying and suggesting the fix.
* Trap when call_indirect's signatures mismatch (#2636)Heejin Ahn2020-02-031-1/+1
| | | | | | | | | | | This makes the interpreter trap when the signature in `call_indirect` instruction and that of the actual function in the table mismatch. This also makes the `wasm-ctor-eval` not evaluate `call_indirect` in case the signatures mismatch. Before we only compared the arguments' signature and the function signature, which was sufficient before we had subtypes, but now the signature in `call_indirect` and that of the actual function can be different even if the argument's signature is OK.
* Verify --version output matches CHANGELOG (#2580)Sam Clegg2020-01-101-0/+15
| | | | | | | | | | | | | | | | | The new version string looks like this: wasm-opt version 90 (version_90-18-g77329439d) The version reported here is the version from the CMakeLists.txt file followed by the git version in brackets. We verify that the main version here matches the CHANGELOG to prevent people from changing one without changeing the other. This will help with emscripten that wants to be able to programaticaly check the --version of binaryen tools. See https://github.com/emscripten-core/emscripten/issues/10175
* Add support for reference types proposal (#2451)Heejin Ahn2019-12-301-6/+11
| | | | | | | | | | | | This adds support for the reference type proposal. This includes support for all reference types (`anyref`, `funcref`(=`anyfunc`), and `nullref`) and four new instructions: `ref.null`, `ref.is_null`, `ref.func`, and new typed `select`. This also adds subtype relationship support between reference types. This does not include table instructions yet. This also does not include wasm2js support. Fixes #2444 and fixes #2447.
* Binary format code section offset tracking (#2515)Alon Zakai2019-12-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Optionally track the binary format code section offsets, that is, when loading a binary, remember where each IR node was read from. This is necessary for DWARF debug info, as these are the offsets DWARF refers to. (Note that eventually we may want to do something else, like first read the DWARF and only then add debug info annotations into the IR in a more LLVM-like manner, but this is more straightforward and should be enough to update debug lines and ranges). This tracking adds noticeable overhead - every single IR node adds an entry in a map - so avoid it unless actually necessary. Specifically, if the user passes in -g and there are actually DWARF sections in the binary, and we are not about to remove those sections, then we need it. Print binary format code section offsets in text, when printing with -g. This will help debug and test dwarf support. It looks like ;; code offset: 0x7 as an annotation right before each node. Also add support for -g in wasm-opt tests (unlike a pass, it has just one - as a prefix). Helps #2400
* Allow test/passes tests to have arbitrary names, with a side file that ↵Alon Zakai2019-12-161-2/+3
| | | | | | contains the passes (#2532) We already supported this, but required that the filename be a number. This lets the name be anything, and we check if *.passes exists for it.
* Use wat over wast for text format filenames (#2518)Sam Clegg2019-12-081-11/+11
|
* Refactor removing module elements (#2489)Heejin Ahn2019-12-021-2/+2
| | | | | | | | | | | This creates utility functions for removing module elements: removing one element by name, and removing multiple elements using a predicate function. And makes other parts of code use it. I think this is a light-handed approach than calling `Module::updateMaps` after removing only a part of module elements. This also fixes a bug in the inlining pass: it didn't call `Module::updateMaps` after removing functions. After this patch callers don't need to additionally call it anyway.
* Update spec test suite (#2484)Heejin Ahn2019-11-291-12/+1
| | | | | | | | | | | | | This updates spec test suite to that of the current up-to-date version of https://github.com/WebAssembly/spec repo. - All failing tests are added in `BLACKLIST` in shared.py with reasons. - For tests that already existed and was passing and started failing after the update, we add the new test to the blacklist and preserve the old file by renaming it to 'old_[FILENAME].wast' not to lose test coverage. When the cause of the error is fixed or the unsupported construct gets support so the new test passes, we can delete the corresponding 'old_[FILENAME].wast' file. - Adds support for `spectest.print_[type] style imports.
* Auto-update spec test outputs (#2481)Heejin Ahn2019-11-261-17/+1
| | | | | | | | This makes auto_update_tests.py update spec test outputs (ones that are printed with `spectest.print` import) and extracts spec tests blacklist into shared.py with comments for reasons why each of them fails. Also deletes if-label-scope.fail.wast.log because it does not seem to match with any of existing tests.
* Remove vanilla tests (#2482)Heejin Ahn2019-11-261-47/+0
| | | These have not been used in years and seem outdated.
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-03 13:26:16 +0000