summaryrefslogtreecommitdiff
path: root/check.py
Commit message (Collapse)AuthorAgeFilesLines
* Add BUILD_TESTS CMake option (#4536)Thomas Lively2022-03-211-5/+8
| | | | Turning it off removes the build dependency on the third-party googletest library.
* Introduce gtest (#4466)Thomas Lively2022-01-201-2/+18
| | | | | | | | | | | | | | | | | | | | | | | | Add gtest as a git submodule in third_party and integrate it into the build the same way WABT does. Adds a new executable, `binaryen-unittests`, to execute `gtest_main`. As a nontrivial example test, port one of the `TypeBuilder` tests from example/ to gtest/. Using gtest has a number of advantages over the current example tests: - Tests are compiled and linked at build time rather than runtime, surfacing errors earlier and speeding up test execution. - Tests are all built into a single binary, reducing overall link time and further reducing test overhead. - Tests are built from the same CMake project as the rest of Binaryen, so compiler settings (e.g. sanitizers) are applied uniformly rather than having to be separately set via the COMPILER_FLAGS environment variable. - Using the industry-standard gtest rather than our own script reduces our maintenance burden. Using gtest will lower the barrier to writing C++ tests and will hopefully lead to us having more proper unit tests.
* [ctor-eval] Add an option to keep some exports (#4441)Alon Zakai2022-01-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | By default wasm-ctor-eval removes exports that it manages to completely eval (if it just partially evals then the export remains, but points to a function with partially-evalled contents). However, in some cases we do want to keep the export around even so, for example during fuzzing (as the fuzzer wants to call the same exports before and after wasm-ctor-eval runs) and also if there is an ABI we need to preserve (like if we manage to eval all of main()), or if the function returns a value (which we don't support yet, but this is a PR to prepare for that). Specifically, there is now a new option: --kept-exports foo,bar That is a list of exports to keep around. Note that when we keep around an export after evalling the ctor we make the export point to a new function. That new function just contains a nop, so that nothing happens when it is called. But the original function is kept around as it may have other callers, who we do not want to modify.
* [ctor-eval] Add --ignore-external-input option (#4428)Alon Zakai2022-01-061-0/+2
| | | | | | | | | | | | This is meant to address one of the main limitations of wasm-ctor-eval in emscripten atm, that libc++ global ctors will read env vars, which means they call an import, which stops us from evalling, emscripten-core/emscripten#15403 (comment) To handle that, this adds an option to ignore external input. When set, we can assume that no env vars will be read, no reading from stdin, no arguments to main(), etc. Perhaps these could each be separate options, but I think keeping it simple for now might be good enough.
* Add DS_Store to non-executable filter in check.py (#3999)Max Graey2021-07-161-1/+1
|
* Lit tests for tool help messages (#3965)Thomas Lively2021-07-071-15/+3
| | | | | | Add list tests for the help messages of all tools, factoring out common options into shared tests. This is slightly brittle because the text wrapping depends on the length of the longest option, but that brittleness should be worth the benefit of being able to see the actual help text in the tests.
* Fix 'example' tests in check.py (#3920)Thomas Lively2021-06-071-5/+5
| | | | | | Rename the test suite from 'gcc' to 'example' to match the name in auto_update_tests.py. Also retrieve the tests to run using `shared.get_tests`, which properly applies the --filter argument, again to match auto_update_tests.py.
* Run lit tests with pass-debug as well, to check for more errors (#3836)Alon Zakai2021-04-221-9/+12
|
* Run spec test all at once after binary transform (#3817)Abbas Mashayekh2021-04-201-13/+15
| | | | | | | | | | | | | | | | | | #3792 added support for module linking and (register command to wasm-shell, but forgot about three problems: - Splitting spec tests prevents linking test modules together. - Registered modules may still be used in assertions or an invoke - Modules may re-export imported objects This PR appends transformed modules after binary checks to a spec.wast file, plus assertion tests and register commands. Then runs wasm-shell on the whole file. It also keeps both the module name and its registered name available in wasm-shell for use in shell commands and linked modules. Furthermore, it correctly finds the module where an object is defined even if it is imported and re-exported several times. The updated version of imports.wast spec test is enabled to verify the fixes.
* Update interpreter for new EH spec (#3498)Heejin Ahn2021-01-211-3/+0
| | | | | | | | | | | | | This updates the interpreter for the EH instructions (modulo `delegate`) to match the new spec. Before we had an `exnref` type so threw a `Literal` of `exnref` type which contained `ExceptionPackage`. But now that we don't have `exnref` anymore, so we add the contents of `ExceptionPackage` to `WasmException`, which is used only for the `ExpressionRunner` class hierarchy. `exnref` and `ExceptionPackage` will be removed in a followup CL. This allows nonzero depths for `rethrow` for now for testing; we disallowed that for safety measure, but given that there are no passes that modifies that field, I think the risk is low.
* wasm-reduce: Fix setting of feature flags after loading (#3493)Alon Zakai2021-01-151-2/+2
| | | | | We mistakenly did not set the flags to all, which meant that if the features section was not present, we'd not have the proper features set, leading to errors on writing.
* Basic EH instrucion support for the new spec (#3487)Heejin Ahn2021-01-151-0/+3
| | | | | | | | | | | | | | | | | | | | This updates `try`-`catch`-`catch_all` and `rethrow` instructions to match the new spec. `delegate` is not included. Now `Try` contains not a single `catchBody` expression but a vector of catch bodies and events. This updates most existing routines, optimizations, and tests modulo the interpreter and the CFG traversal. Because the interpreter has not been updated yet, the EH spec test is temporarily disabled in check.py. Also, because the CFG traversal for EH is not yet updated, several EH tests in `rse_all-features.wast`, which uses CFG traversal, are temporarily commented out. Also added a few more tests in existing EH test functions in test/passes. In the previous spec, `catch` was catching all exceptions so it was assumed that anything `try` body throws is caught by its `catch`, but now we can assume the same only if there is a `catch_all`. Newly added tests test cases when there is a `catch_all` and cases there are only `catch`es separately.
* Improve lit support (#3426)Sam Clegg2020-12-091-4/+4
| | | | | | | | This uses the same technique used in llvm-lit to enable running on in-tree tests with out-of-tree builds. So you can run something like this: ../binaryen-out/bin/binaryen-lit test/lit/
* Introduce lit/FileCheck tests (#3367)Thomas Lively2020-11-181-0/+13
| | | | | | | | | | | | | | | lit and FileCheck are the tools used to run the majority of tests in LLVM. Each lit test file contains the commands to be run for that test, so lit tests are much more flexible and can be more precise than our current ad hoc testing system. FileCheck reads expected test output from comments, so it allows test output to be written alongside and interspersed with test input, making tests more readable and precise than in our current system. This PR adds a new suite to check.py that runs lit tests in the test/lit directory. A few tests have been ported to demonstrate the features of the new test runner. This change is motivated by a need for greater flexibility in testing wasm-split. See #3359.
* Allow setting the path to libbinaryen.so shared library (#3025)rathann2020-11-101-1/+1
| | | | This makes it easier to install libbinaryen.so into an alternative locations. Fixes part of issue #2999 for me.
* Remove remaining bits of C API tracing, which has been removed (#3322)Alon Zakai2020-11-051-30/+17
| | | It was removed in #2841
* Remove remnants of spec interpreter, and excess mozjs warning (#3262)Alon Zakai2020-10-201-5/+0
| | | | | The spec interpreter is no longer used at all. Mozjs is still used optionally, but while it was crucial in the past for test coverage, it is entirely optional now and not run by default, so no need to warn.
* Improve testing on Windows (#3142)Wouter van Oortmerssen2020-09-171-4/+0
| | | | | | This PR contains: - Changes that enable/disable tests on Windows to allow for better local testing. - Also changes many abort() into Fatal() when it is really just exiting on error. This is because abort() generates a dialog window on Windows which is not great in automated scripts. - Improvements to CMake to better work with the project in IDEs (VS).
* Upgrade to C++17 (#3103)Daniel Wirtz2020-09-061-2/+2
|
* Remove asm2wasm (#3042)Alon Zakai2020-08-171-3/+0
| | | | | | | Now that fastcomp has been removed from Emscripten, there is no need for the asm2wasm tool which it used to compile fastcomp's asm.js output to wasm. See emscripten-core/emscripten#11860
* Skip tests that fail on windows and enable all the rest (#3035)Alon Zakai2020-08-111-6/+17
| | | | | | | | | | | | | | This lets us run most tests at least on that platform. Add a new function for skipping those tests, skip_if_on_windows, so that it's easy to find which tests are disabled on windows for later fixing efforts. This fixes a few minor issues for windows, like comparisons should ignore \r in some cases. Rename all passes tests that use --dwarfdump to contain "dwarf" in their name, which makes it easy to skip those (and is clearer anyhow).
* Remove unneeded old binary validation in test suite (#3034)Alon Zakai2020-08-101-2/+0
| | | | | | | | This was useful back when we didn't have many VMs to test in, and we weren't confident in our binaries being valid wasm. Today we have lots of testing on VMs, and in particular, this test tends to fail when we do things like reorder SIMD opcode constants. (This doesn't fail on CI as we don't have v8 installed there, so this path is never reached, but it does happen locally.)
* Put validator test outputs in out/test (#2882)Heejin Ahn2020-05-311-6/+6
| | | | | | | We now put outputs of all other tests in out/test in order not to pollute the test directory, but validator tests didn't have a output file specified so their output files were written in test/validator. This adds `-o a.wasm` to validator tests command lines, in the same way as other tests, to make them go into out/test directory.
* Fix issues with cleanup dSYM in scripts for MacOS (#2871)Max Graey2020-05-261-5/+0
| | | | Remove rmtree call from check.py & auto_update_tests.py
* Use --detect-features in wasm-reduce. Fixes #2813 (#2815)Alon Zakai2020-04-281-3/+3
|
* Fix binaryenjs testing (#2810)Sam Clegg2020-04-271-2/+14
| | | | | | These tests are now optional. However, if you run them and the build is not found they will now error out, in order to avoid silently failing.
* Convert CI from travis + appveyor to github actions (#2646)Sam Clegg2020-04-211-1/+1
| | | | | | The intention is to move away from travis and appveyor which have become very slow. See: #2356
* Emit tuples in the fuzzer (#2695)Thomas Lively2020-04-151-2/+2
| | | | | | | | Emit tuple.make, tuple.extract, and multivalue control flow, and tuple locals and globals when multivalue is enabled. Also slightly refactors the top-level `makeConcrete` function to be more selective about what it tries to make based on the requested type to reduce the number of trivial nodes created because the requested type is incompatible with the requested node.
* Speed up wasm-reduce test (#2756)Alon Zakai2020-04-131-2/+2
| | | | | | The test there just wants to see that the reducer can remove a significant amount of code. I changed it from a file of 3.6K to 200 bytes, which is enough to see the effect, and much faster.
* Use std::cout for interpreter trap logging (#2755)Alon Zakai2020-04-131-1/+5
| | | | | | | | | | | | | | We used std::cerr as a workaround for that this logging interfered with spec testing. But it's easy enough to filter out this stuff for the spec tests. The benefit to using std::cout is that as you can see in the test output here, this is relevant test output - it's not a side channel for debugging. If the rest of the interpreter output is in std::cout but only traps are in std::cerr then they might end up out of order etc., so best to keep them all together. This will allow easier additions of tests for fuzz testcases
* Factor out wasm_opt testing. NFC. (#2737)Sam Clegg2020-04-091-108/+2
| | | | Also, factor out auto-updating of binaryenjs testing so it lives alongside the actual test code.
* Update Precompute to handle tuples (#2687)Thomas Lively2020-03-101-1/+1
| | | | | | This involves replacing `Literal::makeZero` with `Literal::makeZeroes` and `Literal::makeSingleZero` and updating `isConstantExpression` to handle constant tuples as well. Also makes `Literals` its own struct and adds convenience methods on it.
* DWARF: Fix debug_range handling of invalid entries (#2662)Alon Zakai2020-02-181-1/+3
| | | | | | | | | | | | | | If an invalid entry appears - either it began as such, or became invalid after optimization - we should not emit (0, 0) which is an end marker. Instead, emit an invalid entry marker, something with (0, x) for x != 0. As a bonus, if a test/passes case has "noprint" in the name, don't print the wasm, which we do by default. In the testcase here for example we just care about the dwarf, and the printed module would be quite large. Thank you to @paolosevMSFT for identifying and suggesting the fix.
* Trap when call_indirect's signatures mismatch (#2636)Heejin Ahn2020-02-031-1/+1
| | | | | | | | | | | This makes the interpreter trap when the signature in `call_indirect` instruction and that of the actual function in the table mismatch. This also makes the `wasm-ctor-eval` not evaluate `call_indirect` in case the signatures mismatch. Before we only compared the arguments' signature and the function signature, which was sufficient before we had subtypes, but now the signature in `call_indirect` and that of the actual function can be different even if the argument's signature is OK.
* Verify --version output matches CHANGELOG (#2580)Sam Clegg2020-01-101-0/+15
| | | | | | | | | | | | | | | | | The new version string looks like this: wasm-opt version 90 (version_90-18-g77329439d) The version reported here is the version from the CMakeLists.txt file followed by the git version in brackets. We verify that the main version here matches the CHANGELOG to prevent people from changing one without changeing the other. This will help with emscripten that wants to be able to programaticaly check the --version of binaryen tools. See https://github.com/emscripten-core/emscripten/issues/10175
* Add support for reference types proposal (#2451)Heejin Ahn2019-12-301-6/+11
| | | | | | | | | | | | This adds support for the reference type proposal. This includes support for all reference types (`anyref`, `funcref`(=`anyfunc`), and `nullref`) and four new instructions: `ref.null`, `ref.is_null`, `ref.func`, and new typed `select`. This also adds subtype relationship support between reference types. This does not include table instructions yet. This also does not include wasm2js support. Fixes #2444 and fixes #2447.
* Binary format code section offset tracking (#2515)Alon Zakai2019-12-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Optionally track the binary format code section offsets, that is, when loading a binary, remember where each IR node was read from. This is necessary for DWARF debug info, as these are the offsets DWARF refers to. (Note that eventually we may want to do something else, like first read the DWARF and only then add debug info annotations into the IR in a more LLVM-like manner, but this is more straightforward and should be enough to update debug lines and ranges). This tracking adds noticeable overhead - every single IR node adds an entry in a map - so avoid it unless actually necessary. Specifically, if the user passes in -g and there are actually DWARF sections in the binary, and we are not about to remove those sections, then we need it. Print binary format code section offsets in text, when printing with -g. This will help debug and test dwarf support. It looks like ;; code offset: 0x7 as an annotation right before each node. Also add support for -g in wasm-opt tests (unlike a pass, it has just one - as a prefix). Helps #2400
* Allow test/passes tests to have arbitrary names, with a side file that ↵Alon Zakai2019-12-161-2/+3
| | | | | | contains the passes (#2532) We already supported this, but required that the filename be a number. This lets the name be anything, and we check if *.passes exists for it.
* Use wat over wast for text format filenames (#2518)Sam Clegg2019-12-081-11/+11
|
* Refactor removing module elements (#2489)Heejin Ahn2019-12-021-2/+2
| | | | | | | | | | | This creates utility functions for removing module elements: removing one element by name, and removing multiple elements using a predicate function. And makes other parts of code use it. I think this is a light-handed approach than calling `Module::updateMaps` after removing only a part of module elements. This also fixes a bug in the inlining pass: it didn't call `Module::updateMaps` after removing functions. After this patch callers don't need to additionally call it anyway.
* Update spec test suite (#2484)Heejin Ahn2019-11-291-12/+1
| | | | | | | | | | | | | This updates spec test suite to that of the current up-to-date version of https://github.com/WebAssembly/spec repo. - All failing tests are added in `BLACKLIST` in shared.py with reasons. - For tests that already existed and was passing and started failing after the update, we add the new test to the blacklist and preserve the old file by renaming it to 'old_[FILENAME].wast' not to lose test coverage. When the cause of the error is fixed or the unsupported construct gets support so the new test passes, we can delete the corresponding 'old_[FILENAME].wast' file. - Adds support for `spectest.print_[type] style imports.
* Auto-update spec test outputs (#2481)Heejin Ahn2019-11-261-17/+1
| | | | | | | | This makes auto_update_tests.py update spec test outputs (ones that are printed with `spectest.print` import) and extracts spec tests blacklist into shared.py with comments for reasons why each of them fails. Also deletes if-label-scope.fail.wast.log because it does not seem to match with any of existing tests.
* Remove vanilla tests (#2482)Heejin Ahn2019-11-261-47/+0
| | | These have not been used in years and seem outdated.
* Print only literal values when printing literals (#2469)Heejin Ahn2019-11-261-18/+0
| | | | | | | | | | | | | | | Current `<<` operator on `Literal` prints `[type].const` with it. But `[type].const` is rather an instruction than a literal itself, and printing it with the literals makes less sense when we later have literals whose type don't have `const` instructions (such as reference types). This patch - Makes `<<` operator on `Literal` print only its value - Makes wasm-shell's shell interface comply with the spec interpreter's printing format (`value : type`). - Prints wasm-shell's `[trap]` message to stderr These make all `fix_` routines for spec tests in check.py unnecessary.
* Use package name in imports (NFC) (#2462)Heejin Ahn2019-11-221-149/+138
| | | | | Don't directly import names from shared.py and support.py, and use prefixes instead. Also this reorders imports based on PEP recommendation.
* Multivalue type creation and inspection (#2459)Thomas Lively2019-11-221-1/+1
| | | | | | | | | | | | | Adds the ability to create multivalue types from vectors of concrete value types. All types are transparently interned, so their representation is still a single uint32_t. Types can be extracted into vectors of their component parts, and all the single value types expand into vectors containing themselves. Multivalue types are not yet used in the IR, but their creation and inspection functionality is exposed and tested in the C and JS APIs. Also makes common type predicates methods of Type and improves the ergonomics of type printing.
* Simplify test scripts (NFC) (#2457)Heejin Ahn2019-11-201-255/+230
| | | | | | | | This makes test scripts simpler by reducing loop depths and extracting repeating code into methods or variables. - `get_tests` returns a list of tests with specified extensions. This includes files with a full path rather than just file names. - Reduces loop depths by using early exits and `get_tests`.
* Improve type selection in fuzzer (#2424)Heejin Ahn2019-11-061-1/+3
| | | | | | | | | - Adds `items` function for `FeatureOptions` so we can get a vector of eligible types - Replaces hardcoded enumeration of MVP types with `getConcreteTypes`, which also adds v128 type to the list if SIMD is enabled - Removes `getType()` function; this does not seem to be used anywhere - Renames `vectorPick` with `pick` - Use the absolute path for d8 in the fuzzer
* Add --out-dir to allow setting the test output dir. (#2389)Alon Zakai2019-10-171-2/+2
| | | This allows fuzzing in parallel invocations.
* Add feature flags and validation to wasm-metadce (#2364)Thomas Lively2019-09-271-1/+1
| | | | | | Sometimes wasm-metadce is the last tool to run over a binary in Emscripten, and in that case it needs to know what features are enabled in order to emit a valid binary. For example it needs to know whether to emit a data count section.
generated by cgit v1.2.3 (git 2.39.1) at 2025-08-06 15:47:19 +0000