summaryrefslogtreecommitdiff
path: root/scripts/fuzz_opt.py
Commit message (Collapse)AuthorAgeFilesLines
* [NFC] Move more logic about unfuzzable tests to a shared location (#7175)Alon Zakai2024-12-201-19/+1
| | | | | | | It turns out that #7165 was not enough because we had a second (!?) list of tests to ignore, and also a condition. Move all that to the shared location as well, continuing that PR. Also remove simd.wast from the list, as that issue has been fixed.
* Update flake8 and fix errors (#7172)Thomas Lively2024-12-201-1/+1
| | | | | | | | | The flake8 we were running on CI was too old and began giving spurious errors about the uninterpreted contents of f-strings. Update to the latest flake8 and fix all the new errors, including the previously incorrect comment syntax in the .flake8 file. Also remove scripts/storage.py, since it didn't seem to be used for anything we currently need.
* [NFC] Move code from fuzz_opt.py to a shared place (#7165)Alon Zakai2024-12-191-73/+5
| | | | | The list of tests, and which tests cannot be fuzzed, will be useful in a future PR that uses it for ClusterFuzz as well. This just moves things out so they are reusable.
* [NFC] Improve ClusterFuzz testing (#7157)Alon Zakai2024-12-181-4/+9
| | | | | | | 1. Error on retrying due to a wasm-opt issue, locally (in production, we don't want to error on ClusterFuzz). 2. Move some asserts from test_run_py to the helper generate_testcases (so that the asserts happen in all callers).
* Fuzz JSPI (#7148)Alon Zakai2024-12-161-5/+41
| | | | | | | | | | * Add a new "sleep" fuzzer import, that does a sleep for some ms. * Add JSPI support in fuzz_shell.js. This is in the form of commented-out async/await keywords - commented out so that normal fuzzing is not impacted. When we want to fuzz JSPI, we uncomment them. We also apply the JSPI operations of marking imports and exports as suspending/promising. JSPI fuzzing is added to both fuzz_opt.py and ClusterFuzz's run.py.
* Fuzz combinations of two modules (#7144)Alon Zakai2024-12-101-2/+80
| | | | | | | | | | | The new Two fuzz testcase handler generates two wasm files and then runs them in V8, linking them using JS. It then optimizes at least one of the two and runs it again, and checks for differences. This is similar to the Split fuzzer for wasm-split, but the two wasm files are arbitrary and not the result of splitting. Also lower CtorEval priority a little. It is not very important.
* [NFC] Send the closed-world flag to TranslateToFuzzReader (#7136)Alon Zakai2024-12-051-4/+6
| | | | | | | | | | | This sends --closed-world to wasm-opt from the fuzzer, when we use that flag (before we just used it on optimizations, but not fuzz generation). And TranslateToFuzzReader now stores a boolean about whether we are in closed- world mode or not. This has no effect so far, and is a refactoring for a later PR, where we must generate code differently based on whether we are in closed-world mode or not.
* [Fuzzing] Maximize fuzzing of CompareVMs (#7113)Alon Zakai2024-11-251-1/+1
| | | | This increases our testing of V8, and helps find bugs where we are different than V8.
* Fuzzer: Remove --enclose-world from list of fuzz-exec passes (#7093)Alon Zakai2024-11-201-1/+0
| | | | | | | It is ok to use this pass to shape the wasm, but not to test for changes using fuzz-exec, as the pass is destructive: it can alter observable behavior. Specifically, it adds casts which can trap in the wasm, which can replace a JS exception which happens outside, and that difference is noticeable.
* Fuzzer: Use V8's --future flag (#7091)Alon Zakai2024-11-201-3/+13
|
* Improve fuzzing of both closed and open world styles of modules (#7090)Alon Zakai2024-11-191-0/+6
| | | | | | | | | | Before, we would simply not export a function that had an e.g. anyref param. As a result, the modules were effectively "closed", which was good for testing full closed-world mode, but not for testing degrees of open world. To improve that, this PR allows the fuzzer to export such functions, and an "enclose world" pass is added that "closes" the wasm (makes it more compatible with closed-world) that is run 50% of the time, giving us coverage of both styles.
* Fuzzing: ClusterFuzz integration (#7079)Alon Zakai2024-11-191-1/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main addition here is a bundle_clusterfuzz.py script which will package up the exact files that should be uploaded to ClusterFuzz. It also documents the process and bundling and testing. You can do bundle.py OUTPUT_FILE.tgz That bundles wasm-opt from ./bin., which is enough for local testing. For actually uploading to ClusterFuzz, we need a portable build, and @dschuff had the idea to reuse the emsdk build, which works nicely. Doing bundle.py OUTPUT_FILE.tgz --build-dir=/path/to/emsdk/upstream/ will bundle wasm-opt (+libs) from the emsdk. I verified that those builds work on ClusterFuzz. I added several forms of testing here. First, our main fuzzer fuzz_opt.py now has a ClusterFuzz testcase handler, which simulates a ClusterFuzz environment. Second, there are smoke tests that run in the unit test suite, and can also be run separately: python -m unittest test/unit/test_cluster_fuzz.py Those unit tests can also run on a given bundle, e.g. one created from an emsdk build, for testing right before upload: BINARYEN_CLUSTER_FUZZ_BUNDLE=/path/to/bundle.tgz python -m unittest test/unit/test_cluster_fuzz.py A third piece of testing is to add a --fuzz-passes test. That is a mode for -ttf (translate random data into a valid wasm fuzz testcase) that uses random data to pick and run a set of passes, to further shape the wasm. (--fuzz-passes had no previous testing, and this PR fixes it and tidies it up a little, adding some newer passes too). Otherwise this PR includes the key run.py script that is bundled and then executed by ClusterFuzz, basically a python script that runs wasm-opt -ttf [..] to generate testcases, sets up their JS, and emits them. fuzz_shell.js, which is the JS to execute testcases, will now check if it is provided binary data of a wasm file. If so, it does not read a wasm file from argv[1]. (This is needed because ClusterFuzz expects a single file for the testcase, so we make a JS file with bundled wasm inside it.)
* [EH] Fuzz calls from JS by calling wasm exports, sometimes catching (#7067)Alon Zakai2024-11-081-0/+28
| | | | | | | | | | | | | | | | This adds two new imports to fuzzer modules: * call-export, which gets an export index and calls it. * call-export-catch, which does the call in a try-catch, swallowing any error, and returning 1 if it saw an error. The former gives us calls back into the wasm, possibly making various trips between wasm and JS in interesting ways. The latter adds a try-catch which helps fuzz wasm EH. We do these calls using a wasm export index, i.e., the index in the list of exports. This is simple, but it does have the downside that it makes executing the wasm sensitive to changes in exports (e.g. wasm-merge adds more), which requires some handling in the fuzzer.
* [NFC] Refactor fuzzer's can_run_on_feature_opts() (#7066)Alon Zakai2024-11-071-6/+7
| | | | | | | It never used the parameter, so remove that (we always access the features using a global anyhow). But add a new parameter, the wasm file, which does need to be passed in for a later PR (so in this PR it is just for future use).
* [wasm64] Fuzz wasm64 memories (#7064)Alon Zakai2024-11-071-3/+3
| | | | | | | * Remove the code that prevented fuzzing wasm64 test files. * Ignore a run that hits the V8 implementation limit on memory size. * Disable wasm64 fuzzing in wasm2js (like almost all post-MVP features). * Add fuzzer logic to emit a 64-bit memory sometimes. * Fix various places in the fuzzer that assumed 32-bit indexes
* Fix flipped condition on keep_defaults in fuzzer (#7061)Alon Zakai2024-11-061-1/+1
| | | Followup to #7055
* Fuzzer: Handle exported table for wasm-merge (#7055)Alon Zakai2024-11-061-7/+27
| | | | | | When fuzzing wasm-merge, we need to avoid the first module not having an exported table but the second having one, as the way the table operation imports work, they are sensitive to the existence of such an export, so just merging in such an export can alter behavior.
* Fuzz the Table from JS (#7042)Alon Zakai2024-10-311-11/+28
| | | | | Continues the work from #7027 which added throwing from JS, this adds table get/set operations from JS, to further increase our coverage of Wasm/JS interactions (the table can be used from both sides).
* wasm-split: Add fuzzer support (#7014)Alon Zakai2024-10-181-2/+109
| | | | | | | | | | | | The support is added but not enabled as this is still finding bugs. The first part here is to add Split testcase handler to the fuzzer, which runs a wasm, then runs it again after splitting it and then linking it at runtime, and checking for different results. The second part is support for linking two modules at runtime in the fuzzer's JS code, that works in tandem with the first part. New options are added to load and link a second wasm, and to pick which exports to run.
* Fuzz Wasm Exceptions in V8 (#6981)Alon Zakai2024-10-021-3/+1
| | | | | The blocking bug https://issues.chromium.org/issues/332931390 has been fixed.
* [NFC] Move optimizeSubsequentStructSet() to a new pass, ↵Alon Zakai2024-09-031-0/+1
| | | | | | | | | | | | | | | | | | | HeapStoreOptimization (#6882) This just moves code out of OptimizeInstructions to the new pass. The existing test is renamed and now runs the new pass instead. The new pass is run right after each --optimize-instructions invocation, so it should not cause any noticeable effects whatsoever, making this NFC. The motivation here is that there is a bug in the pass, see the new testcase added at the end, which shows the bug. It is not practical to fix that bug in OptimizeInstructions since we need more than peephole optimizations to do so. This PR moves the code to a new pass so we can fix it there properly, later. The new pass is named HeapStoreOptimization since the same infrastructure we will need to fix the bug will also help dead store elimination and related things.
* Ignore fp16 in the fuzzer (#6881)Alon Zakai2024-08-291-1/+2
| | | | Add the feature flag in V8 invocations, but also disable the feature as it isn't quite ready yet.
* [Exceptions] Finish interpreter + optimizer support for try_table. (#6814)Sébastien Doeraene2024-08-201-6/+7
| | | | | | * Add interpreter support for exnref values. * Fix optimization passes to support try_table. * Enable the interpreter (but not in V8, see code) on exceptions.
* Add a pass for minimizing recursion groups (#6832)Thomas Lively2024-08-171-0/+1
| | | | | | | | | | | | Most of our type optimization passes emit all non-public types as a single large rec group, which trivially ensures that different types remain different, even if they are optimized to have the same structure. Usually emitting a single large rec group is fine, but it also means that if the module is split, all of the types will need to be repeated in all of the split modules. To better support this use case, add a pass that can split the large rec group back into minimal rec groups, taking care to preserve separate type identities by emitting different permutations of the same group where possible or by inserting unused brand types to differentiate them.
* Monomorphization: Add a flag to control the required improvement (#6837)Alon Zakai2024-08-141-1/+4
| | | | | | | | | | | | | | | | | | | The argument is the minimum benefit we must see for us to decide to optimize, e.g. --monomorphize --pass-arg=monomorphize-min-benefit@50 When the minimum benefit is 50% then if we reduce the cost by 50% through monomorphization then we optimize there. 95% would only optimize when we remove almost all the cost, etc. In practice I see 95% will actually tend to reduce code size overall, as while we add monomorphized versions of functions, we only do so when we remove a lot of work and size, and after inlining we gain benefits. However, 50% or even lower can lead to better benchmark results, in return for larger code size, just like with inlining. To be careful, the default is set to 95%. Previously we optimized whenever we saw any benefit at all, which is the same as requiring a minimum benefit of 0%. Old tests have the flag applied in this PR to set that value, so they do not change.
* [FP16] Disable float 16 fuzzing for now. (#6822)Brendan Dahl2024-08-071-0/+2
|
* [threads] Update the fuzzer for shared types (#6771)Thomas Lively2024-07-181-17/+9
| | | | | | | | Update the fuzzer to both handle shared types in initial contents and create and use new shared types without crashing or producing invalid modules. Since V8 does not have a complete implementation of shared-everything-threads yet, disable fuzzing V8 when shared-everything is enabled. To avoid losing too much coverage of V8, disable shared-everything in the fuzzer more frequently than other features.
* Validate features for types used in element segments (#6769)Thomas Lively2024-07-181-0/+1
|
* Validate features for types used in tables (#6768)Thomas Lively2024-07-181-0/+1
| | | | We previously special-cased things like GC types, but switch to a more general solution of detecting what features a table's type requires.
* [threads] ref.i31_shared requires shared-everything in validation (#6767)Thomas Lively2024-07-181-0/+1
|
* [threads] Simplify and generalize reftype writing without GC (#6766)Thomas Lively2024-07-181-1/+1
| | | | | | Similar to #6765, but for types instead of heap types. Generalize the logic for transforming written reference types to types that are supported without GC so that it will automatically handle shared types and other new types correctly.
* [threads] Simplify and generalize heap type writing without GC (#6765)Thomas Lively2024-07-171-0/+1
| | | | | | | | | | We represent `ref.null`s as having bottom heap types, even when GC is not enabled. Bottom heap types are a feature of the GC proposal, so in that case the binary writer needs to write the corresponding top type instead. We previously had separate logic for this for each type hierarchy in the binary writer, but that did not handle shared types and would not have automatically handled other new types, either. Simplify and generalize the implementation and test that we can write `ref.null`s of shared types without GC enabled.
* [threads] Fix shared ref.eq and disallow mixed-shareability (#6763)Thomas Lively2024-07-171-0/+1
| | | | | | | Update the validator to reject mixed-shareability ref.eq, although this is still under discussion in https://github.com/WebAssembly/shared-everything-threads/issues/76. Fix the implementation of `Literal::operator==` to work properly with shared i31ref.
* [threads] Validate all features required by ref.null (#6757)Thomas Lively2024-07-161-0/+2
| | | | | | | `ref.null` of shared types should only be allowed when shared-everything is enabled, but we were previously checking only that reference types were enabled when validating `ref.null`. Update the code to check all features required by the null type and factor out shared logic for printing lists of missing feature options in error messages.
* [NFC][threads] Ignore type-ssa-shared.wast in fuzzer (#6754)Thomas Lively2024-07-161-0/+1
| | | | The fuzzer does not yet properly handle initial contents containing shared types.
* [threads] ref.i31_shared (#6735)Thomas Lively2024-07-121-0/+1
| | | | | | | Implement `ref.i31_shared` the new instruction for creating references to shared i31s. Implement binary and text parsing and emitting as well as interpretation. Copy the upstream spec test for i31 and modify it so that all the heap types are shared. Comment out some parts that we do not yet support.
* [StackIR] Allow StackIR to be disabled from the commandline (#6725)Alon Zakai2024-07-101-0/+1
| | | | | | | | | Normally we use it when optimizing (above a certain level). This lets the user prevent it from being used even then. Also add optimization options to wasm-metadce so that this is possible there as well and not just in wasm-opt (this also opens the door to running more passes in metadce, which may be useful later).
* [threads] Ignore shared-array.wast in fuzzer initial contents (#6706)Thomas Lively2024-06-261-0/+1
|
* [threads] Validate shared-polymorphic instructions (#6702)Thomas Lively2024-06-251-0/+1
| | | | Such as `ref.eq`, `i31.get_{s,u}`, and `array.len`. Also validate that struct and array operations work on shared structs and arrays.
* [threads] Binary reading and writing of shared composite types (#6664)Thomas Lively2024-06-141-0/+4
| | | | Also update the parser so that implicit type uses are not matched with shared function types.
* Fuzzer: Stop testing with TurboFan as Turboshaft is rolling out and is ↵Alon Zakai2024-05-281-7/+0
| | | | faster (#6623)
* Fix binary emitting of br_if with a refined value by emitting a cast (#6510)Alon Zakai2024-05-161-1/+18
| | | | | | | | | | | | | | | | This makes us compliant with the wasm spec by adding a cast: we use the refined type for br_if fallthrough values, and the wasm spec uses the branch target. If the two differ, we add a cast after the br_if to make things match. Alternatively we could match the wasm spec's typing in our IR, but we hope the wasm spec will improve here, and so this is will only be temporary in that case. Even if not, this is useful because by using the most refined type in the IR we optimize in the best way possible, and only suffer when we emit fixups in the binary, but in practice those cases are very rare: br_if is almost always dropped rather than used, in real-world code (except for fuzz cases and exploits). We check carefully when a br_if value is actually used (and not dropped) and its type actually differs, and it does not already have a cast. The last condition ensures that we do not keep adding casts over repeated roundtripping.
* Fuzzer: Add another stringview subtyping error message to ignore (#6575)Alon Zakai2024-05-091-0/+1
|
* Fuzzer: Stop emitting nullable stringviews (#6574)Alon Zakai2024-05-081-0/+3
| | | | | | | | | | | | | As of https://chromium-review.googlesource.com/c/v8/v8/+/5471674 V8 requires stringviews to be non-nullable. It might be possible to make that change in our IR, or to remove views entirely, but for now this PR makes the fuzzer stop emitting nullable stringviews as a workaround to allow us to fuzz current V8. There are still rare corner cases where this pattern is emitted, that we have not tracked down, and so this also makes the fuzzer ignore the error for now.
* Re-enable fuzzing of text round trips (#6560)Thomas Lively2024-04-291-2/+1
| | | The bug that had been preventing this fuzzing no longer reproduces.
* [Strings] Add the string heaptype to core fuzzer places (#6527)Alon Zakai2024-04-231-1/+2
| | | | | | | With this we emit strings spontaneously (as opposed to just getting them from initial contents). The relevant -ttf test has been tweaked slightly to show the impact of this change: now there are some string.new/const in the output.
* Fuzzer: Do not run other VMs if the Binaryen interpreter hits a VM ↵Alon Zakai2024-04-101-10/+19
| | | | | | | | | limitation (#6483) The VM limitation might be an OOM (which can change due to opts) or an atomic wait (which can hang on proper VMs with support). We already avoided running VMs on the optimized wasm in this case, but we still ran them on the original wasm, which this changes, mainly to avoid that atomic wait situation.
* [NFC] Fix fuzzer counting of ignored runs due to many errors (#6444)Alon Zakai2024-03-271-4/+12
| | | | | | When the interpreter sees that most exports simply trap we mark the iteration as ignored. But we run the interpreter on the before wasm and also the after wasm, so we were incrementing that counter by 2 each time, which could be misleading.
* [Strings] Represent string values as WTF-16 internally (#6418)Thomas Lively2024-03-221-3/+0
| | | | | | | | | | | | | | | | WTF-16, i.e. arbitrary sequences of 16-bit values, is the encoding of Java and JavaScript strings, and using the same encoding makes the interpretation of string operations trivial, even when accounting for non-ascii characters. Specifically, use little-endian WTF-16. Re-encode string constants from WTF-8 to WTF-16 in the parsers, then back to WTF-8 in the writers. Update the constructor for string `Literal`s to interpret the string as WTF-16 and store a sequence of WTF-16 code units, i.e. 16-bit integers. Update `Builder::makeConstantExpression` accordingly to convert from the new `Literal` string representation back to a WTF-16 string. Update the interpreter to remove the logic for detecting non-ascii characters and bailing out. The naive implementations of all the string operations are correct now that our string encoding matches the JS string encoding.
* Update file name in INITIAL_CONTENTS_IGNORE (#6425)Thomas Lively2024-03-221-1/+1
| | | | The test file was renamed, but the fuzzer still used the old name in INITIAL_CONTENTS_IGNORE.
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-05 20:29:25 +0000