| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
The parser function for `action` returned a `MaybeResult`, but we were
treating it as returning a normal `Result` and not checking that it had
contents in several places. Replace the current `action()` with
`maybeAction()` and add a new `action()` that requires the action to be
present.
Fixes #6872.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Spec tests use constants like `ref.array` and `ref.eq` to assert that
exported function return references of the correct types. Support more
such constants in the wast parser.
Also fix a bug where the interpretation of `array.new_data` for arrays
of packed fields was not properly truncating the packed data. Move the
function for reading fields from memory from literal.cpp to
wasm-interpreter.h, where the function for truncating packed data lives.
Other bugs prevent us from enabling any more spec tests as a result of
this change, but we can get farther through several of them before
failing. Update the comments about the failures accordingly.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Spec tests pass the value `ref.extern n`, where `n` is some integer,
into exported functions that expect to receive externrefs and receive
such values back out as return values. The payload serves to distinguish
externrefs so the test can assert that the correct one was returned.
Parse these values in wast scripts and represent them as externalized
i31refs carrying the payload. We will need a different representation
eventually, since some tests explicitly expect these externrefs to not
be i31refs, but this suffices to get several new tests passing.
To get the memory64 version of table_grow.wast passing, additionally fix
the interpreter to handle growing 64-bit tables correctly.
Delete the local versions of the upstream tests that can now be run
successfully.
|
| |
|
|
|
| |
Also use TableInit in the interpreter to initialize module's table
state, which will now handle traps properly, fixing #6431
|
| |
|
| |
Fixes #6833
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The syntax for handler clauses in `resume` instructions has recently
changed, using `on` instead of `tag` now.
Instead of
```
(resume $ct (tag $tag0 $block0) ... (tag $tagn $blockn))
```
we now have
```
(resume $ct (on $tag0 $block0) ... (on $tagn $blockn))
```
This PR adapts parsing, printing, and some tests accordingly.
(Note that this PR deliberately makes none of the other changes that
will arise from implementing the new, combined stack switching proposal,
yet.)
|
| |
|
|
|
|
| |
Match the current spec and clarify terminology by renaming the old
`deftype` to `rectype` and renaming the old `subtype` to `typedef`. Also
split the parser for actual `subtype` out of the parser for the newly
named `typedef`.
|
| |
|
|
|
|
| |
The type index from the TypeBuilder error was mapped to a file location
incorrectly, resulting in an assertion failure.
Fixes #6816.
|
| |
|
|
|
| |
The `timport$` prefix is already used for tables, so the binary parser
currently uses `eimport$` to name tags (I guess because they are
normally exception tags?).
|
| |
|
|
|
|
|
| |
Implement `ref.i31_shared` the new instruction for creating references
to shared i31s. Implement binary and text parsing and emitting as well
as interpretation. Copy the upstream spec test for i31 and modify it so
that all the heap types are shared. Comment out some parts that we do
not yet support.
|
| |
|
|
|
|
| |
Add an `isUTF8` utility and use it in both the text and binary parsers.
Add missing checks for overlong encodings and overlarge code points in
our WTF8 reader, which the new utility uses. Re-enable the spec tests
that test UTF-8 validation.
|
| |
|
|
|
|
| |
Fix the wast parser to accept IDs on quoted modules, remove tests that are
invalidated by the multimemory proposal, and add validation that the total
number of variables in a function is less than 2^32 and that the code section is
present if there is a non-empty function section.
|
| |
|
|
|
|
|
|
|
|
|
| |
Implement binary and text parsing and printing of shared basic heap types and
incorporate them into the type hierarchy.
To avoid the massive amount of code duplication that would be necessary if we
were to add separate enum variants for each of the shared basic heap types, use
bit 0 to indicate whether the type is shared and replace `getBasic()` with
`getBasic(Unshared)`, which clears that bit. Update all the use sites to record
whether the original type was shared and produce shared or unshared output
without code duplication.
|
| |
|
|
|
|
|
|
|
| |
Not all uses of the `reftype` parser handled the fact that it returned a
`MaybeResult`. Change its name to `maybeReftype`, add a new `reftype`
parser
that returns an error if there is no reftype, and update all the use
sites.
Fixes #6655.
|
| |
|
|
|
|
|
| |
As an abbreviation, a `typeuse` can be given as just a list of parameters and
results, in which case it corresponds to the index of the first function type
with the same parameters and results. That function type must also be an MVP
function type, i.e. it cannot have a nontrivial rec group, be non-final, or have
a declared supertype. The parser did not previously implement all of these rules.
|
| |
|
|
| |
Also update the parser so that implicit type uses are not matched with shared
function types.
|
| |
|
|
|
|
|
| |
Since the BasicHeapTypes are in an enum, calling HeapType methods on them
requires something like `HeapType(HeapType::func).someMethod()`. This is
unnecessarily verbose, so add a new `HeapTypes` namespace that contains
constexpr HeapType globals that can be used instead, shorting this to
`HeapTypes::func.someMethod()`.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Parse the text format for shared composite types as described in the
shared-everything thread proposal. Update the parser to use 'comptype' instead
of 'strtype' to match the final GC spec and add the new syntactic class
'sharecomptype'.
Update the type canonicalization logic to take sharedness into account to avoid
merging shared and unshared types. Make the same change in the TypeMerging pass.
Ensure that shared and unshared types cannot be in a subtype relationship with
each other.
Follow-up PRs will add shared abstract heap types, binary parsing and emitting
for shared types, and fuzzer support for shared types.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Because the parser has five stages, it requires instantiating all of the
templates in parsers.h with up to five different contexts. Instantiating all
those templates in a single compilation unit takes a long time. On my machine, a
release build of wat-parser.cpp.o took 32 seconds. To reduce the time of
incremental rebuilds on machines with many cores, split the code across several
compilation units so that the templates need to be instantiated for just a
single context in each unit. On my machine the longest compilation time after
this splitting is 17 seconds. The time for a full release build also drops from
42 seconds to 33 seconds. On machines with fewer cores, the benefit may be
smaller or even negative, though.
|
| |
|
|
|
| |
Remove `SExpressionParser`, `SExpressionWasmBuilder`, and `cashew::Parser`.
Simplify gen-s-parser.py. Remove the --new-wat-parser and
--deprecated-wat-parser flags.
|
| |
|
|
| |
When generating assertions, traverse the `WASTScript` data structure rather than
interleaving assertion parsing with emitting.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the new wast parser to parse a full script up front, then traverse the
parsed script data structure and execute the commands. wasm-shell had previously
used the new wat parser for top-level modules, but it now uses the new parser
for module assertions as well. Fix various bugs this uncovered.
After this change, wasm-shell supports all the assertions used in the upstream
spec tests (although not new kinds of assertions introduced in any proposals).
Uncomment various `assert_exhaustion` tests that we can now execute.
Other kinds of assertions remain commented out in our tests: wasm-shell now
supports `assert_unlinkable`, but the interpreter does not eagerly check for the
existence of imports, so those tests do not pass. Tests that check for NaNs also
remain commented out because they do not yet use the standard syntax that
wasm-shell now supports for canonical and arithmetic NaN results, and our
interpreter would not pass all of those tests even if they did use the standard
syntax.
|
| |
|
|
|
| |
The whole annotation was parsed as a keyword, which prevented file paths with non-ascii characters or paths starting with `/` or `.`.
Also, there was a typo: one was comparing `fileSize` rather than `lineSize` to `contents->npos`.
|
| |
|
|
|
|
| |
The stringref proposal has been superseded by the imported JS strings proposal,
but the former has many more operations than the latter. To reduce complexity,
remove all operations that are part of stringref but not part of imported
strings.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The stringview types from the stringref proposal have three irregularities that
break common invariants and require pervasive special casing to handle properly:
they are supertypes of `none` but not subtypes of `any`, they cannot be the
targets of casts, and they cannot be used to construct nullable references. At
the same time, the stringref proposal has been superseded by the imported
strings proposal, which does not have these irregularities. The cost of
maintaing and improving our support for stringview types is no longer worth the
benefit of supporting them.
Simplify the code base by entirely removing the stringview types and related
instructions that do not have analogues in the imported strings proposal and do
not make sense in the absense of stringviews.
Three remaining instructions, `stringview_wtf16.get_codeunit`,
`stringview_wtf16.slice`, and `stringview_wtf16.length` take stringview operands
in the stringref proposal but cannot be removed because they lower to operations
from the imported strings proposal. These instructions are changed to take
stringref operands in Binaryen IR, and to allow a graceful upgrade path for
users of these instructions, the text and binary parsers still accept but ignore
`string.as_wtf16`, which is the instruction used to convert stringrefs to
stringviews. The binary writer emits code sequences that use scratch locals and `string.as_wtf16` to keep the output valid.
Future PRs will further align binaryen with the imported strings proposal
instead of the stringref proposal, for example by making `string` a subtype of
`extern` instead of a subtype of `any` and by removing additional instructions
that do not have analogues in the imported strings proposal.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
(#6520)
;;@
with nothing else (no source:line) can be used to specify that the following
expression does not have any debug info associated to it. This can be used
to stop the automatic propagation of debug info in the text parsers.
The text printer has also been updated to output this comment when needed.
|
| |
|
|
|
|
|
|
|
|
|
| |
The spec tests use an extension of the standard text format that includes
various commands and assertions used to test WebAssembly implementations. Add a
utility to parse this extended WebAssembly script format and use it in
wasm-shell to check that it parses our spec tests without error. Fix a few
errors the new parser found in our spec tests.
A future PR will rewrite wasm-shell to interpret the results of the new parser,
but for now to keep the diff smaller, do not do anything with the new parser
except check for errors.
|
| |
|
|
|
|
|
| |
Tests is still very limited. Hopefully we can use the upstream spec
tests soon and avoid having to write our own tests for
`.set/.set/.fill/etc`.
See https://github.com/WebAssembly/memory64/issues/51
|
| |
|
|
|
| |
The lexer previously had both `getPos` and `getIndex` APIs that did different
things, but after a recent refactoring there is no difference between the index
and the position. Deduplicate the API surface.
|
| |
|
|
| |
Lex integers and floats on demand to avoid wasted work. Remove `Token`
completely now that all kinds of tokens are lexed on demand.
|
| |
|
| |
Lex them on demand instead to avoid wasted work.
|
| |
|
| |
Lex them on demand instead to avoid wasted work.
|
| |
|
| |
Lex them on demand instead to avoid wasted work.
|
| |
|
|
|
|
|
|
|
|
|
| |
The lexer currently lexes tokens eagerly and stores them in a `Token` variant
ahead of when they are actually requested by the parser. It is wasteful,
however, to classify tokens before they are requested by the parser because it
is likely that the next token will be precisely the kind the parser requests.
The work of checking and rejecting other possible classifications ahead of time
is not useful.
To make incremental progress toward removing `Token` completely, lex parentheses
on demand instead of eagerly.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new text parser is faster and more standards compliant than the old text
parser. Enable it by default in wasm-opt and update the tests to reflect the
slightly different results it produces. Besides following the spec, the new
parser differs from the old parser in that it:
- Does not synthesize `loop` and `try` labels unnecessarily
- Synthesizes different block names in some cases
- Parses exports in a different order
- Parses `nop`s instead of empty blocks for empty control flow arms
- Does not support parsing Poppy IR
- Produces different error messages
- Cannot parse `pop` except as the first instruction inside a `catch`
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updating just one or the other of these tools would cause the tests
spec/import-after-*.fail.wast to fail, since only the updated tool would
correctly fail to parse its contents. To avoid this, update both tools at
once. (The tests erroneously pass before this change because check.py does not
ensure that .fail.wast tests fail, only that failing tests end in .fail.wast.)
In wasm-shell, to minimize the diff, only use the new parser to parse modules
and instructions. Continue using the legacy parsing based on s-expressions for
the other wast commands. Updating the parsing of the other commands to use
`Lexer` instead of `SExpressionParser` is left as future work. The boundary
between the two parsing styles is somewhat hacky, but it is worth it to enable
incremental development.
Update the tests to fix incorrect wast rejected by the new parser. Many of the
spec/old_* tests use non-standard forms from before Wasm MVP was standardized,
so fixing them would have been onerous. All of these tests have non-old_*
variants, so simply delete them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR creates a pass to propagate debug location from parent node to child nodes which has no debug location with pre-order traversal. This is useful for compilers that use Binaryen API to generate WebAssembly modules.
It behaves like `wasm-opt` read text format file: children are tagged with the debug info of the parent, if they have no annotation of their own.
For compilers that use Binaryen API to generate WebAssembly modules, it is a bit redundant to add debugInfo for each expression, Especially when the compiler wrap expressions.
With this pass, compilers just need to add debugInfo for the parent node, which is more convenient.
For example:
```
(drop
(call $voidFunc)
)
```
Without this pass, if the compiler only adds debugInfo for the wrapped expression `drop`, the `call` expression has no corresponding source code mapping in DevTools debugging, which is obviously not user-friendly.
|
| |
|
|
|
|
|
| |
After the initial parsing pass to find the locations of all the module elements
and after the type definitions have been parsed, the next phase of parsing is to
visit all of the module elements and parse their types. This phase does not
require parsing function bodies, but it previously parsed entire functions
anyway for simplicity. To improve performance, skip that useless work.
|
| |
|
|
|
| |
The parsing of idchars was hot enough to show up while profiling the parsing of
a very large module. Optimize it to speed up the overall parse by about 16% in a
very unscientific measurement.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Creating an error in the parser is an extremely expensive operation for very
large files because it has to traverse the input buffer and count newlines to
compute the error message. Despite that, there are a few places were we create
errors just to discard them and continue parsing. The most notable of these
places was where we parsed the list of label index immediates for the br_table
instruction. The parser determined the end of the list by intercepting the error
produced when trying to parse one more label index.
Fix this significant performance problem causing parsing to be quadratic by
introducing and using `maybeLabelidx`, which tries to parse a label index but
does not produce an error if it fails.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
WTF-16, i.e. arbitrary sequences of 16-bit values, is the encoding of Java and
JavaScript strings, and using the same encoding makes the interpretation of
string operations trivial, even when accounting for non-ascii characters.
Specifically, use little-endian WTF-16.
Re-encode string constants from WTF-8 to WTF-16 in the parsers, then back to
WTF-8 in the writers. Update the constructor for string `Literal`s to interpret
the string as WTF-16 and store a sequence of WTF-16 code units, i.e. 16-bit
integers. Update `Builder::makeConstantExpression` accordingly to convert from
the new `Literal` string representation back to a WTF-16 string.
Update the interpreter to remove the logic for detecting non-ascii characters
and bailing out. The naive implementations of all the string operations are
correct now that our string encoding matches the JS string encoding.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR is part of a series that adds basic support for the [typed
continuations/wasmfx proposal](https://github.com/wasmfx/specfx).
This particular PR adds support for the `suspend` instruction for suspending
with a given tag, documented
[here](https://github.com/wasmfx/specfx/blob/main/proposals/continuations/Overview.md#instructions).
These instructions are of the form `(suspend $tag)`. Assuming that `$tag` is
defined with _n_ `param` types `t_1` to `t_n`, the instruction consumes _n_
arguments of types `t_1` to `t_n`. Its result type is the same as the `result`
type of the tag. Thus, the folded textual representation looks like
`(suspend $tag arg1 ... argn)`.
Support for the instruction is implemented in both the old and the new wat
parser.
Note that this PR does not implement validation of the new instruction.
This PR also fixes finalization of `cont.new`, `cont.bind` and `resume` nodes in
those cases where any of their children are unreachable.
|
| |
|
|
|
|
|
|
|
| |
Add a pass that propagates debug locations to unannotated child and sibling
expressions after parsing. The new parser on its own only attaches debug
locations to directly annotated instructions, but this pass, which we run
unconditionally, emulates the behavior of the previous parser for compatibility
with existing programs. It does unintuitive things to programs using the
non-nested format because it runs on nested Binaryen IR, so we may want to
rethink this at some point.
|
| |
|
|
|
|
|
| |
and fix a bug with sourcemap annotations on folded `if` conditions. Update
IRBuilder to apply prologue and epilogue source locations when beginning and ending
a function scope. Add basic support in the parser for explicitly tracking
annotations on module fields, although only do anything with them in the case of
prologue source location annotations.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR is part of a series that adds basic support for the [typed
continuations/wasmfx proposal](https://github.com/wasmfx/specfx).
This particular PR adds support for the `cont.bind` instruction for partially
applying continuations, documented
[here](https://github.com/wasmfx/specfx/blob/main/proposals/continuations/Overview.md#instructions).
In short, these instructions are of the form `(cont.bind $ct_before $ct_after)`
where `$ct_before` and `$ct_after` are related continuation types. They must
only differ in the number of arguments, where `$ct_before` has _n_ additional
parameters as compared to `$ct_after`, for some _n_ ≥ 0. The idea is that
`(cont.bind $ct_before $ct_after)` then takes a reference to a continuation of
type `$ct_before` as well as _n_ operands and returns a (reference to a)
continuation of type `$ct_after`. Thus, the folded textual representation looks
like `(cont.bind $ct_before $ct_after arg1 ... argn c)`.
Support for the instruction is implemented in both the old and the new wat
parser.
Note that this PR does not implement validation of the new instruction.
|
| |
|
|
| |
This new form of the abbreviated memory declaration with inline data is
introduced in the memory64 proposal.
|
| |
|
|
|
| |
We previously required a memory to exist while parsing all `StringNew` and
`StringEncode` instructions, even though some variants of the instructions use
GC arrays instead. Require a memory only for those instructions that use one.
|
| |
|
|
|
|
|
|
|
|
| |
Parse annotations using the standards-track `(@annotation ...)` format as well
as the `;;@ source-map:0:1` format. Have the lexer implicitly collect
annotations while it skips whitespace and add lexer APIs to access the
annotations since the last token was parsed. Collect annotations before parsing
each instruction and pass the annotations explicitly to the parser and parser
context functions for instructions. Add an API to `IRBuilder` to set a debug
location to be attached to the next visited or created instruction and use it
from the parser.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR is part of a series that adds basic support for the [typed
continuations/wasmfx proposal](https://github.com/wasmfx/specfx).
This particular PR adds support for the `cont.new` instruction for creating
continuations, documented [here(https://github.com/wasmfx/specfx/blob/main/proposals/continuations/Overview.md#instructions).
In short, these instructions are of the form `(cont.new $ct)` where `$ct` must
be a continuation type. The instruction takes a single (nullable) function
reference as its argument, which means that the folded representation of the
instruction is of the form `(cont.new $ct (foo ...))`.
Support for the instruction is implemented in both the old and the new wat
parser.
Note that this PR does not implement validation of the new instruction.
|
