| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
When IRBuilder builds an empty non-block scope such as a function body,
an if arm, a try block, etc, it needs to produce some expression to
represent the empty contents. Previously it produced a nop, but change
it to produce an empty block instead. The binary writer and printer have
special logic to elide empty blocks, so this produces smaller output.
Update J2CLOpts to recognize functions containing empty blocks as
trivial to avoid regressing one of its tests.
|
| |
|
|
|
|
|
|
| |
The module splitting utility has a configuration option for minimizing
new export names, but it was previously applied only to newly exported
functions. Using the new multi-split mode can produce lots of exported
tables and splitting WasmGC programs can produce lots of exported
globals, so minimizing these export names can have a big impact on code
size.
|
| |
|
|
|
|
|
|
| |
The configuration for the module splitting utility previous took a set
of functions to keep in the primary module. Change it to take a list of
functions to split into the secondary module instead. This improves the
code quality in multi-split mode because it keeps stub functions
generated by previous splits from being moved into secondary modules
during later splits.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Maintain the invariant that every defined functions belongs to either
the set of kept functions or the set of split functions. Functions are
kept by default except when --keep-funcs is specified without
--split-funcs on the command line. This is mostly NFC except that it
changes the default behavior when no arguments are specified on the
command line to keep all functions.
This will simplify a follow-on PR that switches from passing the kept
functions to the module splitting utility to passing the split
functions.
|
| |
|
|
|
|
|
|
|
| |
Rather than analyze what module elements from the primary module a
secondary module will need, the splitting logic conservatively imports
all module elements from the primary module into the secondary module.
Run RemoveUnusedElements on the secondary module to remove any of these
imports that happen to be unnecessary. Leave a TODO mentioning the
possibility of being more selective about which module elements get
exported to reduce code size in the primary module, too.
|
| |
|
|
|
|
|
| |
Add a mode that splits a module into arbitrarily many parts based on a
simple manifest file. This is currently implemented by splitting out one
module at a time in a loop, but this could change in the future if
splitting out all the modules at once would improve the quality of the
output.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Wasm-split generally assumes that calls to secondary functions made
before the secondary module has been loaded and instantiated should go
to imported placeholder functions that can be responsible for loading
the secondary module and forwarding the call to the loaded function.
That scheme makes the loading entirely transparent from the
application's point of view, which is not always a good thing. Other
schemes would make it impossible for a secondary function to be called
before the secondary module has been explicitly loaded, in which case
the placeholder functions would never be called. To improve code size
and simplify instantiation under these schemes, add a new
`--no-placeholders` option that skips adding imported placeholder
functions.
|
| |
|
|
|
|
|
| |
Rather than trying to trampoline primary-to-secondary calls through an
existing table, just create a fresh table for this purpose. This ensures
that modifications to the existing tables cannot interfere with
primary-to-secondary calls and conversely that loading the secondary
module cannot overwrite modifications to the tables.
|
| |
|
|
|
|
|
|
|
| |
There are times after collecting a profile, we wish to manually include
specific functions into the primary module.
It could be due to non-deterministic profiling or functions for error
scenarios (e.g. _trap).
This PR helps to unlock this workflow by honoring both the
`--keep-funcs` flag as well as the `--profile` flag
|
| |
|
|
| |
The `filecheck` command used for tests does not support `CHECK-SAME`, so
use should be avoided.
|
| |
|
|
|
|
|
|
| |
The module splitting code incorrectly assumed that there would be at least one
active element segment and failed to initialize the table slot manager with a
function table if that was not the case. Fix the bug by setting the table even
when there are no active segments and add a test.
Fixes #6572 and #6637.
|
| |
|
|
|
| |
When we have a ref.func that refers to the secondary module then make a trampoline
that calls it directly. The trampoline's call is then fixed up like all direct calls to the
secondary module.
|
| | |
|
| |
|
|
|
|
| |
With the old version of JSPI, the JSPI pass was required to be run before
splitting and would automatically add an export to be able to find the
load_secondary_module function. Now that the pass is no longer needed,
just add an import manually for the load_secondary_module function.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new text parser is faster and more standards compliant than the old text
parser. Enable it by default in wasm-opt and update the tests to reflect the
slightly different results it produces. Besides following the spec, the new
parser differs from the old parser in that it:
- Does not synthesize `loop` and `try` labels unnecessarily
- Synthesizes different block names in some cases
- Parses exports in a different order
- Parses `nop`s instead of empty blocks for empty control flow arms
- Does not support parsing Poppy IR
- Produces different error messages
- Cannot parse `pop` except as the first instruction inside a `catch`
|
| |
|
|
|
|
|
| |
Since data and elem segments cannot be imported or exported, there is no way to
access them from the secondary module, so functions that need to refer to them
cannot be split out.
Fixes #6512.
|
| |
|
|
|
|
|
|
| |
The new parser enforces the rule that imports must come before declarations
(except for type declarations). The old parser does not enforce this rule, so
many of our tests did not follow it. Fix them to follow that rule and fix other
invalid syntax. Also add missing finalization of Load expressions in
wasm-builder.h that was causing a test to fail under the new parser and guard
against an error case in wasm-ir-builder.cpp that used to cause a segfault.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
We previously supported (and primarily used) a non-standard text format for
conditionals in which the condition, if-true expression, and if-false expression
were all simply s-expression children of the `if` expression. The standard text
format, however, requires the use of `then` and `else` forms to introduce the
if-true and if-false arms of the conditional. Update the legacy text parser to
require the standard format and update all tests to match. Update the printer to
print the standard format as well.
The .wast and .wat test inputs were mechanically updated with this script:
https://gist.github.com/tlively/85ae7f01f92f772241ec994c840ccbb1
|
| |
|
|
|
| |
Update the legacy text parser and all tests to use the standard text format for shared memories, e.g. `(memory $m 1 1 shared)` rather than `(memory $m (shared 1 1))`. Also remove support for non-standard in-line "data" or "segment" declarations.
This change makes the tests more compatible with the new text parser, which only supports the standard format.
|
| |
|
|
|
|
|
|
|
|
| |
Previously the lit test update script interpreted module names as the names of
import items and export names as the names of export items, but it is more
precise to use the actual identifiers of the imported or exported items as the
names instead.
Update update_lit_checks.py to use a more correct regex to match names and to
correctly use the identifiers of import and export items as their names. In some
cases this can improve the readability of test output.
|
| |
|
|
|
| |
Apparently the version of node on the Alpine runner was updated and no longer
recognizes the --experimental-wasm-threads option. Delete this option out of the
test that was using it.
|
| |
|
|
|
| |
This test file cannot be auto-generated because its output contains a hash of
its input, but the comment enabling that auto updater was accidentally left in
during development. Remove the comment.
|
| |
|
|
| |
Correctly use the output memory's index type when generating the __write_profile
function. Requires moving some code around, but is a very small fix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When printing Binaryen IR, we previously generated names for unnamed heap types
based on their structure. This was useful for seeing the structure of simple
types at a glance without having to separately go look up their definitions, but
it also had two problems:
1. The same name could be generated for multiple types. The generated names did
not take into account rec group structure or finality, so types that differed
only in these properties would have the same name. Also, generated type names
were limited in length, so very large types that shared only some structure
could also end up with the same names. Using the same name for multiple types
produces incorrect and unparsable output.
2. The generated names were not useful beyond the most trivial examples. Even
with length limits, names for nontrivial types were extremely long and visually
noisy, which made reading disassembled real-world code more challenging.
Fix these problems by emitting simple indexed names for unnamed heap types
instead. This regresses readability for very simple examples, but the trade off
is worth it.
This change also reduces the number of type printing systems we have by one.
Previously we had the system in Print.cpp, but we had another, more general and
extensible system in wasm-type-printing.h and wasm-type.cpp as well. Remove the
old type printing system from Print.cpp and replace it with a much smaller use
of the new system. This requires significant refactoring of Print.cpp so that
PrintExpressionContents object now holds a reference to a parent
PrintSExpression object that holds the type name state.
This diff is very large because almost every test output changed slightly. To
minimize the diff and ease review, change the type printer in wasm-type.cpp to
behave the same as the old type printer in Print.cpp except for the differences
in name generation. These changes will be reverted in much smaller PRs in the
future to generally improve how types are printed.
|
| |
|
| |
Renaming the multimemory flag in Binaryen to match its naming in LLVM.
|
| |
|
|
|
|
|
|
|
|
| |
All top-level Module elements are identified and referred to by Name, but for
historical reasons element and data segments were referred to by index instead.
Fix this inconsistency by using Names to refer to segments from expressions that
use them. Also parse and print segment names like we do for other elements.
The C API is partially converted to use names instead of indices, but there are
still many functions that refer to data segments by index. Finishing the
conversion can be done in the future once it becomes necessary.
|
| |
|
|
|
|
|
|
|
|
| |
When using JSPI with wasm-split, any calls to secondary module functions
will now first check a global to see if the module is loaded. If not
loaded it will call a JSPI'ed function that will handle loading module.
The setup is split into the JSPI pass and wasm-split tool since the JSPI
pass is first run by emscripten and we need to JSPI'ify the load secondary
module function. wasm-split then injects all the checks and calls to the
load function.
|
| |
|
| |
This PR maintains the first memory's import/export in the single combined memory after multi-memories are lowered.
|
| |
|
|
|
|
|
|
|
|
| |
This makes Binaryen's default type system match the WasmGC spec.
Update the way type definitions without supertypes are printed to reduce the
output diff for MVP tests that do not involve WasmGC. Also port some
type-builder.cpp tests from test/example to test/gtest since they needed to be
rewritten to work with isorecursive type anyway.
A follow-on PR will remove equirecursive types completely.
|
| |
|
| |
Adds an --in-secondary-memory switch to the wasm-split tool that allows profile data to be stored in a separate memory from module main memory. With this option, users do not need to reserve the initial memory region for profile data and the data can be shared between multiple threads.
|
| |
|
|
|
| |
A memory must be explicitly defined before being exported.
Fix CI for 6b3f3af.
|
| |
|
| |
Follow up to #4771 to test new --print-profile options for wasm-split.
|
| |
|
|
|
|
|
| |
This PR removes the single memory restriction in IR, adding support for a single module to reference multiple memories. To support this change, a new memory name field was added to 13 memory instructions in order to identify the memory for the instruction.
It is a goal of this PR to maintain backwards compatibility with existing text and binary wasm modules, so memory indexes remain optional for memory instructions. Similarly, the JS API makes assumptions about which memory is intended when only one memory is present in the module. Another goal of this PR is that existing tests behavior be unaffected. That said, tests must now explicitly define a memory before invoking memory instructions or exporting a memory, and memory names are now printed for each memory instruction in the text format.
There remain quite a few places where a hardcoded reference to the first memory persist (memory flattening, for example, will return early if more than one memory is present in the module). Many of these call-sites, particularly within passes, will require us to rethink how the optimization works in a multi-memories world. Other call-sites may necessitate more invasive code restructuring to fully convert away from relying on a globally available, single memory pointer.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This update includes a
[fix](https://github.com/mull-project/FileCheck.py/pull/188) to how the
filecheck Python package matches whitespace to more closely match the behavior
of upstream filecheck in LLVM. We have one test affected by this change, so all
users who run the test suite will have to update their installed filecheck. This
can be done via
```
pip3 install -r requirements-dev.txt
```
|
| |
|
|
|
|
|
| |
Add an option for running the asyncify transformation on the primary module
emitted by wasm-split. The idea is that the placeholder functions should be able
to unwind the stack while the secondary module is asynchronously loaded, then
once the placeholder functions have been patched out by the secondary module the
stack should be rewound and end up in the correct secondary function.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Previously the set of functions to keep was initially empty, then the profile
added new functions to keep, then the --keep-funcs functions were added, then
the --split-funcs functions were removed. This method of composing these
different options was arbitrary and not necessarily intuitive, and it prevented
reasonable workflows from working. For example, providing only a --split-funcs
list would result in all functions being split out not matter which functions
were listed.
To make the behavior of these options, and --split-funcs in particular, more
intuitive, disallow mixing them and when --split-funcs is used, split out only
the listed functions.
|
| |
|
|
|
|
| |
We can assume that imported memories (and the profiling data they contain) are
already accessible from the module's environment, so there's no need to export
them. This also avoids needing to add knowledge of "profile-memory" to
Emscripten's library_dylink.js.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To avoid requiring a static memory allocation, wasm-split's instrumentation
defaults to recording profile data in Wasm globals. This causes problems for
multithreaded applications because the globals are thread-local, but it is not
always feasible to arrange for a separate profile to be dumped on each thread.
To simplify the profiling of such multithreaded applications, add a new
instrumentation mode that stores the profiling data in shared memory instead of
in globals. This allows a single profile to be written that correctly reflects
the called functions on all threads.
This new mode is not on by default because it requires users to ensure that the
program will not trample the in-memory profiling data. The data is stored
beginning at address zero and occupies one byte per declared function in the
instrumented module. Emscripten can be told to leave this memory free using the
GLOBAL_BASE option.
|
| | |
|
| |
|
|
|
|
| |
Add list tests for the help messages of all tools, factoring out common options
into shared tests. This is slightly brittle because the text wrapping depends on
the length of the longest option, but that brittleness should be worth the
benefit of being able to see the actual help text in the tests.
|
| |
|
|
|
| |
Improve the legibility of the option documentation by adding vertical space
between options. This is particularly helpful to delimit the text of options
with longer explanations.
|
| |
|
|
|
|
| |
The new instruction emits a file containing a map between placeholder index and
the name of the split out function that placeholder is replacing in the table.
This map is intended to be useful for debugging, as discussed in
https://github.com/emscripten-core/emscripten/issues/14330.
|
| |
|
|
|
|
|
| |
Given a list of profiles for the same module, --merge-profiles produces a single
combined profile the contains the minimum timestamp among the original profiles
for each function. When verbose output is enabled, also emit a message for each
profile that could individually be removed without affecting the set of
functions in the combined profile, as suggested in #3912.
|
| |
|
|
|
|
| |
In anticipation of adding a third wasm-split mode, merge-profiles, in addition
to the existing split and instrument modes, refactor wasm-split's option
validation to let the valid modes be declared for each option. This approach is
more scalable and robust than the ad-hoc validation we had previously.
|
| |
|
|
|
|
|
|
|
| |
wasm-split would previously use internal function names to create the external
names of the functions that are newly exported from the primary module to be
imported into the secondary module. When the input module contains full function
names (as is commonly the case when emitting symbol maps), this caused the
function names to be preserved as the export names, even when names are
otherwise being stripped. To save on code size and properly anonymize functions,
generate minimal export names when debuginfo is disabled instead.
|
| |
|
|
|
|
| |
Even when other names are stripped, it can be useful for wasm-split to preserve
the module name so that the split modules can be differentiated in stack traces.
Adding this option to wasm-split requires adding similar options to ModuleWriter
and WasmBinaryWriter.
|
| |
|
|
|
|
|
|
| |
Imported functions come first when modules are emitted, so to ensure the
function indices are correct, they need to come first in the symbol maps. We
never noticed this bug before because imported functions are always the first
functions when a module is parsed, so the bug never mattered in practice.
However, wasm-split adds new imported functions after parsing and these were
causing the symbol map indices to be incorrect.
|
| |
|
|
|
| |
The new option emits a symbol map file for each of the split modules. The file
names are created by appending ".symbols" to each of the Wasm output file
names.
|
| |
|
|
|
|
|
|
|
|
|
| |
We used to print active element segments right after corresponding
tables, and passive segments came after those. We didn't print internal
segment names, and empty segments weren't being printed at all. This
meant that there was no way for instructions to refer to those table
segments after round tripping.
This will fix those issues by printing segments in the order they were
defined, including segment names when necessary and not omitting
empty segments anymore.
|
