Introduce lit/FileCheck tests (#3367)

lit and FileCheck are the tools used to run the majority of tests in LLVM. Each
lit test file contains the commands to be run for that test, so lit tests are
much more flexible and can be more precise than our current ad hoc testing
system. FileCheck reads expected test output from comments, so it allows test
output to be written alongside and interspersed with test input, making tests
more readable and precise than in our current system.

This PR adds a new suite to check.py that runs lit tests in the test/lit
directory. A few tests have been ported to demonstrate the features of the new
test runner.

This change is motivated by a need for greater flexibility in testing wasm-split.
See #3359.

1 files changed, 30 insertions, 0 deletions

diff --git a/scripts/not.py b/scripts/not.py
new file mode 100755
index 000000000..98898f451
--- /dev/null
+++ b/scripts/not.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python3
+
+# Copyright 2020 WebAssembly Community Group participants
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import sys
+import subprocess
+
+
+# Emulate the `not` tool from LLVM's test infrastructure for use with lit and
+# FileCheck. It succeeds if the given subcommand fails and vice versa.
+def main():
+    cmd = sys.argv[1:]
+    result = subprocess.run(cmd)
+    sys.exit(0 if result.returncode != 0 else 1)
+
+
+if __name__ == '__main__':
+    main()