Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 73a513c

Browse files
committed
tests: Require run-fail ui tests to have an exit code (SIGABRT not ok)
And introduce two new directives for ui tests: * `run-crash` * `run-fail-or-crash` Normally a `run-fail` ui test like tests that panic shall not be terminated by a signal like `SIGABRT`. So begin having that as a hard requirement. Some of our current tests do terminate by a signal/crash however. Introduce and use `run-crash` for those tests. Note that Windows crashes are not handled by signals but by certain high bits set on the process exit code. Example exit code for crash on Windows: `0xc000001d`. Because of this, we define "crash" on all platforms as "not exit with success and not exit with a regular failure code in the range 1..=127". Some tests behave differently on different targets: * Targets without unwind support will abort (crash) instead of exit with failure code 101 after panicking. As a special case, allow crashes for `run-fail` tests for such targets. * Different sanitizer implementations handle detected memory problems differently. Some abort (crash) the process while others exit with failure code 1. Introduce and use `run-fail-or-crash` for such tests.
1 parent 98dafe7 commit 73a513c

File tree

2 files changed

+19
-7
lines changed

2 files changed

+19
-7
lines changed

‎src/tests/directives.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,10 @@ expectations](ui.md#controlling-passfail-expectations).
7575
| `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A |
7676
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A |
7777
| `build-fail` | Building should fail | `ui`, `crashes` | N/A |
78-
| `run-pass` | Running the test binary should pass | `ui`, `crashes`, `incremental` | N/A |
79-
| `run-fail` | Running the test binary should fail | `ui`, `crashes` | N/A |
78+
| `run-pass` | Program must exit with code `0` | `ui`, `crashes`, `incremental` | N/A |
79+
| `run-fail` | Program must exit with code `1..=127` | `ui`, `crashes` | N/A |
80+
| `run-crash` | Program must crash | `ui` | N/A |
81+
| `run-fail-or-crash` | Program must `run-fail` or `run-crash` | `ui` | N/A |
8082
| `ignore-pass` | Ignore `--pass` flag | `ui`, `crashes`, `codegen`, `incremental` | N/A |
8183
| `dont-check-failure-status` | Don't check exact failure status (i.e. `1`) | `ui`, `incremental` | N/A |
8284
| `failure-status` | Check | `ui`, `crashes` | Any `u16` |

‎src/tests/ui.md‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -448,7 +448,7 @@ even run the resulting program. Just add one of the following
448448
- `//@ build-pass` — compilation and linking should succeed but do
449449
not run the resulting binary.
450450
- `//@ run-pass` — compilation should succeed and running the resulting
451-
binary should also succeed.
451+
binary should make it exit with code 0 which indicates success.
452452
- Fail directives:
453453
- `//@ check-fail` — compilation should fail (the codegen phase is skipped).
454454
This is the default for UI tests.
@@ -457,10 +457,20 @@ even run the resulting program. Just add one of the following
457457
- First time is to ensure that the compile succeeds without the codegen phase
458458
- Second time is to ensure that the full compile fails
459459
- `//@ run-fail` — compilation should succeed, but running the resulting
460-
binary should fail.
461-
462-
For `run-pass` and `run-fail` tests, by default the output of the program itself
463-
is not checked.
460+
binary should make it exit with a code in the range `1..=127` which
461+
indicates regular failure. On targets without unwind support, crashes
462+
are also accepted.
463+
- `//@ run-crash` — compilation should succeed, but running the resulting
464+
binary should fail with a crash. Crashing is defined as "not exiting with
465+
a code in the range `0..=127`". Example on Linux: Termination by `SIGABRT`
466+
or `SIGSEGV`. Example on Windows: Exiting with the code for
467+
`STATUS_ILLEGAL_INSTRUCTION` (`0xC000001D`).
468+
- `//@ run-fail-or-crash` — compilation should succeed, but running the
469+
resulting binary should either `run-fail` or `run-crash`. Useful if a test
470+
crashes on some targets but just fails on others.
471+
472+
For `run-pass`. `run-fail`, `run-crash` and `run-fail-or-crash` tests, by
473+
default the output of the program itself is not checked.
464474

465475
If you want to check the output of running the program, include the
466476
`check-run-results` directive. This will check for a `.run.stderr` and

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /