ostjc

A zero dependency Rust program for comparing 2 os-test result json files.

os-test is a program that tests operating systems to determine how they behave, especially with reference to the POSIX standard. You can find the os-test website here: https://sortix.org/os-test/

os-test allows to export the results of the tests as a json file.

This program is intended to make it a little easier to compare 2 of the json result files from different runs to determine the differences between them.

Installation

This program has not yet been uploaded to crates.io so the only way to run it is to compile it yourself.

For installing Rust see this link: https://rust-lang.org/tools/install/

When you have Rust installed you can do the following:

git clone https://codeberg.org/auronandace/ostjc.git
cd ostjc
cargo install --path .

To uninstall:

cargo uninstall ostjc

Usage

You'll need 2 json files generated from os-test.

To show the differences between them simply invoke ostjc with the 2 files as arguments:

ostjc path/to/before.json path/to/after.json

I find it helpful to send the output to a file (especially when the output is long):

ostjc before.json after.json > comparison.txt

Output

Here is an example of the output you can expect:

Before:
redox
good: 4170
bad: 1406
unknown: 74
total tests: 5650
After:
redox
good: 4175
bad: 1401
unknown: 74
total tests: 5650
Total amount of changed tests: 5
Tests that used to fail but now pass: 5
In basic/unistd test: nice
Before:
Outcome: compile_error
Output for this test was null.
After:
Outcome: good
Output: "exit: 0\n"
In include/unistd test: nice
Before:
Outcome: undeclared
Output for this test was null.
After:
Outcome: good
Output for this test was null.
In namespace test: sched
Before:
Outcome: pollution
Output for this test was null.
After:
Outcome: good
Output for this test was null.
In namespace test: sched-xsi
Before:
Outcome: pollution
Output for this test was null.
After:
Outcome: good
Output for this test was null.
In udp test: trio-send-wrong-y-connect-send-right-x-recv
Before:
Outcome: bad
Output: "../misc/run.sh: line 3: ./trio-send-wrong-y-connect-send-right-x-recv: Operation not permitted\nexit: 126\n"
After:
Outcome: good
Output: "x\n"