-
Notifications
You must be signed in to change notification settings - Fork 279
Description
Description of the enhancement request
Currently C# support is offered by default via mono. However, mono development is has slowed down significantly, and it does not implement the newer language features. This leads to a mismatch between what contestants expect when they see C# and what they actually get, usually in the form of annoying compilation errors.
The linux support of dotnet works well enough for these small cli programs that mostly do number crunching. And in fact, C# can be supported with dotnet.
The goal you want to achieve
Have C# with dotnet in the default offering of domjudge.
Any other information that you want to share?
The remaining sections summarize a working approach with the modifications needed.
1. getting dotnet into the chroot
The dotnet with all its libraries is needed to compile the program and to execute it afterwards. So it needs to be added to the chroot environment. Microsoft offers debian package repositories for debian. As described at https://learn.microsoft.com/en-gb/dotnet/core/install/linux-debian the steps are as follows:
$ wget https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
$ sudo dpkg -i packages-microsoft-prod.deb
$ sudo apt update
$ sudo apt install dotnet-sdk-9.0 # or 8.0, etc.
For customization https://github.com/DOMjudge/domjudge-packaging/blob/b7b59c5b1227eb4924ceec705d0e9319d9d0418a/docker/README.md#judgehost suggests modifying chroot-and-tar.sh, specifically adding arguments arguments to dj_make_chroot.
And indeed, it has the -s option to add extra package repositories:
domjudge/misc-tools/dj_make_chroot.in
Lines 75 to 78 in d2e8b6e
However, there are two issues, there does not seem to be a way to install a package from an external url. So I ended up downloading packages-microsoft-prod.deb and extracting microsoft-prod.list and microsoft-prod.gpg from it. Which then can be used with the script.
Still, this does not appear to be readily usable because when chroot-and-tar.sh runs, inside a docker container, nothing seems to be mounted for the -s option to be usable:
So I ended up modifying build-judgehost.sh and chroot-and-tar.sh as follows:
diff --git a/docker/build-judgehost.sh b/docker/build-judgehost.sh index e556c0b..f407cd4 100755 --- a/docker/build-judgehost.sh +++ b/docker/build-judgehost.sh @@ -12,7 +12,7 @@ docker build -t "${docker_tag}-build" -f judgehost/Dockerfile.build . # Build chroot builder_name=$(echo "${docker_tag}" | sed 's/[^a-zA-Z0-9_-]/-/g') docker rm -f "${builder_name}" > /dev/null 2>&1 || true -docker run --name "${builder_name}" --privileged "${docker_tag}-build" +docker run --name "${builder_name}" --privileged -v "`pwd`:/build-context:ro" "${docker_tag}-build" docker cp "${builder_name}:/chroot.tar.gz" . docker cp "${builder_name}:/judgehost.tar.gz" . docker rm -f "${builder_name}" diff --git a/docker/judgehost/chroot-and-tar.sh b/docker/judgehost/chroot-and-tar.sh index 56da9a9..4b1ee65 100755 --- a/docker/judgehost/chroot-and-tar.sh +++ b/docker/judgehost/chroot-and-tar.sh @@ -3,7 +3,9 @@ set -euo pipefail # Usage: https://github.com/DOMjudge/domjudge/blob/main/misc-tools/dj_make_chroot.in#L58-L87 -/opt/domjudge/judgehost/bin/dj_make_chroot +/opt/domjudge/judgehost/bin/dj_make_chroot \ + -s /build-context/tmp/microsoft-prod.list \ + -i dotnet-sdk-9.0 cd / echo "[..] Compressing chroot"
This mounts the current working directory as /build-context in the container. Then the two extracted files are placed somewhere inside it, and can then be used from the container.
Important
The extracted microsoft-prod.gpg needs to be renamed to microsoft-prod.list.gpg because that is what the script expects.
After these changes the "judgehost" container should build without issues, now with dotnet available.
2. the new csharp domjudge executable
Now that dotnet is in the chroot, the correct incantations must be found that will work in the sandboxed environment.
Note
In this example the already existing csharp language and executable are edited.
First, domjudge apparently needs a "Compiler version command" and a "Runner version command". These both can be set to dotnet --version.
As for the executable, similarly for the implementation with mono, only a "run" step is required:
#!/bin/sh # C# compile wrapper-script for 'compile.sh'. # See that script for syntax and more info. # # # This script requires dotnet-sdk to be installed and a working /tmp in the chroot. # https://learn.microsoft.com/en-gb/dotnet/core/install/linux-debian set -e DEST="1ドル" ; shift MEMLIMIT="1ドル" ; shift MAINSOURCE="1ドル" SOURCEDIR="${MAINSOURCE%/*}" [ "$SOURCEDIR" = "$MAINSOURCE" ] && SOURCEDIR='.' WORKDIR=$(mktemp -d "./tmp.XXXXXXXXXX") trap 'rm -rf "$WORKDIR"' EXIT PROJDIR="$WORKDIR/project" mkdir -p "$PROJDIR" cat > "$PROJDIR/$DEST.csproj" <<EOF <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <OutputType>Exe</OutputType> <TargetFramework>net9.0</TargetFramework> <ImplicitUsings>enable</ImplicitUsings> <Nullable>enable</Nullable> </PropertyGroup> </Project> EOF cp -r "$@" "$PROJDIR" mkdir -p "$WORKDIR/tmp" mkdir -p "$WORKDIR/home" env -i \ TMPDIR="$WORKDIR/tmp" \ HOME="$WORKDIR/home" \ DOTNET_CLI_TELEMETRY_OPTOUT=1 \ DOTNET_EnableWriteXorExecute=0 \ DOTNET_NOLOGO=1 \ DOTNET_SKIP_WORKLOAD_INTEGRITY_CHECK=1 \ dotnet publish \ -o . \ --configuration=Release \ /p:DefineConstants=ONLINE_JUDGE \ "$PROJDIR" mv "$DEST" "$DEST.exe" cat > "$DEST" <<EOF #!/bin/sh # Detect dirname and change dir to prevent class not found errors. if [ "\${0%/*}" != "\$0" ]; then cd "\${0%/*}" fi export DOTNET_EnableWriteXorExecute=0 exec "./$DEST.exe" EOF chmod a+x "$DEST" exit 0
Unfortunately the dotnet executable does a lot of things, especially when building programs, related to nuget and whatnot. The idea of the myriad of environment variables is to make it use temporary directories to store the files that it will create for various reasons. See https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-environment-variables for a complete list of available environment variables and their explanations.
The above script can most certainly be improved, some things probably removed. There is also the option of creating the project using dotnet new console instead of manually creating the project file (the project xml seen above was extracted from such a command run).
DOTNET_EnableWriteXorExecute
DOTNET_EnableWriteXorExecute remains undocumented, but it is probably the most crucial environment variable that needs to be set. By default the runtime tries to allocate a memfd that will then be resized to 2TiB, which exceeds the maximum file size in the sandbox. This causes the delivery of SIGXFSZ, killing the process. It can be tested easily:
$ dotnet --version
9.0.110
$ ulimit -f 16384
$ strace -y dotnet --version
[...]
memfd_create("doublemapper", MFD_CLOEXEC) = 8</memfd:doublemapper>(deleted)
ftruncate(8</memfd:doublemapper>(deleted), 2199023255552) = -1 EFBIG (File too large)
--- SIGXFSZ {si_signo=SIGXFSZ, si_code=SI_USER, si_pid=1189651, si_uid=1000} ---
+++ killed by SIGXFSZ (core dumped) +++
This has been reported: dotnet/runtime#117819, and apparently fixed for dotnet 10: dotnet/runtime#119316 (although I am not entirely sure the fix will work well in the sandbox).
3. adding /tmp to the chroot environment
With the above script, things are almost all in place, but if one tries to evaluate a C# submission, they will be met with a similar error:
System.IO.IOException: The system cannot open the device or file specified. : 'NuGet-Migrations'
The reason is, as far as I was able to determine, that the runtime absolutely needs a working /tmp for shared mutexes because it wants to create them in /tmp. This path is hard-coded, not easily changeable:
- https://github.com/dotnet/runtime/blob/v9.0.9/src/coreclr/pal/src/init/pal.cpp#L1360
- https://github.com/dotnet/runtime/blob/v9.0.9/src/coreclr/pal/src/include/pal/palinternal.h#L233
The NuGet-Migrations global mutex is created to synchronize multiple nuget processes: https://github.com/NuGet/NuGet.Client/blob/e8da0a86d46b42133ee80c28fe091d80e72a527d/src/NuGet.Core/NuGet.Common/Migrations/MigrationRunner.cs#L40.
So I decided to mount a tmpfs as /tmp in the chroot environment. This requires the following modifications to domjudge:
diff --git a/etc/sudoers-domjudge.in b/etc/sudoers-domjudge.in index 8f96c4472..84f831b77 100644 --- a/etc/sudoers-domjudge.in +++ b/etc/sudoers-domjudge.in @@ -19,3 +19,5 @@ @DOMJUDGE_USER@ ALL=(root) NOPASSWD: /bin/cp -pR /dev/random /dev/urandom /dev/null dev @DOMJUDGE_USER@ ALL=(root) NOPASSWD: /bin/chmod o-w dev/random dev/urandom +@DOMJUDGE_USER@ ALL=(root) NOPASSWD: /bin/mount -t tmpfs -o size=256M tmpfs tmp +@DOMJUDGE_USER@ ALL=(root) NOPASSWD: /bin/umount /*/tmp diff --git a/judge/chroot-startstop.sh.in b/judge/chroot-startstop.sh.in index a4fb2caec..081bea0d8 100755 --- a/judge/chroot-startstop.sh.in +++ b/judge/chroot-startstop.sh.in @@ -106,12 +106,19 @@ case "1ドル" in mkdir -p dev sudo -n cp -pR /dev/random /dev/urandom /dev/null dev < /dev/null sudo -n chmod o-w dev/random dev/urandom < /dev/null + + # add /tmp + mkdir -p tmp + sudo -n mount -t tmpfs -o size=256M tmpfs tmp ;; stop) dj_umount "$PWD/proc" + dj_umount "$PWD/tmp" + rm -rf tmp + rm -f dev/urandom dev/random dev/null rmdir dev || true
Important
The "judgehost" container needs to be rebuilt after the above modifications.
The size of 256MiB of the tmpfs was chosen randomly, probably a smaller size like 16MiB would also be sufficient.
Warning
It seems to me that the chroot environment is not reset between each test case evaluation, in that case /tmp can be used to communicate between test case evaluations. In our case this was deemed to be a kind of acceptable risk.
4. profit
And that's it. You should now have C# evaluation working with dotnet.