A users solution may trigger errors and warnings. This would be contained in the output and would obviously fail the StdOutCheck. However, could we provide more meaningful feedback?

PhpErrorCheck

We could grep the output for PHP Notice: and PHP Warning. This seems a bit flaky as that could be valid output.
We could lint the file first, check if it's valid PHP code, then parse the file and add in some code, like an error-handler which would exit with a non-zero error code.