Manual:PHP unit testing/Running the tests
Contents
- Running the tests
- Generate code coverage
- Writing testable PHP code
- Writing tests
- Continuous integration
- Understanding build failures
- Appendix
(how to help, resources..)
Tools
Entrypoints for running the tests
[edit ]We use composer to run PHPUnit tests. If you have not already, run composer update in the root of the MediaWiki core repository.
All tests
[edit ]The entry point for running all tests at once is the composer phpunit:entrypoint command, that must be run in the MediaWiki directory:
$ composer phpunit:entrypoint -- path/to/my/test
Error "Cannot access the database: Unknown database"
[edit ]
If you get this error you may need to set the environment variable PHPUNIT_WIKI, e.g:
$PHPUNIT_WIKI=wikicomposerphpunit:entrypoint--path/to/my/test
The "--filter" option does not exist
[edit ]This and similar errors can occur if you omit the -- when invoking the Composer script. Make sure to always use the double dash when running the Composer script, to separate Composer and PHPUnit arguments.
Integration tests
[edit ]
These are tests that are contained in tests/phpunit/integration folders.
$ composer phpunit:integration -- path/to/my/test
Unit tests
[edit ]These are tests that are contained in tests/phpunit/unit folders.
Run only the pure "unit" tests for MediaWiki core and all extensions and skins:
$ composer phpunit:unit > phpunit --colors=always --testsuite=core:unit,extensions:unit,skins:unit PHPUnit 8.5.8 by Sebastian Bergmann and contributors. ............................................................. 61 / 8977 ( 0%) ............................................................. 122 / 8977 ( 1%) ............................................................. 183 / 8977 ( 2%) ............................................................. 244 / 8977 ( 2%) ............................................................. 305 / 8977 ( 3%)
To run specific pure "unit" tests only, use the PHPUnit entrypoint located at vendor/bin/phpunit
$ vendor/bin/phpunit --help PHPUnit 8.5.8 by Sebastian Bergmann and contributors. Usage: phpunit [options] UnitTest [UnitTest.php] phpunit [options] <directory>
Linter and code style
[edit ]To check PHP code style in a file you can run:
composerlint./some/path/SomeFile.php composerphpcs./some/path/SomeFile.php
To run auto-fixes execute PHPCBF (PHP Code Beautifier and Fixer) like so:
vendor/bin/phpcbf./some/path/SomeFile.php
All of these commands take a file or directory as a parameter. If you modified a few files in different folders, you can temporarily add your commands to the scripts section in composer.json (just to run the commands quicker).
Select by directory or file
[edit ]Running all tests from all core components and installed extensions may take a while. During development you may want to focus on just a particular test suite or directory of test suites. Pass the directory or file to the phpunit command.
For example, if you're working on a patch that modifies includes/ResourceLoader, with a test covering it under unit/includes/ResourceLoader/, then you might run the tests like so:
$ composer phpunit:entrypoint -- tests/phpunit/unit/includes/ResourceLoader/
PHPUnit by Sebastian Bergmann.
........................................................... 59 / 59 (100%)
Time: 1.78 seconds, Memory: 22.00 MB
OK (59 tests, 80 assertions)
To run tests for a specific extension or skin:
$ composer phpunit:entrypoint -- extensions/EventLogging/tests/phpunit/ ........................................................... 59 / 59 (100%)
Select by test name
[edit ]If you're debugging a specific test case and are adding breakpoints or var_dump statements, it may be useful to filter even more closely than by the file, so as to skip past all other invocations of the same underlying code. To do this, use the --filter option to match substrings of the test name. For example, given a test suite like the following:
namespace MediaWiki\Tests\ResourceLoader; class FileModuleTest extends MediaWikiIntegrationTestCase { protected function setUp(): void { parent::setUp(); // ... } public function testLessCompiler() { // ... } public function testGetScriptPackageFiles() { // ... } public function testGetVersionHash() { // ... } }
You can combine the directory selection with --filter to run only the second testGetScriptPackageFiles test case, like so:
$ composer phpunit:entrypoint -- --filter Package tests/phpunit/includes/ResourceLoader/FileModuleTest.php
PHPUnit by Sebastian Bergmann.
........................................................... 21 / 21 (100%)
Time: 0.17 seconds, Memory: 22.00 MB
OK (21 tests, 21 assertions)
Select by group
[edit ]Each test case can be grouped using the @group annotation in the test case file. The group names may look obscure at first, we will describe them later on in the Writing unit tests chapter.
To only run tests in a particular group, use the --group option.
$ composer phpunit:entrypoint -- --group Cache PHPUnit by Sebastian Bergmann. ................................................IIIII........ 61 / 2311 ( 2%) ............I................................................ 122 / 2311 ( 5%) ............................................................. 183 / 2311 ( 7%) ............................................................. 244 / 2311 ( 10%) ............................................................. 305 / 2311 ( 13%) ............................................................. 366 / 2311 ( 15%) ............................................................. 427 / 2311 ( 18%) ............................................................. 488 / 2311 ( 21%) ............................................................. 549 / 2311 ( 23%) .....................I....................................... 610 / 2311 ( 26%) ............................................................. 671 / 2311 ( 29%) .......................................................S...II 732 / 2311 ( 31%) IIIIIIIIIII...IIIIIIIIIIIIIII................................ 793 / 2311 ( 34%) ............................ Time: 10 seconds, Memory: 97.25Mb There were 35 incomplete tests: [snip] OK, but incomplete or skipped tests! Tests: 821, Assertions: 78982, Incomplete: 35, Skipped: 1.
To find all available groups, run $ composer phpunit:entrypoint -- --list-groups to print the available test groups. To run a particular test group, pass it to phpunit, for example:
$ composer phpunit:entrypoint -- --group Editing
Output format
[edit ]To see the name of each running test, use the TAP format by passing the --tap option to PHPUnit:
$ composer phpunit:entrypoint -- --tap tests/phpunit/includes/libs/IPTest.php TAP version 13 not ok 1 - Error: IPTest::testisIPAddress ok 2 - IPTest::testisIPv6 ok 3 - IPTest::testisIPv4 ok 4 - IPTest::testValidIPs ^C $
This format is great to filter the output for non passing test. For example with the grep command:
$ composer phpunit:entrypoint -- --tap tests/phpunit/includes/libs/IPTest.php | egrep '^not' not ok 1 - Error: IPTest::testisIPAddress $
phpunit also provide a kind of checklist that give out a great output for people not familiar with tests or shells: the testdox format:
$ composer phpunit:entrypoint -- --testdox tests/phpunit/includes/specials/ PHPUnit by Sebastian Bergmann. Configuration read from /srv/trunk/tests/phpunit/suite.xml QueryAllSpecialPages [x] Querypage sql query SpecialRecentchanges [x] Rc ns filter [x] Rc ns filter inversion [ ] Rc ns filter association [x] Rc ns filter association with inversion SpecialSearch [x] Profile and namespace loading
An unchecked box ([ ]), means the test failed, such as the one named « RC ns filter association » above.
How it works
[edit ]Database
[edit ]Integration tests that involve a database are automatically run against a temporary clone of your current wiki's database (the one configured in LocalSettings.php).
Because of this, it is important that the MediaWiki installation being tested has an up to date and correct database, or error messages (such as "TestUser.php: Can't create user on real database") will result.