GCC Testing Efforts
This page describes regular efforts to test GCC thoroughly, plus ideas
for additional testing.
For information about running the GCC testsuites, see
Installing GCC: Testing.
For information about testsuite organization and adding new tests, see
Test Suites in the GCC Internals manual and the README files in
the testsuite directories.
Current efforts
- H.J. Lu has several automated Linux/ia32 and Linux/Intel64
testers which build GCC and run the GCC testsuite as well as
SPEC CPU 2K/2006 with various optimization options. All
test results are sent to the
gcc-testresults
mailing list and any regressions are sent to the
gcc-regression
mailing list.
- Several people perform regular builds and regression test runs and
send their test results to the
gcc-testresults
mailing list.
- Jan-Benedict Glaw is running a
build robot that
tries to build various cross-targets (stage1 only) on some machines.
Ideas for further testing
- Perform regular builds and testing of current GCC sources that
are not already being reported regularly; see
Installing GCC:
Testing for instructions on submitting test results.
- Build cross compilers and test with simulators as described in
How to test GCC
on a simulator.
- If your system is beefy enough, do regular builds and test runs with
RTL consistency checks enabled. This slows the compiler down by an
order of magnitude but has found plenty of bugs in the past.
- Set up an autobuilder to nag people in e-mail when they submit
patches that break builds. Ideally we would have one of these
for at least each of the primary evaluation platforms listed in
the current release criteria, but the more the merrier.
Kaveh Ghazi <
ghazi@caip.rutgers.edu> suggests that the autobuilders
should keep track of regressions in the number of warnings and nag
patchers until the new warnings are fixed, just as for testsuite
regressions. It's important to have the autobuilders coordinate
with each other, to avoid flooding contributors with mail.
- Build and test some or all of the following applications, some of
which are part of the GCC release criteria. Links for downloads
and for information about the packages are available in the build
and test guides. If the instructions
are incomplete for your target, update them. Additions to this
list (accompanied with build and test guides) are welcome.
- If the operating system kernel you use is normally compiled with
GCC, try building it with the current sources. Make sure it boots.
If you're building with relatively stable GCC sources such as a
release branch, use the newly built kernel for running further GCC
tests (keeping in mind the NO WARRANTY section of the GPL).
- Build and test applications that are important to you; investigate
and report any problems you find.
- Build and test packages that are normally available on your
platform and for which you have access to source.
- Run benchmarks regularly and report performance regressions.
- Extend the build robot to also do local builds, run the testsuite,
visualize test result differences and probably use something like
BuildBot. Some of the
Compile Farm machines
could also be used.