Testsuites and Test Results

Mon Jun 9 09:04:00 GMT 2003

> Ranjit> Apart from the humungous GCC testresults mailing
> Ranjit> list archive, is there a place where I can see "canonical"
> Ranjit> testresults for a i686-pc-linux-gnu target for GCJ so
> Ranjit> that I can compare how my builds (and modifications 
> Ranjit> thereof) fare?
>> In theory we should have 0 FAIL. In practice there are always a few
> things we aren't XFAILing. It would be nice to keep on top of this a
> bit better.

I meant "canonical" results for libjava testsuite, Mauve
and Jacks - maybe we should have weekly test runs against
Mauve and Jacks and have them posted to the GCJ site
so that we can see our progress (or otherwise). I know
that these testsuites keep getting added to, but still...
Now I have a couple of (possibly dumb) questions and
issues (everything is on i686-pc-linux-gnu with a
clean bootstrap of 3.3):
1. What exactly does XFAIL mean? Does it mean that "the
 testcase code is wrong and the compiler flagged an
 error as expected" or "we know that this is a bug
 in the compiler and it will fail until the bug is
 fixed"?
2. I download only "core", "g++" and "java" for any
 GCC release or snapshot. So the normal "make check"
 doesn't really work for me. It's only after seeing
 this page:
 http://gcc.gnu.org/onlinedocs/gccint/libgcj-Tests.html
 that I knew how to go about testing libgcj with
 just the packages I have. IMHO, the first paragraph
 in the page above should also be included in this
 page:
 http://gcc.gnu.org/install/test.html
3. For some weird reason that is not immediately
 obvious to me by looking at the testsuite/Makefile,
 my DejaGNU "runtest" is not picked up by make.
 "runtest" is in "/usr/local/bin" and it definitely
 is in my PATH. I have to explicitly say
 make RUNTEST="/usr/local/bin/runtest" check
4. Even after this, only about 85 tests are actually
 run (all PASS-es).
5. Tcl keeps complaining about INTERPRETER not being
 defined and definining it to "yes" on the make
 command line does not help.
6. I don't know why, but the executables are
 produced with a ".exe" extension. For example,
 "TestEarlyGC.exe", etc. Is this a bug or
 a feature?
Many thanks to anyone who helps me with these
issues.
Ranjit.
-- 
Ranjit Mathew Email: rmathew AT hotmail DOT com
Bangalore, INDIA. Web: http://ranjitmathew.tripod.com/