Analysis of Mauve failures - Part 1

Tue Apr 2 17:38:00 GMT 2002

Hi,
Here are some notes on the Mauve failures I am currently seeing. I don't
have results for 3.0, so I don't know if we are doing better or worse.
But we are able to run a lot more tests now then with 3.0.
> 		=== libjava Summary ===
>> # of expected passes		12252
> # of unexpected failures	208
> # of unexpected successes	3
> # of expected failures	32
> # of untested testcases	14

The following are just notes to myself so I can either fix the issues or
make real bug reports later. But if you have any insights please let me
know.
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 1)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 2)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 4)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 9)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 10)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 12)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 13)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 14)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 15)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 16)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 17)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 19)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 21)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 22)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 24)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 25)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 27)
> FAIL: gnu.testlet.java.beans.Introspector.jdk11: getBeanInfo (number 28)
All these failures seem to be because the test tries to initialize a
java.awt class which fails since java.awt.Toolkit cannot be initialized.
Some tests succeed for Object, BitSet and Applet. So it seems we must
implement more of java.awt first before seeing progress on this test.
> FAIL: gnu.testlet.java.io.BufferedByteOutputStream.interrupt: single-byte writes (number 3)
> FAIL: gnu.testlet.java.io.BufferedByteOutputStream.interrupt: single-byte writes (number 4)
Interaction between a BufferedByteOutputStream wrapped around an
OutputStream that throws an InterruptedIOException after a couple of
bytes. Would be nice to get right but using an OutputStream that is
interrupted seems dangerous anyway.
> FAIL: gnu.testlet.java.io.File.jdk11: getCanonicalPath () java.io.IOException: No such file or directory (number 1)
Don't know if this is actually wrong. getCanonicalPath() may throw an
IOException if the File does not exist since it might need help from the
file system to construct the path (which natFilePosix does since it uses
realpath() if it exists).
> FAIL: gnu.testlet.java.io.PipedStream.close (number 1)
Don't know why this one fails. The PipedOutputStream seems to notify the
PipedInputStream but somehow it keeps waiting till it gets the
InterruptedException.
Skipping analysis of Character failures (seem to be about 80) since Eric
has recently rewritten Character support for Unicode 3.0 and the tests
seem to be written for an older Unicode version. Probably needs a
rewrite of the Mauve tests.
> FAIL: gnu.testlet.java.lang.Class.ClassTest: Error: test_getClassLoader failed - 1 (number 1)
It seems to expect a core class like java.lang.String to have a 'null'
ClassLoader which seems reasonable.
> FAIL: gnu.testlet.java.lang.Class.ClassTest: Error: test_getClassLoader failed - 2 (number 1)
It expects the Class gnu.testlet.java.lang.Class.ClassTest to have a
'null' classloader probably because it is on the classpath. But this is
not true anymore for Java2 based systems. So the tests is now wrong.
> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_toString failed - 8 (number 1)
> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_equals failed CYGNUS: Double.equals - 8 (number 1)
> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_intValue returned wrong results CYGNUS: Float to int conversions - 1 (number 1)
> FAIL: gnu.testlet.java.lang.Double.DoubleTest: Error: test_longValue returned wrong results CYGNUS: Float to int conversions - 1 (number 1)
Don't know what is going on here. If I run these by hand they succeed!?!
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_Basics failed - 3 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_Basics failed - 4 (number 1)
NumberFormatException thrown when constructing new Float("234.34f") and
new Float("1.4e-45f"). The f is legal here. Have to check Classpath
version to see if it fixes this bug.
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_toString failed - 8 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_equals failed - 8 (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: Error: test_intValue returned wrong results - 1 (number 1)
Another Heisenbug. Running by hand makes them succeed!?!
> FAIL: gnu.testlet.java.lang.Float.FloatTest: test_valueOf 3.4e+32f (number 1)
> FAIL: gnu.testlet.java.lang.Float.FloatTest: test_valueOf " -23.45f " (number 1)
Another case of 'f' at the end is legal, but now in Float.valueOf().
Check Classpath implementation.
> FAIL: gnu.testlet.java.lang.Integer.getInteger (number 12)
Fixed by recent patch on branch. Another fix is in Classpath which
should go to the trunk.
> FAIL: gnu.testlet.java.lang.Integer.getInteger (number 17)
Integer.getInteger(null) throws NullPointerException. Believed fixed in
Classpath.
> FAIL: gnu.testlet.java.lang.Integer.new_Integer: decode (number 6)
> FAIL: gnu.testlet.java.lang.Integer.new_Integer: uncaught exception at "decode" number 11: java.lang.NumberFormatException
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 1)
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 2)
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 3)
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 4)
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 5)
> FAIL: gnu.testlet.java.lang.Long.LongTest: getLong (number 6)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 1)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 2)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 4)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 6)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 7)
> FAIL: gnu.testlet.java.lang.Long.getLong (number 8)
All fixed by recent patches on branch. Believed to be also fixed in
Classpath. Wrapper classes should be merged for 3.2.
> FAIL: gnu.testlet.java.lang.Math.MathTest: Error : test_atan2 failed - 5 (number 1)
> FAIL: gnu.testlet.java.lang.Math.max (number 2)
> FAIL: gnu.testlet.java.lang.Math.max (number 18)
Heisenbugs!
> FAIL: Compile gnu/testlet/java/lang/Math/min.o for gnu.testlet.java.lang.Math.min
Strange. Seems to compile fine by hand. But then it gives:
FAIL: gnu.testlet.java.lang.Math.min: Float NaNs (number 1)
got 0.0 but expected NaN
FAIL: gnu.testlet.java.lang.Math.min: Float NaNs (number 2)
got 0.0 but expected NaN
2 of 41 tests failed
So these two tests fail
 harness.check (Float.toString (Math.min (2.0f, Float.NaN)), "NaN");
 harness.check (Math.min (2.0f, Float.NaN), Float.NaN);
But these two succeed???
 harness.check (Float.toString (Math.min (Float.NaN, 2.0f)), "NaN");
 harness.check (Math.min (Float.NaN, 2.0f), Float.NaN);
Strangeness since when making a test program it seems to always do the
right thing...
> FAIL: gnu.testlet.java.lang.String.CASE_INSENSITIVE_ORDER: unicode mappings (number 1)
> FAIL: gnu.testlet.java.lang.String.CASE_INSENSITIVE_ORDER: unicode mappings (number 2)
The source says: "These two tests are unspecified - see Sun bug 4425387"
> FAIL: gnu.testlet.java.lang.StringBuffer.plus (number 1)
Seems to be a compiler error. method_returning_null() + "" should result
in the String "null" not null.
> FAIL: gnu.testlet.java.lang.ref.PhantomReference.phantom: unreachable (number 1)
The test says:
// In this test we make some assumptions about how the GC operates
// that are probably not quite sound. In particular we assume
// System.gc() will collect everything.
Lets assume that is the case for now.
> FAIL: gnu.testlet.java.lang.reflect.Method.invoke: takes_int (number 7)
Test expects a NullPointerException not an IllegalArgumentException when
it supplies a args[0] = null. Quick reading of API makes me think that
the test is wrong.
> FAIL: gnu.testlet.java.lang.reflect.Modifier.toString12 (number 1)
> FAIL: gnu.testlet.java.lang.reflect.Modifier.toString12 (number 2)
Test expects Modifier.STRICT to be represented by the String "strict",
we return "strictfp" which seems correct to me.
OK. That is it for now. Going to get some sleep. Analysis of the other
~60 failures will come later.
Cheers,
Mark
P.S. My machine seems to be unable to reach gcc.gnu.org although I do
get some messages from the mailinglist I seem to be unable to send them
at the moment. We will see if this reaches the list.