homepage

Attached patch is a first crack at a unit test for
whichdb.
I think that all functionality required for use by the
anydbm module is tested, but only for the
database modules found in a given installation.
The test case is built up at runtime to cover all the 
available modules, so it is a bit introspective,
but I think it is obvious that it should run correctly.
Unfortunately it crashes on my box (Redhat 6.2) and 
this seems to be a real problem with whichdb:
it assumes things about the dbm format which turn out
to be wrong sometimes.
I only discovered this because test_anydbm was
crashing,
when whichdb failed to work on dbm files. It would not
have crashed if dbhash was available... and dbhash was
not available
because bsddb was not built correctly. So I think
there is a build bug there, but I have little idea how
to solve that one at this
point.
Would I be correct in thinking that if this test really
uncovers bugs in whichdb, it can't be checked in until
they are fixed? Unfortunately I don't know much about
the various 
databases, but I'll try to work with someone on it.

Logged In: YES 
user_id=6380
What kind of crash do you experience?
Do you have a patch that fixes whichdb?

Logged In: YES 
user_id=11365
I get two failures...
First, using the dbm module as the engine, whichdb fails 
to indentify the type. This is apparently a platform 
problem... whichdbm.py has the comment
# Check for dbm first -- this has a .pag and a .dir file
but on my system the dbm modules creates a .db file.
The 'file' utility says
duh.db: Berkeley DB 2.X Hash/Little Endian (Version 5, 
Logical sequence number:
file - 0, offset - 0, Bucket Size 4096, Overflow Point 1, 
Last Freed 0, Max Bucket 1, High Mask 0x1, Low Mask 0x0, 
Fill Factor 40, Number of Keys 0)
Now, a very simple patch would be to look for .db files 
and call them 'dbm'. I have no idea though whether there 
might be other database formats which use this extension. 
So the thing to do might be to look for .db files and test 
their magic. Actually, the .db files are identified as 
"dbhash" databases if named explicitly to whichdb...
But the dbhash module isn't available due to missing bsddb!
I'm not sure what to make of all this.
I could just assume .db files with dbhash magic are always 
of kind dbm... sound reasonable?
Secondly, dumbdbm doesn't work either, if the database is 
empty... f.read(1) in ["'", '"'] doesn't turn out to be 
true, since the .dir file is empty.
Ok, I've attached a naive patch. Note I'm not even 
looking at testing dbhash or gdbm since they're not built 
on my system. On the other hand since anydbm tries these 
first, maybe they are effectively tested by test_anydbm.

Logged In: YES 
user_id=11365
More detail... the failure mode of test_anydbm is that 
a database freshly created with anydbm.open() can't be 
reopened using the 'r' mode.
Since whichdb returns None we wind up at
 raise error, "need 'c' or 'n' flag to open new db"
Of course, whichdb is to blame for this.

Logged In: YES 
user_id=6380
Greg, you assigned this to Neal Norwitz. Why? Usually bug
reports stay unassigned until a developer shows interest. I
doubt that this is Neal's kind of bug: he's not commented on
the bug report, nor does this match the other bugs that he's
interested in.

Logged In: YES 
user_id=33168
I have reviewed it, but others have stayed on top of this
and I didn't have anything to contribute. I will be glad to
check the patch in when it is in the proper state. But I
don't know much about anydbm.

Logged In: YES 
user_id=11365
Neal posted a list to python-dev of standard library modules 
without unit tests. (<3CB3093C.B7A22727@metaslash.com>, 
1 week ago, subject Re: Stability and change)
That prompted me to address the breakage that I was seeing
in test_anydbm due to whichdb. So I thought he might be
interested. If this interrupts the workflow I'll refrain
from making assignments with so little justification...

Logged In: YES 
user_id=33168
Greg, what is the status of this patch? Do you think it's
ready? It's not clear to me if you think there is a bug in
whichdb or not (your last paragraph in the report).
The current code uses os.extsep, but you use '.' inline. 
This should probably be kept consistent.
I don't understand why you add the check for
filename.endswith('.db') and whichdb(filename+'.db') ==
'dbhash'. Was this to fix your problem on redhat 6.2?
Is it possible to test more cases generically in whichdb?
Not directed at Greg, I don't understand why the code does:
char in ["'", '"'], rather than char in "\"'".
I don't love the code duplication (_fname & _delete_files),
but I'm not sure there's any particular way to deal with
that, rather than add the code to unittest.

Logged In: YES 
user_id=6380
Assigning to Skip on a lark, since he seems to be interested
in these things.

Logged In: YES 
user_id=11365
Skip,
The whichdb patch here is naive. But I think the unit test
might be a good starting point for catching the sort of
problems you've been seeing. Sorry I didn't respond to
Neal's comment, I don't remember getting an email for that
one.

Logged In: YES 
user_id=44345
I'm having trouble getting the unit test to run successfully on my 
system...
Barry and I are fiddling with bsddb building. I think everything in this
area is related, so hopefully some of what we come up with will improve
the chances that this unit test passes.
Skip

Logged In: YES 
user_id=44345
I've attached a somewhat different patch to whichdb.py that I hope 
helps the test case pass in the situation Greg described: anydbm uses 
dbm as the underlying database module and dbmmodule.so was linked 
with the dbm emulation library available with Berkeley DB. In this 
situation, dbm.open("foo", "c") will actually create a Berkeley DB Hash 
file named "foo.db". The patch adds a test for "foo.db" and an attempt 
to open that file using dbm.open() to the check for "dbmness".
I don't think this problem can be completely solved by whichdb.py, since
you can always construct a test case using multiple db modules where
you'll have a filename collision. This may be the best we can do though.
Skip

Logged In: YES 
user_id=44345
Checked in v 1.1