homepage

I see occasional failures where a broken module prevents "pydoc -k" ("apropos") from working.
Examples of failures (from our downstream bug tracker) are:
https://bugzilla.redhat.com/show_bug.cgi?id=461419 : "pydoc -k" yields "NameError: name 'wglUseFontBitmapsW' is not 
defined"
(a broken OpenGL/WGL/__init__.py module)
https://bugzilla.redhat.com/show_bug.cgi?id=447779 : "pydoc turbogears" doesn't work
(broken TurboGears module)
https://bugzilla.redhat.com/show_bug.cgi?id=246212 : "pydoc -k searchterm" MemoryError. permission denied.
(module only intended to be importable as "root" user)
In each case one or more of the many modules on the system are broken, and importing them leads to an exception being 
raised that isn't ImportError.
In each case, the exception bubbles up through the pydoc call ands leads to it exiting.
Obviously the broken modules should be fixed, but it seems to me that pydoc could be more robust against this situation.
I'm attaching a simple patch which makes "pydoc -k" more robust against this situation, by silently discarding all 
exceptions raised by imported modules.
How does this look?
One downstream bug report requested taking it further:
> Running "pydoc -k" should catch all exceptions while importing modules, and
> display failed modules _after_ all positive keyword matches (not in between).
> It also should redirect stdout/stderr while importing so error message e.a.
> don't clutter up the list of hits. 
to deal with broken modules that spew error messages (this was in https://bugzilla.redhat.com/show_bug.cgi?id=461419#c3 
); I've seen some do it to stdout and some to stderr.
How does this sound? I can try to update the patch for this too, if this sounds sane to you.

Nice improvement. This would also solve the "help(), modules" brokenness, right?

Is it a dup of issue1785?

It doesn't look like it is the same bug to me. This one is more general.

I notice this issue is in stage "unit test needed".
It's not clear to me how to add a unit test for this: one idea I had is to create a broken module in some subdir somewhere, and invoke "pydoc -k" as a subprocess with PYTHONPATH containing the extra subdir (and assert no traceback seen in stderr of the child).
Does that sound sane?

Yes, and I can't think of any other way to approach it.

Does this patch solves only pydoc -k, not all "invalid spam module makes pydoc crash"? (I wanted to consolidate the handful of reports we have into one but did not get the time.)

It turns out that the proposed fix here for pydoc was independently added in the early days of Python 3 but was not backported. That fix for 2.7 plus a fix-in-progress for Issue7367 (for both 2.7 and 3.x) and additional test cases (also in progress) should address most if not all of the pydoc crash issues. I'll also review the other open issues.

New changeset 45862f4ab1c5 by Ned Deily in branch '2.7':
Issue #7425: Refactor test_pydoc test case for '-k' behavior and add
http://hg.python.org/cpython/rev/45862f4ab1c5
New changeset 3acf90f71178 by Ned Deily in branch '2.7':
Issue #7425: Prevent pydoc -k failures due to module import errors.
http://hg.python.org/cpython/rev/3acf90f71178
New changeset 6a45f917f167 by Ned Deily in branch '3.2':
Issue #7425: Refactor test_pydoc test case for '-k' behavior and add
http://hg.python.org/cpython/rev/6a45f917f167
New changeset add444274c3d by Ned Deily in branch '2.7':
Issue #7425 and Issue #7367: add NEWS items.
http://hg.python.org/cpython/rev/add444274c3d 

The applied changesets backport the "ignore exceptions" fix for pydoc -k from 3.x to 2.7 and also refactor test_pydoc to remove unneeded complexity and add test cases for importing bad packages and unreadable package directories (a problem addressed in Issue7367). Applied to 2.7 (for 2.7.3), 3.2 (for 3.2.3), and default (for 3.3.0).