homepage

The current behaviour of logging.Logger.findCaller() makes it awkward to add custom logging interfaces.
E.g. suppose that you define a custom logging level (NOTICE) then add a notice() function (analogous to logging.info() etc), the resulting LogRecord structure will have pathname, filename, module, funcName and lineno members which refer to the notice() function itself, rather than to its caller.
This can be hacked around e.g. by using code.compile_command to lie about the source filename, but that's ugly and highly unintuitive.

What about subclassing Logger? Then you could override findCaller() to do whatever you need, as well as adding a notice() method, for example.
I agree using the source filename isn't ideal, but it's hard to know how people might arbitrarily wrap logging calls and still expect the wrappers to be skipped by findCaller(). Do you have any suggested mechanism for doing this?

Subclassing findCaller() is likely to be a bit hairy for the average user, as it deals with some fairly arcane aspects of Python internals.
As for mechanism, maybe allow functions/methods to be registered with the logging module, which would store a referenceto the function's func_code member. findCaller() would check the frame's f_code member against the set of registered functions, and continue up the stack until it found an unregistered function.

I will give this some thought, but it won't be possible to add it to Python 2.x (which is closed for feature additions). Any enhancement in this area would apply to Python 3.4 and later.

Passing a globals() instance looks hackish.
importlib uses another approach:
http://hg.python.org/cpython/file/5ef49659935f/Lib/importlib/_bootstrap.py#l303
Note that it is a pity that code objects don't have a __module__ attribute. Perhaps it would be a nice feature for 3.4.

> Passing a globals() instance looks hackish.
Agreed it's not ideal, but we're constrained by what's available to compare against via the frame. Apart from globals() there's only really the filename, which isn't ideal either.

Too late for 3.4 now, bumping to 3.5. Implementation now uses a set and '__name__' in f.f_globals to match module names passed in - this seems good enough. I plan to commit this as soon as default is updated to 3.5, unless there are alternative suggestions.

I think we need to look seriously at the frame annotations idea discussed in other issues. Eliminating noise from tracebacks and correctly reporting user code rather than infrastructure could should be achievable through local state rather than needing global registries.
The workaround we put in place for importlib is an awful hack, and there's a problem where PEP 3144 allows the creation of exception *trees*, but we can currently only record stacks properly.

> the frame annotations idea discussed in other issues
If you mean #19585 and #18861, they seem to be related to exceptions - the logging use case is not exception-related. I couldn't find any other discussions about frame annotations.

My idea is to annotate the frames appropriately so they can be *displayed*
differently when showing a traceback (either hiding them entirely or
displaying additional information). This would be another use case -
annotating the frame to say the logging module should skip over it when
looking for the caller.

I recently implemented a custom logger derived from Logger and to get the
reporting of modules, etc, correct, I implemented the findCaller using
the same code as library code with the minor change of
if filename == _srcfile:
to
if filename in [logging._srcfile, _mysrcfile]:
with the local _mysrcfile being determined the same as in logging.
Maybe simply having a class list "srcfiles = [_srcfile]" and allowing
people who customize extend this would be the easiest way of handling
this: "Logger.srcfiles.append(_mysrcfile)"

> Maybe simply having a class list
Thanks, I'm aware of this approach and have considered it. The current patch's skipCallers API seems better than exposing an internal list, but I have held off implementing this in the hope that an approach based on frame annotations can be used.

This should be addressed by the fix for bpo-33165 - addition of a stacklevel parameter to the logging APIs.