Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

mypy and logging.Logger subclasses #980

Unanswered
randolf-scholz asked this question in Q&A
Discussion options

EDIT: I added a pull request in python/typeshed to enable workaround 2.


So, let's say we want to use a custom logging.Logger sub-class with additional methods e.g. trace, verbose, passed, failed that provide specialized formatting and/or use custom log-levels.

However, the proposed solutions, either monkey-patching or using a logging.Logger subclass both do not work well with mypy, because the standard way to get a Logger is using logging.getLogger which is unaware of the subclass. See also this blog post by Sam Hooke.

The issue

Suppose we have the following setup code: In config/base module we define a new Logger subclass:

class MyLogger(logging.Logger):
 ...
logging.setLoggerClass(MyLogger)
logger = logging.getLogger(__name__)

Then in submodules:

import logging
logger = logging.getLogger(__name__)

We will get type errors (undefined attribute) each and every time we do logger.verbose / logger.trace etc. There was already a very brief discussion here python/typeshed#1801 without any good resolution.

Possible workarounds

So I was wondering, what could be an acceptable solution? Plastering # type: ignore every time we use logger.verbose does not seem reasonable to me. I think there are two possibilities, which however currently are not quite compatible with mypy.

Workaround proposal 1 - add explicit type hint for the subclass

import logging
from mymodule.logger import MyLogger
logger: MyLogger = logging.getLogger(__name__)

This does not quite work because getLogger has the following signature in typeshed/stdlib/logging`:

def getLogger(name: Optional[str] = ...) -> Logger: ...

Instead, we would need something like

T = TypeVar("T", bound=Logger) 
def getLogger(name: Optional[str] = ...) -> T: ...

In conjunction with a change of how TypeVar works: if no type hint is given assume the type is the bound type. This was discussed here python/mypy#4236 with no resolution.

Workaround proposal 2 - Import the Logger object and create a child.

from mymodule.logger import logger # returns actual object
logger = logger.getChild(__name__)

Here are 2 issues:

  1. I am not sure what possible unintended consequences this could have compared to using getLogger.
  2. We still need a signature change in typeshed/stdlib/logging: there getChild is defined as
def getChild(self, suffix: str) -> Logger: ...

However, we need something along the lines of

T = TypeVar("T", bound=Logger)
def getChild(self: T, suffix: str) -> T: ...

Cf. python/mypy#1212

You must be logged in to vote

Replies: 2 comments 2 replies

Comment options

The second workaround seems better to me. It seems unlikely that TypeVar semantics will be changed just because of this.

I am not sure what could be possible unintended consequences this may have compared to using getLogger.

The source code for getChild is:

 if self.root is not self:
 suffix = '.'.join((self.name, suffix))
 return self.manager.getLogger(suffix)

So getLogger("foo.bar").getChild("baz") just calls getLogger("foo.bar.baz"), and there doesn't seem to be any "unintended consequences" or gotchas to be aware of.

As you said, we still need to change the definition of getChild to use a TypeVar. The runtime doesn't do anything to make child loggers to have the same type as their parent, but writing the stubs that way still seems to be the most practical type-safe way to support using custom logger classes.

You must be logged in to vote
1 reply
Comment options

I opened an issue in typeshed: python/typeshed#6606

Comment options

Is there a proper workaround for this today? I'm actually not sure how applicable Workaround 2 is for custom Logger classes

Edit: I just the linked blog post. I am certainly not keen on adding a partial typed for overriding parameters from the logging subclass

You must be logged in to vote
1 reply
Comment options

I think this is the best way today...

foo.py:

import logging
from typing import TYPE_CHECKING, cast
VERBOSE = 15
class LoggerWithVerbose(logging.Logger):
 # Use the same type for verbose() as for info().
 if TYPE_CHECKING:
 verbose = logging.Logger.info
 else:
 def verbose(self, msg, *args, **kwargs):
 if self.isEnabledFor(VERBOSE):
 self._log(VERBOSE, msg, args, **kwargs)
logging.basicConfig(level=VERBOSE)
logging.addLevelName(VERBOSE, "VERBOSE")
logging.setLoggerClass(LoggerWithVerbose)
root_logger = cast(LoggerWithVerbose, logging.getLogger())

bar.py:

from foo import root_logger
logger = root_logger.getChild(__name__)
logger.verbose("hello")

If I now run python3 -c 'import bar', I get VERBOSE:bar:hello. Note that python3 bar.py runs bar.py with __name__ set to "__main__", and will instead print a confusing VERBOSE:__main__:hello.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet

AltStyle によって変換されたページ (->オリジナル) /