-
Notifications
You must be signed in to change notification settings - Fork 287
mypy and logging.Logger subclasses #980
-
EDIT: I added a pull request in python/typeshed to enable workaround 2.
- issue in typeshed: Logger.getChild bad signature typeshed#6606
- pr: Logger.getChild subclass compatible typehint typeshed#6609
So, let's say we want to use a custom logging.Logger sub-class with additional methods e.g. trace, verbose, passed, failed that provide specialized formatting and/or use custom log-levels.
- Python3 add logging level
- How to extend the logging.Logger Class?
- How to add a custom loglevel to Python's logging facility
However, the proposed solutions, either monkey-patching or using a logging.Logger subclass both do not work well with mypy, because the standard way to get a Logger is using logging.getLogger which is unaware of the subclass. See also this blog post by Sam Hooke.
The issue
Suppose we have the following setup code: In config/base module we define a new Logger subclass:
class MyLogger(logging.Logger): ... logging.setLoggerClass(MyLogger) logger = logging.getLogger(__name__)
Then in submodules:
import logging logger = logging.getLogger(__name__)
We will get type errors (undefined attribute) each and every time we do logger.verbose / logger.trace etc. There was already a very brief discussion here python/typeshed#1801 without any good resolution.
Possible workarounds
So I was wondering, what could be an acceptable solution? Plastering # type: ignore every time we use logger.verbose does not seem reasonable to me. I think there are two possibilities, which however currently are not quite compatible with mypy.
Workaround proposal 1 - add explicit type hint for the subclass
import logging from mymodule.logger import MyLogger logger: MyLogger = logging.getLogger(__name__)
This does not quite work because getLogger has the following signature in typeshed/stdlib/logging`:
def getLogger(name: Optional[str] = ...) -> Logger: ...
Instead, we would need something like
T = TypeVar("T", bound=Logger) def getLogger(name: Optional[str] = ...) -> T: ...
In conjunction with a change of how TypeVar works: if no type hint is given assume the type is the bound type. This was discussed here python/mypy#4236 with no resolution.
Workaround proposal 2 - Import the Logger object and create a child.
from mymodule.logger import logger # returns actual object logger = logger.getChild(__name__)
Here are 2 issues:
- I am not sure what possible unintended consequences this could have compared to using
getLogger. - We still need a signature change in
typeshed/stdlib/logging: theregetChildis defined as
def getChild(self, suffix: str) -> Logger: ...
However, we need something along the lines of
T = TypeVar("T", bound=Logger) def getChild(self: T, suffix: str) -> T: ...
Cf. python/mypy#1212
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 2 comments 2 replies
-
The second workaround seems better to me. It seems unlikely that TypeVar semantics will be changed just because of this.
I am not sure what could be possible unintended consequences this may have compared to using getLogger.
The source code for getChild is:
if self.root is not self:
suffix = '.'.join((self.name, suffix))
return self.manager.getLogger(suffix)
So getLogger("foo.bar").getChild("baz") just calls getLogger("foo.bar.baz"), and there doesn't seem to be any "unintended consequences" or gotchas to be aware of.
As you said, we still need to change the definition of getChild to use a TypeVar. The runtime doesn't do anything to make child loggers to have the same type as their parent, but writing the stubs that way still seems to be the most practical type-safe way to support using custom logger classes.
Beta Was this translation helpful? Give feedback.
All reactions
-
I opened an issue in typeshed: python/typeshed#6606
Beta Was this translation helpful? Give feedback.
All reactions
-
Is there a proper workaround for this today? I'm actually not sure how applicable Workaround 2 is for custom Logger classes
Edit: I just the linked blog post. I am certainly not keen on adding a partial typed for overriding parameters from the logging subclass
Beta Was this translation helpful? Give feedback.
All reactions
-
I think this is the best way today...
foo.py:
import logging from typing import TYPE_CHECKING, cast VERBOSE = 15 class LoggerWithVerbose(logging.Logger): # Use the same type for verbose() as for info(). if TYPE_CHECKING: verbose = logging.Logger.info else: def verbose(self, msg, *args, **kwargs): if self.isEnabledFor(VERBOSE): self._log(VERBOSE, msg, args, **kwargs) logging.basicConfig(level=VERBOSE) logging.addLevelName(VERBOSE, "VERBOSE") logging.setLoggerClass(LoggerWithVerbose) root_logger = cast(LoggerWithVerbose, logging.getLogger())
bar.py:
from foo import root_logger logger = root_logger.getChild(__name__) logger.verbose("hello")
If I now run python3 -c 'import bar', I get VERBOSE:bar:hello. Note that python3 bar.py runs bar.py with __name__ set to "__main__", and will instead print a confusing VERBOSE:__main__:hello.
Beta Was this translation helpful? Give feedback.