Python logging handler

Question

I'm working on a logger that has a name of the module that called the logger (when I create an instance of a logger in my program, I call LoggingHandler(__name__)) to send all the messages, including info and debug, to the log file, and print the messages specified by max_level to console (so, by default, it will not print info and debug messages to console, but will still write them into file).

The problem came when I was managing levels. If I set level in basicConfig to "WARNING", then it will not print info and debug to file, even though I've set fh.setLevel(logging.DEBUG). It just won't go to levels lower than the one specified in basicConfig. Okay, I could just go ahead and specify filename in basicConfig to make it output to file, but I want a RotatingFileHandler to take care of it (because I require its rollover functionality). So, I've set level in basicConfig to "NOTSET", the lowest one possible. Things go better now except one problem. The output to console doubles. It prints

[2016年08月29日 10:58:20,976] __main__: logging_handler.py[LINE:51]# WARNING hello
[2016年08月29日 10:58:20,976] __main__: logging_handler.py[LINE:51]# WARNING hello
[2016年08月29日 10:58:20,977] __main__: logging_handler.py[LINE:48]# ERROR hola
[2016年08月29日 10:58:20,977] __main__: logging_handler.py[LINE:48]# ERROR hola
[2016年08月29日 10:58:20,977] __main__: logging_handler.py[LINE:54]# INFO info message
[2016年08月29日 10:58:20,977] __main__: logging_handler.py[LINE:57]# DEBUG debug message

So, the global logger does the output and the StreamHandler does. I need to prevent the global logger from outputting anything. So I redirect its output to a "dummy" class Devnull. Now the code works exactly as I need, but it feels like such approach is what they call "bodging". So, I'd like to know if there's a better way to write this code.

#!/usr/bin/python3 -u
# -*- coding: utf-8 -*-
import logging
from logging.handlers import RotatingFileHandler
from os import path, makedirs
LOGS_DIR = "logs"
DEBUG_FILE_NAME = 'debug.log'
MAX_LOG_SIZE = 10*1024*1024
BACKUP_FILES_AMOUNT = 3
LOG_FORMAT = u'[%(asctime)s] %(name)s: %(filename)s[LINE:%(lineno)d]# %(levelname)-8s %(message)s'
class Devnull(object):
 def write(self, *_, **__): pass
class LoggingHandler:
 def __init__(self, logger_name, max_level="WARNING"):
 makedirs(LOGS_DIR, exist_ok=True)
 logging.basicConfig(format=LOG_FORMAT,
 level="NOTSET",
 stream=Devnull(),
 )
 self.main_logger = logging.getLogger(logger_name)
 # create file handler which logs even debug messages
 fh = RotatingFileHandler(path.join(LOGS_DIR, DEBUG_FILE_NAME),
 maxBytes=MAX_LOG_SIZE, backupCount=BACKUP_FILES_AMOUNT)
 fh.setLevel(logging.DEBUG)
 # create console handler with a higher log level
 ch = logging.StreamHandler()
 ch.setLevel(max_level)
 # create formatter and add it to the handlers
 fmter = logging.Formatter(LOG_FORMAT)
 fh.setFormatter(fmter)
 ch.setFormatter(fmter)
 # add the handlers to the logger
 self.main_logger.addHandler(fh)
 self.main_logger.addHandler(ch)
 def error(self, message):
 self.main_logger.error(message)
 def warning(self, message):
 self.main_logger.warning(message)
 def info(self, message):
 self.main_logger.info(message)
 def debug(self, message):
 self.main_logger.debug(message)
if __name__ == '__main__':
 # Tests
 log = LoggingHandler(__name__)
 log.warning("hello")
 log.error("hola")
 log.info("info message")
 log.debug("debug message")

• Dair: Looks like your Devnull solution is used here: stackoverflow.com/questions/2929899/… There is another alternative which appears to be opening a file to os.devnull, but there doesn't seem to many benefits doing that over your current solution.
• Highstaker: @Dair yeah, that's basically where I took it from. I just thought that this solution is better, because it does not rely on external devnull entity.

Answer 1

You simply need to understand what's going on when you write:

 logging.basicConfig(format=LOG_FORMAT,
 level="NOTSET",
 stream=Devnull(),
 )

This creates a root logger with level NOTSET and a default handler which won't log anything thanks to Devnull. I think this also has the added benefit of settings the level of all future loggers to NOTSET.

But you don't want a root logger! You're defining your own logger with your own handlers and so on. So basicConfig() is doing more harm than good here. Remove it!

Now there's one remaining issue: if you do this, you won't see debug logs, because the default log level is WARNING. So you need to add self.main_logger.setLevel(logging.DEBUG): the logger will accept all log levels, and each handler will decide whether or not to process the log record depending on the log level of the record and its own log level. (In other words, you're back to the original code you used which worked very well.)

• Highstaker: Indeed, it helped. I just didn't think about self.main_logger.setLevel(logging.DEBUG). Thanks!