How to accept argparse.log_level parameter and propagate its value to all other modules

Wed Jan 27 16:59:23 EST 2021

On 27/01/2021 20:04, Zoran wrote:
> In the same folder I have three files: main.py, app_logger.py and mymodule.py with their contents:
>> # main.py
> import argparse
> import app_logger
> import mymodule
> import logging
>> logger = app_logger.get_logger(__name__)
>> def log_some_messages():
> logger.debug(f'{__name__} - debug message')
> logger.info(f'{__name__} - info message')
> logger.warning(f'{__name__} - warning message')
>> if __name__ == '__main__':
> parser = argparse.ArgumentParser(description="Script for executing data quality checks.")
> parser.add_argument("--log-level", default='info', choices=['notset', 'debug', 'info', 'warning', 'error', 'critical'], help="set log level")
> args = parser.parse_args()
>> logger.setLevel(eval(f"logging.{args.log_level.upper()}"))
>> log_some_messages()
> mymodule.log_some_messages()
>>> #mymodule.py
> import logging
>> from script_logging import app_logger
>> logger = app_logger.get_logger(__name__)
>>> def log_some_messages():
> logger.debug(f'{__name__} - debug message')
> logger.info(f'{__name__} - info message')
> logger.warning(f'{__name__} - warning message')
>>> #app_logger.py
> import logging
>> _log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - (%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
> _log_level = logging.DEBUG
>> def get_stream_handler():
> stream_handler = logging.StreamHandler()
> stream_handler.setLevel(_log_level)
> stream_handler.setFormatter(logging.Formatter(_log_format))
> return stream_handler
>> def get_logger(name):
> logger = logging.getLogger(name)
> logger.setLevel(_log_level)
> logger.addHandler(get_stream_handler())
> return logger
>> When I run it with main.py --log-level debug I get the following output:
>> 2021年01月25日 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) - __main__ - debug message
> 2021年01月25日 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - __main__ - info message
> 2021年01月25日 13:21:01,152 - [WARNING] - __main__ - (main.py).log_some_messages(12) - __main__ - warning message
> 2021年01月25日 13:21:01,152 - [DEBUG] - mymodule - (mymodule.py).log_some_messages(10) - mymodule - debug message
> 2021年01月25日 13:21:01,152 - [INFO] - mymodule - (mymodule.py).log_some_messages(11) - mymodule - info message
> 2021年01月25日 13:21:01,152 - [WARNING] - mymodule - (mymodule.py).log_some_messages(12) - mymodule - warning message
>> which is OK. If I change --log-level parameter to 'warning', than output is the following:
>> 2021年01月25日 13:22:12,760 - [WARNING] - __main__ - (main.py).log_some_messages(12) - __main__ - warning message
> 2021年01月25日 13:22:12,760 - [DEBUG] - mymodule - (mymodule.py).log_some_messages(10) - mymodule - debug message
> 2021年01月25日 13:22:12,761 - [INFO] - mymodule - (mymodule.py).log_some_messages(11) - mymodule - info message
> 2021年01月25日 13:22:12,761 - [WARNING] - mymodule - (mymodule.py).log_some_messages(12) - mymodule - warning message
>> As you can see, in main.py log level is set correctly, but in mymodule.py it is not.
>> How to accept argparse.log_level parameter in if __name__ == '__main__': and propagate its value to all other modules?

You just have to set the level in the root logger. The easiest way to 
achieve that is to forget about your app_logger wrapper around logging 
and call
logging.basicConfig(level=yourlevel, format=yourformat)
in main.py and to create the loggers in main and mymodule
with
logger = logging.getLogger(__name__)