Python logging - wrapper

Question

I am new to writing utility functions. I wrote a python program to return a Python logger with custom settings. Please review this code and provide comments on how to improve it. I am looking for comments on

1. Coding Style
2. Coherence & Coupling
3. Readability

I am intending to share this code across all projects. So any comments on this to improve are highly appreciated.

import logging
import sys
import logging.handlers
def get_default_log_format():
 """ Method to return the default log format"""
 log_format = logging.Formatter("%(module)s - %(asctime)s — %(name)s — %(levelname)s - %(funcName)s:%(lineno)d — %(message)s")
 return log_format
def get_new_console_handler(log_format=None):
 """This method returns a console handler with the given log_format
 Keyword Arg: log_format accepts logging.Formatter object.
 """
 console_handler = logging.StreamHandler(sys.stdout)
 if log_format is None:
 log_format = get_default_log_format()
 console_handler.setFormatter(log_format)
 return console_handler
def get_new_file_handler(log_format=None, log_file_name="out.log"):
 file_handler = logging.FileHandler(log_file_name)
 if log_format is None:
 log_format = get_default_log_format()
 file_handler.setFormatter(log_format)
 return file_handler
def get_new_rotating_file_handler(log_file_name="out.log", max_log_bytes=2000000, max_log_backup_files=20, log_format=None):
 """ This method returns a file handler with the given file name, max_log_bytes, max_log_backup_files and log_format. This log_format accepts logging.Formatter object"""
# Here, max_log_bytes is the maximum size of file in Bytes and max_log_backup_files is the number of backup files to store.
 file_handler = logging.handlers.RotatingFileHandler(log_file_name, maxBytes=max_log_bytes, backupCount=max_log_backup_files)
 if log_format is None:
 log_format = get_default_log_format()
 file_handler.setFormatter(log_format)
 return file_handler
def get_new_console_logger(logger_name, log_format=None, log_level=logging.DEBUG):
 handler = get_new_console_handler()
 logger = logging.getLogger(logger_name)
 logger.setLevel(log_level)
 logger.addHandler(handler)
 return logger
def get_new_file_logger(logger_name, log_file, log_format=None, log_level=logging.DEBUG):
 handler = get_new_file_handler(log_format, log_file)
 if log_file is None:
 log_file = "execution.log"
 logger = logging.getLogger(logger_name)
 logger.setLevel(log_level)
 logger.addHandler(handler)
 return logger
def get_new_file_console_logger(logger_name, log_file, log_format=None, log_level=logging.DEBUG):
 if log_file is None:
 log_file = "execution.log"
 logger = get_new_file_logger(logger_name, log_file)
 console_handler = get_new_console_handler()
 logger.addHandler(console_handler)
 return logger
if __name__ == "__main__":
 # To get a console handler, run this code
 console_logger = get_new_console_logger(logger_name="logger1")
 console_logger.debug("Hello World")
 # To get a file logger, run this code
 file_logger = get_new_file_logger(logger_name="logger3", log_file="out.log")
 file_logger.critical("This is a critical message")
 # To get a console and file logger, run this code
 console_file_logger = get_new_file_console_logger(logger_name="my_logging2", log_file="out3.log")
 console_file_logger.error("Error: This is from console file logger")
 # logger.info("Begining of execution")
 # logger.info("logging Info message")
 # logger.debug("This is a debug message")
 # logger.debug(logging.DEBUG)
 # logger.info(logging.INFO)
 # logger.warning(logging.WARNING)
 # logger.error(logging.ERROR)
 # logger.critical(logging.CRITICAL)
 # logger.info("End of Execution")

Answer 1

Restructuring and consolidation

Logging configuration

When having a constant logging options like posted log_format or log_level=logging.DEBUG you might have been used logging.basicConfig or even dictConfig to apply specified options globally to all underlying cases.
But since your intention is to allow the client create different specialized loggers with dynamic logging options, "base" config won't give much benefit. At least, it's good to remind of such a base configuration capability.

---

Handlers

All 3 handlers run the same common set of statements to ensure logging format:

if log_format is None:
 log_format = get_default_log_format()
console_handler.setFormatter(log_format)

That calls for consolidation and reducing duplication.
Moreover, when creating some concrete logger instance a client may pass log_format as a raw format string like log_format='%(funcName)s:%(lineno)d — %(message)s', not as logging.Formatter instance. Thus, for further optimizations I'll eliminate get_default_log_format function and declare a constant:

LOG_FMT = '%(module)s - %(asctime)s — %(name)s — %(levelname)s - %(funcName)s:%(lineno)d — %(message)s'

According to the scenario where the client/caller operates only on .._logger functions to instantiate concrete loggers, that points to direct relation between specific logger and respective handler(s). And assuming that duplicated common code should be moved out from all handlers - they ceased pulling their weigh and are removed in favor of direct handler usage. See the final implementation below.

---

..._logger functions

"logger" creation functions perform a common set of actions:

logger = logging.getLogger(logger_name)
logger.setLevel(log_level)
logger.addHandler(handler)

that also calls for consolidation.
The condition

if log_file is None:
 log_file = "execution.log"

is eliminated by simply making log_file a default argument log_file="execution.log".

---

To perform consolidation and reducing duplication a factory function logger_factory(logger_name, handlers_list, log_format, log_level) is introduced. It concentrates a common behavior and also allows to attach multiple handlers to the same logger.

The final optimized version:

import logging
import sys
import logging.handlers
LOG_FMT = '%(module)s - %(asctime)s — %(name)s — %(levelname)s - %(funcName)s:%(lineno)d — %(message)s'
def logger_factory(logger_name, handlers_list, log_format, log_level):
 logger = logging.getLogger(logger_name)
 logger.setLevel(log_level)
 if not isinstance(handlers_list, (list, tuple)):
 handlers_list = [handlers_list]
 for handler in handlers_list:
 handler.setFormatter(logging.Formatter(log_format))
 logger.addHandler(handler)
 return logger
def create_console_logger(logger_name, log_format=LOG_FMT, log_level=logging.DEBUG):
 return logger_factory(logger_name, handlers_list=[logging.StreamHandler(sys.stdout)],
 log_format=log_format, log_level=log_level)
def create_file_logger(logger_name, log_file="execution.log", log_format=LOG_FMT, log_level=logging.DEBUG):
 return logger_factory(logger_name, handlers_list=[logging.FileHandler(log_file)],
 log_format=log_format, log_level=log_level)
def create_rotating_file_logger(logger_name, log_file="out.log", max_log_bytes=2000000, max_log_backup_files=20,
 log_format=LOG_FMT, log_level=logging.DEBUG):
 """ Creates rotating file logger with the given file name, max_log_bytes, max_log_backup_files and log_format.
 :param log_file: log file name
 :param max_log_bytes: the maximum size of file in Bytes
 :param max_log_backup_files: the number of backup files to store
 :param log_format: custom format as logging.Formatter object
 :param log_level: logging level
 :return: logging.Logger
 """
 handler = logging.handlers.RotatingFileHandler(log_file, maxBytes=max_log_bytes,
 backupCount=max_log_backup_files)
 return logger_factory(logger_name, handlers_list=[handler],
 log_format=log_format, log_level=log_level)
def create_file_console_logger(logger_name, log_file="execution.log", log_format=LOG_FMT, log_level=logging.DEBUG):
 handlers = [logging.FileHandler(log_file), logging.StreamHandler(sys.stdout)]
 return logger_factory(logger_name, handlers_list=handlers,
 log_format=log_format, log_level=log_level)
if __name__ == "__main__":
 # To get a console handler, run this code
 console_logger = create_console_logger(logger_name="logger1")
 console_logger.debug("Hello World")
 # To get a file logger, run this code
 file_logger = create_file_logger(logger_name="logger3", log_file="out.log")
 file_logger.critical("This is a critical message")
 # To get a console and file logger, run this code
 console_file_logger = create_file_console_logger(logger_name="my_logging2", log_file="out3.log")
 console_file_logger.error("Error: This is from console file logger")

• Mithilesh_Kunal: Thanks for the detailed answer. It helped me :)
• RomanPerekhrest: @Mithilesh_Kunal, you're welcome