Python code template for scripts to be run in batch

Question

I will be writing a number of Python scripts which will be run in batch daily on a Windows server. I am trying to develop a template for writing the scripts. My intention is to use a configuration file to hold the settings (for example: the location the log files will be saved in).

Below is the template I have developed so far and was wondering - can it be improved in some way? Thank you.

import datetime
from pathlib import Path
import configparser, argparse, logging
def main():
 logging.info('Start Execution')
 #
 # Code goes here
 #
 logging.info('Execution Complete')
if __name__ == '__main__':
 parser = argparse.ArgumentParser()
 parser.add_argument(
 "-cf", "--config-file", type=str, required=True,
 help="Path and filename to the config file.",
 default=None
 )
 parser.add_argument(
 "-ll", "--loglevel", type=str, required=False,
 help="Defines the level of detail to be recorded in the log files; default is INFO.",
 default='INFO'
 )
 argvs = parser.parse_args()
 #Check a valid log level has been given
 numeric_level = getattr(logging, argvs.loglevel.upper(), None)
 if not isinstance(numeric_level, int):
 raise ValueError(f"Invalid log level: {argvs.loglevel}")
 
 # Check the config file exists
 if not Path(argvs.config_file).is_file():
 raise FileNotFoundError(f"Config File: {argvs.config_file}")
 # Open and read-in the config file
 # The Config File is open and read here as it makes the values available globally.
 config = configparser.ConfigParser()
 config.read(argvs.config_file)
 #Configure logging
 logging.basicConfig(
 filename=config['SETTINGS']['log_path']+datetime.datetime.now().strftime("LOG_%G-%m-%d")+'.log'
 ,format='%(asctime)s - %(message)s'
 ,level=numeric_level
 )
 
 main()

• JimmyJames: When you say 'template', do you mean you plan to copy this into each project that will run here?
• Garrie Powers: Yes @JimmyJames I do intend to copy it into each project.

Answer 1

There's nothing about your script that strikes me as especially wrong. You might find an issue here or there when you start using it. One potential improvement, however is to not copy this into a lot of projects but instead use one module to drive all the scripts. You can do this using the add_subparsers method of argparse. If you've used commandline git or docker, the style of CLI is like that: there's a main argument which determines the subcommand which can then have it's own help and arguments specific to it.

If you were going down that route (and I'm not sure you need to), I would make each script a package which has a method that can take a parent parser and add it's own arguments as well as a method which accepts and args object.

This is absolutely a viable approach that I have used in actively maintained application. If you want me to provide some example code, please ask. I'm not sure, however, if your situation would indicate this solution. If this top level script will need to be copied to a a significant number of projects e.g. more than 3, I would consider it (or a similar approach) from a long-term maintenance perspective.

• Garrie Powers: Thank you! That is very helpful and encouraging. I will stick with this approach for the moment but I will also look into the add_subparsers.