Extract value from config.json (Python)

Question

Usually my project directory structure look like:

project_name/
 lib/
 utils.py
 logs/
 main.py
 requirements.txt
 config.json

My config.json file looks like:

{
 "api": {
 "api_key": "abcdefg",
 "api_secret": "hijklmn"
 },
 "trading": {
 "invest_amount": 50,
 "mode": "futures",
 "cumulative_position": false
 }
}

In utils.py, I have a class Config

import typing
import inspect
import os
import json
from typing import Any
class Config(object):
 _CONFIG_FILE: typing.Optional[str] = None
 _CONFIG: typing.Optional[dict] = None
 def __init__(self):
 config_file = Config.get_config_path(frame=inspect.stack()[1])
 with open(config_file, 'r') as f:
 Config._CONFIG = json.load(f)
 @staticmethod
 def get_config_path(frame: inspect.FrameInfo, config_file_name: str = "config.json") -> str:
 caller_file_name = frame[0].f_code.co_filename
 caller_folder_name = os.path.dirname(caller_file_name)
 config_file_name = os.path.join(caller_folder_name, config_file_name)
 return config_file_name
 @staticmethod
 def get_config_value(param: str) -> Any:
 for key in Config._CONFIG.keys():
 value = Config._CONFIG.get(key).get(param, None)
 if value is not None:
 return value
 raise ValueError(f"{param} doesn't exist")

And I will use it in main.py usually:

from lib.utils import Config
def main(trading_mode, invest_amount, cumulative_position):
 print(trading_mode, invest_amount, cumulative_position)
if __name__ == "__main__":
 config = Config()
 main(trading_mode=config.get_config_value(param="mode"),
 invest_amount=config.get_config_value(param="invest_amount"),
 cumulative_position=config.get_config_value(param="cumulative_position"))

Is there any code that can improve or rewrite in class Config, for the purpose of efficiency, readability or clear logic, etc?

• Richard Neumann: I think this is over-engineered. Just load a config file from an explicit path and use the dict returned by json.load().
• tung: Do you mean using with open("config.json", "r")? How should I store the desired values and input these into main()? Should I constructure a function like config.get_config_value for this?
• Richard Neumann: How is assigning a dict to a variable config any different from assigning an instance of Config to such a variable? And given your config is a dict, you can just use cofig.get(key) instead of config.get_config_value(key).
• tung: Can you post the correction on how should I edit the class Config or using config.get(key)? It's a bit confusing for me to understand the words, better in answer the question instead of comment, it would be helpful, many thanks!
• Barb: seems very complex for a simple read function?

Answer 1

The good parts

You code is very readable. You stick to PEP 8 for the most part (except the annoying line length limitation) and use reasonable naming.

Managing configuration

Your first mistake, imho, is that you make the config part of the project. Having the config in the project's repo, especially if it contains things like api_secret (or password etc.) is a common source for secret leakage via repo hosting services such as Git{Hub,Lab}.

Load config explicitly.

Currently, one of the first things your script main.py does is to read data from the file system. Though this is not immediately visible to the reader, since it is hidden in Config.__init__().

You can write a simple function like

from json import load
from pathlib import Path
from typing import Any
...
def load_config(filename: Path | str) -> dict[str, Any]:
 """Load configuration from a JSON file."""
 with open(filename, 'rb') as file:
 return load(file)

to load configuration from a JSON file by explicitly specifying it's path and use the returned dict.

If you have JSON, use JSON

Python's json library already does all the work to parse JSON objects into associative arrays aka. dicts. Use them directly to access the desired data:

from lib.utils import Config
def main(trading_mode, invest_amount, cumulative_position):
 print(trading_mode, invest_amount, cumulative_position)
if __name__ == "__main__":
 config = load_config('/path/to/my/config.json')
 trading_cfg = config.get('trading')
 main(
 trading_mode=trading_cfg.get('mode'),
 invest_amount=trading_cfg.get('invest_amount'),
 cumulative_position=trading_cfg.get('cumulative_position')
 )

All the introspection automagic might feel comfortable at first, but it hides away a lot of potential side-effects. Furthermore, it won't be needed anyways if you separate your configuration from your project source code.

• tung: I usually add config.json into .ignore when using github/gitlab, is there any better alternative for that? If using the load_config function you suggested, will it be a bit slower or redundant to read the config file path everytime I call the function, e.g. I need to extract ten values in config.json then I need to use with open(file_name, "rb") ten times.
• Grajdeanu Alex: @tung have a read about dotenv files
• tung: but I am using config.json instead of .env @GrajdeanuAlex