Singleton handler for reading and writing configuration.json

Question

In a current project of I have multiple modules that need to read from and write to a config.json file, where the settings are saved as nested dictionaries. I chose json here, because I want the config file to be human-readable.

Simplified example config.json:

{
 "server_data": {
 "email_address": "[email protected]",
 "server_address": "imap.server.com",
 "server_port": 993
 },
 "handle_emails": {
 "output_directory": "C:\\Users\\riskypenguin\\some_directory",
 "mark_read": false,
 "write_to_file": true,
 "folders": ["inbox"]
 },
 "output": {
 "output_directory": "C:\\Users\\riskypenguin\\some_other_directory",
 "print_timeinfo": true,
 "print_form_counts": true
 }
}

---

My config_handler.py provides standardised access to the file via get_setting and set_setting. For different use cases throughout the project I need to be able to access the configuration settings at varying depths. For example: Some boolean settings are read and written directly, but the dict at server_data will be read and processed as a whole. Concurrency is not relevant for this project, so there are no collisions regarding read and write access.

config_handler.py

import os
import json
from functools import reduce
from definitions import CONFIG_DIR, CONFIG_FILE, DEFAULT_CONFIG_FILE
__all__ = ["get_setting", "set_setting"]
# Helper functions
def _save_config_data(content):
 if not os.path.exists(CONFIG_DIR):
 os.makedirs(CONFIG_DIR)
 with open(CONFIG_FILE, 'w') as f:
 json.dump(obj=content, fp=f, indent=2)
def _load_config_data():
 file = CONFIG_FILE if os.path.exists(CONFIG_FILE) else DEFAULT_CONFIG_FILE
 with open(file, 'r') as f:
 return json.load(f)
_config_data = _load_config_data()
# Exported functions
# Old version of get_setting, currently not in use
def get_setting_old(*args, default=None):
 current_value = _config_data
 for key in args:
 if isinstance(current_value, dict):
 current_value = current_value.get(key, default)
 else:
 return default
 return current_value
def get_setting(*config_keys, default=None):
 drill_down = lambda x, y: dict.get(x, y, default) if isinstance(x, dict) else default
 return reduce(drill_down, config_keys, _config_data)
def set_setting(*config_keys, value):
 current_value = _config_data
 for key in config_keys[:-1]:
 if isinstance(current_value, dict):
 current_value = current_value.get(key)
 else:
 return False
 last_key = config_keys[-1]
 if isinstance(current_value, dict):
 current_value[last_key] = value
 else:
 return False
 _save_config_data(_config_data)
 return True

---

As this is my first shot at handling configurations in a central way like this I am grateful for all feedback. I'm particularly interested in feedback about:

1. General approach
2. Comparison between the haskell-like approach of get_setting vs. the simpler and more readable approach of get_setting_old
3. set_setting seems rather unelegant to me. As the types of a setting can be either mutable or immutable I don't see a better solution though. I've thought about wrapper classes for primitive types, but that feels rather unpythonic and creates some unnecessary overhead when reading from and before writing to json. Maybe I'm missing something here.
4. I've been thinking about creating a simple class for a path of config_keys, to be able to pass a ConfigKeyPath object instead of a bunch of separated string arguments. I'm not sure which of the two approaches is better.

• Peilonrayz: Why are you only allowed to get items from dictionaries? Why are lists on anything else that implements __getitem__ not allowed?
• riskypenguin: The short answer is that I hadn't thought of that up until now. I guess my current approach is: Anything that is not a dict marks the lowest level of a setting. Any further handling needs to be done by the code that retrieves the setting. Refactoring get_setting it to use __getitem__ instead of dict.get should be easy enough though.

Answer 1

1. Rather than os and os.path use pathlib.Path.

   def _save_config_data(content):
    CONFIG.parent.mkdir(parents=True, exist_ok=True)
    with open(CONFIG, 'w') as f:
    json.dump(obj=content, fp=f, indent=2)
   def _load_config_data():
    file = CONFIG if CONFIG.exists() else DEFAULT_CONFIG
    with open(file, 'r') as f:
    return json.load(f)
   

2. Your code doesn't follow idiomatic Python. Python uses errors for control flow so returning True/False in set_setting is not idiomatic.

3. You should make a walk function which you call from both get_setting and set_setting.

4. Reduce is generally not a good solution. I would say in the specific example of your code, get_setting_old has far superior readability to get_setting.

import os
import json
from functools import reduce
from definitions import CONFIG, DEFAULT_CONFIG
__all__ = [
 "get_setting",
 "set_setting",
]
# Helper functions
def _save_config_data(content):
 CONFIG.parent.mkdir(parents=True, exist_ok=True)
 with open(CONFIG, 'w') as f:
 json.dump(obj=content, fp=f, indent=2)
def _load_config_data():
 file = CONFIG if CONFIG.exists() else DEFAULT_CONFIG
 with open(file, 'r') as f:
 return json.load(f)
def _walk(obj, path):
 for segment in path:
 obj = obj[segment]
 return obj
def get_setting(*args, default=_SENTINEL):
 try:
 return _walk(_CONFIG_DATA, args)
 except LookupError:
 if default is _SENTINEL:
 raise LookupError(f"cannot walk path; {args}") from None
 else:
 return default
def set_setting(value, *args):
 *args, segment = args
 try:
 node = _walk(_CONFIG_DATA, args)
 node[segment] = value
 except LookupError:
 raise LookupError(f"cannot set path; {args}") from None
_SENTINEL = object()
_CONFIG_DATA = _load_config_data()

5. I think using a class would be better as then you can define __getitem__ and __setitem__ for a potentially nicer interface.

class Walker:
 def __init__(self, data):
 self._data = data
 
 def __getitem__(self, args):
 if not isinstance(args, tuple):
 args = (args,)
 return self.get(*args)
 def __setitem__(self, args, value):
 if not isinstance(args, tuple):
 args = (args,)
 self.set(value, *args)
 def _walk(self, obj, path):
 for segment in path:
 obj = obj[segment]
 return obj
 def get(self, *args, default=_SENTINEL):
 try:
 return _walk(_CONFIG_DATA, args)
 except LookupError:
 if default is _SENTINEL:
 raise LookupError(f"cannot walk path; {args}") from None
 else:
 return default
 def set(self, value, *args):
 *args, segment = args
 try:
 node = _walk(_CONFIG_DATA, args)
 node[segment] = value
 except LookupError:
 raise LookupError(f"cannot set path; {args}") from None

• riskypenguin: I guess I was a bit too caught up in last semesters functional programming lecture when opting for reduce. Although I enjoyed its conciseness, I do agree it harms readability. Thank you for the comprehensive suggestions. I will take a closer look at pathlib as well as your suggestions regarding the general approach.
• Peilonrayz: @riskypenguin wrt pathlib the very last section of the docs will likely be very helpful. Yeah different languages have different idioms. A Haskell programmer would look favourably on code which uses reduce. A C# programmer would use the tryGet pattern here. A good course will expose you to many common patterns, but not all patterns are good in every language.