Saving multiple files in python

Question

I have this stupid code block below to save multiple files in Python. It is kind of repetitive (but it works). I was wondering how to improve it, notably:

Is it worth it to write a function for two lines of code ? Is it possible to get the generic object name so that i don't have to specify it in the 'open' ? Are there other approaches / optimisations to make it better / looks nicer ?

with open(RES_PATH + 'res_folds_seeds.p', 'wb') as f:
 pickle.dump(res_folds_seeds, f)
 
with open(RES_PATH + 'res_folds.p', 'wb') as f:
 pickle.dump(res_folds, f)
 
with open(RES_PATH + 'res_folds_neutralised.p', 'wb') as f:
 pickle.dump(res_folds_neutralised, f) 
 
with open(RES_PATH + 'dict_coeffs.p', 'wb') as f:
 pickle.dump(dict_coeffs, f)
 
with open(RES_PATH + 'dict_coeffs_seeds.p', 'wb') as f:
 pickle.dump(dict_coeffs_seeds, f)
 
with open(RES_PATH + 'preds.p', 'wb') as f:
 pickle.dump(preds, f)
 
with open(RES_PATH + 'exps.p', 'wb') as f:
 pickle.dump(exps, f)

Answer 1

Is it worth it to write a function for two lines of code?

In this case, yes.

Are there other approaches / optimisations to make it better / looks nicer ?

Yes: use pathlib -

from pathlib import Path
import pickle
from typing import Any
RES_PATH = Path('.')
def pdump(stem: str, data: Any) -> None:
 path = (RES_PATH / stem).with_suffix('.p')
 with path.open('wb') as f:
 pickle.dump(obj=data, file=f)

You can then call this function with a sequence as Harith demonstrates.

Is it possible to get the generic object name so that i don't have to specify it in the 'open'?

Yes. With your current code you would need to use locals() (the bad kind of clever; I don't recommend doing this). A somewhat cleaner approach would put all of your objects in a parent class Persisted(NamedTuple), and then iterate over all of them.

from pathlib import Path
import pickle
from typing import Any, NamedTuple
RES_PATH = Path('.')
def pdump(stem: str, data: Any) -> None:
 path = (RES_PATH / stem).with_suffix('.p')
 with path.open('wb') as f:
 pickle.dump(obj=data, file=f)
Mystery = Any
class Persisted(NamedTuple):
 res_folds_seeds: Mystery
 res_folds: Mystery
 res_folds_neutralised: Mystery
 dict_coeffs: dict[Mystery, Mystery]
 preds: Mystery
 exps: Mystery
 def dump(self) -> None:
 for name in self.__annotations__.keys():
 pdump(stem=name, data=getattr(self, name))

Answer 2

You can simplify your code by creating a list of tuples where each tuple consists of the filename and the corresponding data, and then looping over the list and performing the operation in a loop like so:

files_data = [
 ('res_folds_seeds.p', res_folds_seeds),
 ('res_folds.p', res_folds),
 ('res_folds_neutralised.p', res_folds_neutralised),
 ('dict_coeffs.p', dict_coeffs),
 ('dict_coeffs_seeds.p', dict_coeffs_seeds),
 ('preds.p', preds),
 ('exps.p', exps)
]
for file_name, data in files_data:
 with open(RES_PATH + file_name, 'wb') as f:
 pickle.dump(data, f)

• tripleee: If the correspondence is trivial, you could just have a list of strings and convert those into a tuple of file name, variable. (The variable's value is globals()[string] if it's global.)