What is the best way to store configurations of shell/script code to execute later?

Question

I am working on a tool (Python, may or may not be important) that will allow a user to maintain a configuration file containing arbitrary shell and/or language code to be executed on particular events and intervals. The code will potentially involve multiple statements.

I understand there are potential behavioral and security risks associated with executing arbitrary code from a file, but I don't think this is a concern because the configuration file is managed only by the end user. However, let me know if I'm overlooking something major.

My main question: what is the best way to store this code in a configurable way?

---

Some concepts I'm considering to store the code...

In a shared config file (which many Python libraries already look for):

# setup.cfg
[my_program]
command_1 = cd /tmp && touch foobar

In a YAML file:

# my_program.yaml
command_1: cd /tmp && touch foobar

In a source file as arguments to subprocess.call():

# settings.py
command_1 = [['cd', '/tmp'], ['touch', 'foobar']] # requires special handling of 'cd'

In a source file as a function:

# settings.py
import os
import subprocess
def command_1():
 os.chdir('/tmp')
 subprocess.call(['touch', 'foobar'])

• Doc Brown: IMHO that is somewhat a matter of taste, a matter how complex your config code may be (for example, may it contain multiple functions calling each other?) and a matter of the kind of end-user you have in mind.
• Jace Browning: @DocBrown: the end users are programmers setting up build and test commands for various code repositories.
• Izkata: A note, your last example doesn't do the same thing as the first two. The third can go either way depending on implementation, but removes the option from the user.
• Izkata: And I wasn't even referring to the point user13739 makes.. Forgot about that
• Jace Browning: @Izkata: It's fixed now.

Answer 1

How about storing the commands in an actual executable file? The end user creates a bash/python/ruby script, and all they supply to your configuration is the name of the script to execute.

This allows the script to be arbitrarily simple or complex, and also lets the end user use any scripting language supported by their own environment.

Finally, they will need permissions to make the script file executable for this to work, which means they already had permissions to create and run any script anyways.

• Jace Browning: What if I need this file to store many different commands? Should there me multiple scripts? A standard set of arguments I will pass to their scripts? They also tell me what arguments to pass to their script for each command?
• Chris Pitman: @Jace Browning Yes, there should be multiple scripts. If the user wants code reuse, the can always make all of their scripts call into a common codebase. As far as arguments, I would keep it as simple as possible to start and pass a standard set of arguments.
• S.K. Venkat: @ChrisPitman, any examples will make better to understand the ideology you are referring. Thanks...

Answer 2

Well, for your example script only the first two methods will actually work. It has a cd in it, and therefore the two commands must be part of the same shell script.

Therefore, if you want to use commands that have the form of a shell script, you have to store them in form of a string anyway. If the code you want to store may become longer, YAML might be the better idea, since it can handle multi-line strings better.

Answer 3

One option I'm considering is to have users create a Makefile with specific target names.

Advantages:

• all commands are locally testable

Disadvantages:

• requires make