ParTcl - a minimal Tcl interpreter

Note: This is a fork; see https://github.com/zserge/partcl for the original.

In 1988, John Ousterhout developed Tcl with the goal to be a light-weight extension language for applications; a "macro language", so to speak. It has grown significantly, and is now often used to create complete tools or utilities in.

ParTcl reverts back to the roots of Tcl: ParTcl's goal is to be easily embeddable in applications, and function as an extension language for these applications. In pursuit of simplicitly and compactness of the implementation, ParTcl is closer to the original versions of Tcl (pre 7.0), in syntax and in semantics, than to the lastest release (9.0, at the time of writing).

There is documentation in this repository: Tcl-Tutorial.pdf is a tutorial specific for ParTcl. For more details, or background information on Tcl, you can pick up a general book on Tcl. Old books are fine (because ParTcl draws back to the original Tcl). See, for example, "Tcl and the Tk Toolkit" by John Ousterhout, and which is available for free.

Features

• Easily embeddable, plain C99 code (one C file, one H file).
• No external dependencies, apart from the standard C library.
• Good test coverage.
• Flexible and easy-to-use interface to C/C++ programs, can be extended with custom Tcl commands.
• A decent number of built-in commands; some less-essential commands can be removed on compilation (by setting a macro).
• Includes an expression parser (the expr command) which supports integer and floating point arithmetic.

Usage

The API is documented in the tcl.h file.

struct tcl tcl;
const char *script = "set x 4; puts [expr 2 + $x * 10]";
tcl_init(&tcl, NULL);
if (tcl_eval(&tcl, script, strlen(script)) != FERROR) {
 struct tcl_value *retval = tcl_return(&tcl);
 printf("Return: %.*s\n", tcl_length(retval), tcl_data(retval));
} else {
 int line;
 const char *msg = tcl_errorinfo(&tcl, &line);
 printf("Error on or after line %d: %s\n", line, msg);
}
tcl_destroy(&tcl);

When TCL is used to script applications, a common scenario is that the user-supplied script contains procedures (and possibly global variables), and that the application calls these procedures. The first step (after initializing the interpreter context with tcl_init) is to call tcl_eval on the user-supplied script. This adds all procs and all global variables to the interpreter context. Until the context is cleared with tcl_destroy, the application can invoke any proc (with parameters) by constructing a command and passing that to tcl_eval.

TCL Synopsis

There are a few key concepts in ParTcl: values, lists and variables.

Values

A "value" is a string, as used internally by ParTcl. It can be used as a C-language string (it is zero-terminated). However, it may contain embedded zero bytes (in case the Tcl script works on binary data), so it is safer to explicitly get the length. Values are allocated dynamically. You create a value with tcl_value() and delete it with tcl_free().

Function tcl_data() returns a pointer to the byte string, tcl_length() its length. You should not modify the contents directly; instead, you create a new value and delete the old one. The raw data of a value may move in memory; you should therefore not cache the pointer returned by tcl_data() either.

An exception to the "don't modify a value" rule, is that you can append a value to another with tcl_append(). The tcl_append() function modifies the primary value (first argument) and deletes the "tail" argument.

/* Raw string values */
struct tcl_value *tcl_value(const char *data, size_t len);
struct tcl_value *tcl_dup(struct tcl_value *value);
bool tcl_append(struct tcl_value *value, struct tcl_value *tail);
struct tcl_value *tcl_free(struct tcl_value *value);
/* Helpers to access raw string or numeric value */
const char *tcl_data(struct tcl_value *value);
int tcl_length(struct tcl_value *value);
bool tcl_type(const struct tcl_value *value);
long tcl_intvalue(struct tcl_value *value);
double tcl_floatvalue(struct tcl_value *value);

Function tcl_type() is a helper function to check whether a value represents a string or a number (and whether that is in integer or floating point format). The functions tcl_intvalue and tcl_floatvalue return the value of a numeric parameter, in long or double formats respectively.

Lists

A list is a string, as is common in Tcl. Thus, in ParTcl, a list is a "value". There are a few special functions on lists, however, that make sure that the list is well-formed. You start a new (empty) list with tcl_list_new() and add items to it with tcl_list_append(). Function tcl_list_append() takes two parameters: a list and a value to append. After the call, the value is owned by the list, and you should therefore not free the item.

When done, the entire list is freed with tcl_free(), just like values.

/* List values */
struct tcl_value *tcl_list_new();
struct tcl_value *tcl_list_append(struct tcl_value *value, struct tcl_value *tail);
struct tcl_value *tcl_list_item(struct tcl_value *value, int index);
int tcl_list_length(struct tcl_value *value);

In the default implementation lists are implemented as raw strings that add some escaping (braces) around each item (lists thus resemble Tcl source code). It's a simple solution that also reduces the code, but in some exotic cases the escaping can go wrong and invalid results will be returned.

Variables

A variable holds a value. Or in the case of an array, a variable holds multiple values. You create a variable with tcl_var(). Currently, you should do this before tcl_eval() (and after tcl_init()). This creates a global variable. When you create C functions that you register to ParTcl as commands, and you would create a variable in that function (when it is called), that variable will be a local variable.

When you create a variable, you pass a value as its content. After the call, the value is owned by the variable. You must not free the value.

You also use tcl_var() to have it return its value. Note that the returned value may not be modified and should not be cached (as a pointer). You can, of course, make a local copy of the value.

There is currently no function to delete a single variable. Calling tcl_destroy() removes all variables (and you should then call tcl_init() again).

Language syntax

Tcl script is made up of commands separated by semicolons or newline symbols. Commands in their turn are made up of fields separated by whitespace. A word is a field, as long as it does not contain a space character. To make whitespace a part of the field one may use double quotes or curly braces.

An important part of the language is command substitution, when the result of a command inside square braces is returned as a part of the outer command, e.g. puts [expr 1 + 2]. Apart from command substitution, Tcl also knows variable substitution. However, command and variable substitution does not happen inside fields that are enveloped in curly braces.

The only data type of the language is a string. When a variable (or field) contains only digits, it may be implicitly interpreted as a number (specifically in the expr builtin command), but that same variable can still be used in string operations.

Builtin commands

The table below is a brief summary of the built-in commands in ParTcl. See the "Tcl Primer" (or other documentation on Tcl) for details on these commands.

Operator table

These are the operators that can be used in the parameter of the expr command. Note that some operators only work on integers.

Integer arithmetic is used when all operands of an operator have integer type. So notably, "7 / 2" result in 3 (integer division); for floating point division use "7.0 / 2" (or "7 / 2.0" or "7.0 / 2.0"). Also, integer division in Tcl (and thus ParTcl) is floored integer division, in the sense that the remainder after division is always a positive value, and the following relation holds:

 v2 * (v1 / v2) + (v1 % v2) == v1

For positive numerators & denominators, floored division gives the same results as the (more common) truncated division: the division 14 / 3 is 4 (and with remainder 2). The difference is with negative results: with truncated division -14 / 3 equals -4 with remainder -2, but with floored division, -14 / 3 equals -5 with remainder 1.

Functions

The expr command supports the following functions:

Predefined variables

Mostly for the use of the trigonometric functions, the following variables are predefined:

Internals

Lexer

Any symbol can be part of the word, except for the following special symbols:

• whitespace, tab - used to delimit words
• \r, \n, semicolon or EOS (zero-terminator) - used to delimit commands
• Braces, square brackets, dollar sign - used for substitution and grouping

ParTcl has special helper functions for these char classes:

static bool tcl_is_space(char c);
static bool tcl_is_end(char c);
static bool tcl_is_special(char c, bool quote);

tcl_is_special behaves differently depending on the quoting mode (quote parameter). Inside a quoted string braces, semicolon and end-of-line symbols lose their special meaning and become regular printable characters.

ParTcl lexer is implemented in one function:

int tcl_next(const char *list, size_t length, const char **from, const char **to, unsigned *flags);

tcl_next function finds the next token in the string list. Paramters from and to are set to point to the token start/end. Parameter flags holds flags for the quoting mode (toggled if " is met), and marks whether a comment is allowed (and must be gobbled), plus other(s). When calling the lexer from your own code, flags should be initialized to 0.

A special macro tcl_each(s, len, skip_error) can used to iterate over all the tokens in the string. If skip_error is false - loop ends when string ends, otherwise loop can end earlier if a syntax error is found. It allows to "validate" input string without evaluating it and detect when a full command has been read.

Memory management

Tcl uses strings as a primary data type. When a Tcl script is evaluated, many of the strings are created, disposed or modified. In embedded systems, memory management can be complex, so all operations with Tcl values are moved into isolated functions that can be rewritten to optimize certain parts (e.g. to use a pool of strings, a custom memory allocator, cache numerical or list values to increase performance etc).

/* Functions calling malloc() or free() */
struct tcl_value *tcl_value(const char *data, size_t len);
bool tcl_append(struct tcl_value *value, struct tcl_value *tail);
bool tcl_list_append(struct tcl_value *list, struct tcl_value *tail);
struct tcl_value *tcl_free(struct tcl_value *value);
struct tcl_value *tcl_var(struct tcl *tcl, const char *name, struct tcl_value *value);
struct tcl_cmd *tcl_register(struct tcl *tcl, const char *name, tcl_cmd_fn_t fn,
 unsigned short minargs, unsigned short maxargs,
 struct tcl_value *user);
void tcl_destroy(struct tcl *tcl);
/* Internal functions also doing memory allocation */
static struct tcl_scope *tcl_scope_alloc(struct tcl_scope *parent);
static struct tcl_var *tcl_scope_var(struct tcl_scope *scope, const char *name);
static struct tcl_scope *tcl_scope_free(struct tcl_scope *scope);
static void tcl_var_free_values(struct tcl_var *var);
static void tcl_var_free(struct tcl_scope *scope, struct tcl_var *var);
static int tcl_error_result(struct tcl *tcl, int code, const char *info);
/* Built-in commands */
static int tcl_cmd_array(struct tcl *tcl, struct tcl_value *args, void *arg);
static int tcl_cmd_binary(struct tcl *tcl, struct tcl_value *args, void *arg);
static int tcl_cmd_format(struct tcl *tcl, struct tcl_value *args, void *arg);
static int tcl_cmd_read(struct tcl *tcl, struct tcl_value *args, void *arg);

Scopes

A special type, struct tcl_scope is used to keep the evaluation scope, which holds to local variables at each call nesting level. The interpreter creates a new scope each time a user-defined procedure is called; Also there is one global scope per interpreter.

There are only 3 functions related to the scopes. One creates a new scope, another seeks for a variable (or creates a new one), the last one destroys the scope and all the variables in it.

These functions use malloc()/free(), but can easily be rewritten to use memory pools instead.

static struct tcl_scope *tcl_scope_alloc(struct tcl_scope *parent);
static struct tcl_var *tcl_scope_var(struct tcl_scope *scope, tcl_value_t *name);
static struct tcl_scope *tcl_scope_free(struct tcl_scope *scope);

Variables are implemented as a single-linked list, each variable is a pair of values (name + value) and a pointer to the next variable.

Interpreter

ParTcl interpreter is a simple structure struct tcl which keeps the current scope (see above), an array of available commands, and a "last result" value.

Interpreter logic is wrapped around two functions - evaluation and substitution.

Substitution:

• If argument starts with $ - evaluate the name that follows and return the variable's value. If the variable does not exist, an empty variable is created.
• If argument starts with [ - evaluate what's inside the square brackets and return the result.
• If argument is a quoted string (e.g. {foo bar}) - return it as is, just without braces.
• Otherwise return the argument as is.

Evaluation:

• Iterates over each token in a list
• Appends words into a list
• If the command end is met (semicolon, or newline, or end-of-file - our lexer has a special token type TCMD for them) - then find a suitable command (the first word in the list) and call it.

Where the commands are taken from? Initially, a ParTcl interpeter starts with no commands, but one may add the commands by calling tcl_register().

Each command has a name, arity (how many arguments it shall take - interpreter checks it before calling the command, use zero arity for varargs) and a C function pointer that actually implements the command.

Building and testing

All sources are in one file, tcl.c. It can be used as a standalone interpreter, or made part of a bigger application. The structure declarations and function prototypes are in tcl.h.

Tests are run with clang and coverage is calculated. Just run "make test" and you're done.

Code is formatted using clang-format to keep the clean and readable coding style. Please run it for pull requests, too.

License

Code is distributed under MIT license, feel free to use it in your proprietary projects as well.