Include with Extensible Reader

Abstract

Reader macros in the style of Common Lisp cause phasing and hygiene problems, which stem from destructively modifing the current reader. The reader macro system proposed here allows for different readers to be used with the include form, obviating the phasing and hygiene issues.

Rationale

Scheme has no standard ability to change its reader, for good reason. The implementation needs to read the code that defines the custom reader before the reader is defined or imported into a library, causing phasing problems. The reader code and the read code must be kept separate.

If the two are in separate source files, then the reader can open the other file, parse its contents, and insert the contents into a file using a macro. This is precisely what include in the R7RS does. The include expression form in the R7RS (and that can be implemented using syntax-case) suffers from hygiene issues, and the include form in define-library cannot be customized. This SRFI fixes the hygiene issues of include and makes the include form more general.

There is interest in having some way of modifying the reader. There have been proposals for limited reader macros and general reader modification in the R7RS-Large. Guile, CHICKEN, Sagittarius, and Racket have support for modifying the reader similar to how Common Lisp does it. Racket has include forms that allow for custom readers. At the time of writing, there are 20 SRFIs labeled "Reader Syntax". Reader macros allows for more expressive DSLs and language experimentation.

All implementations of the R7RS support include, and relative inclusion is almost universal across all implementations of the R7RS. Almost all Scheme source files are stored in a hierarchical filesystem, and it is common practice to split up libraries into a library declaration part and a source code part.

This proposal allows for the power of reader macros without actually changing the syntax of Scheme. Other proposals would radically change what a Scheme source file means, and could end up with a Scheme source file being anything that starts with #!lang or some similar construct. Here, Scheme source files still conform to the description as given in the reports, and extensions to the Scheme syntax are really parsers that transform text into Scheme syntax objects.

Possible applications of include are string interpolation, custom syntaxes such as SRFI 119, and readtable based interfaces. These are left for libraries or future proposals.

Specification

On implementations of the R6RS, the identifiers defined here are exported from (srfi srfi-XXX) and (srfi srfi-XXX include). On implementations of the R7RS, the identifiers defined here are exported from (srfi XXX).

Locations

A location is a string that describes where a file is located. A relative location describes a file relative to some other location. Conceptually, there exists a function location-append such that given location A and relative location B, (location-append A B) is another relative location.

A location A is valid if open-input-file executes without error for A. A location A is equivalent to a location B if acting with any of the file operations on both A and B are equivalent. Location A can be navigated to from a location B if there exists a relative location S such that (location-append B S) is equivalent to A.

The set of locations is partitioned into disjoint sets, where each partition is a set of locations that can be navigated to from one location. Not all locations in any of those sets are valid locations.

Each Scheme source has a non-relative location, which may be a location with no other accessable relative locations (it lives in a partition where the only valid location is itself).

A string that

1. does not start with / or CHARS: where CHARS is a sequence of one or more alphanumeric characters
2. does not contain ASCII or C1 escape sequences except space, or non-characters

is a relative location.

Examples of relative locations:

• 1.scm
• 1/lists.scm
• :1/lists.scm

Any other string may be a relative location, a non-relative location, or not a location. This is implementation defined. In particular, an implementation could make all strings relative locations.

Possible examples of non-relative locations:

• C:\Program Files\Scheme\lib\code.scm
• /usr/share/scheme/lib/code.scm
• https://example.com/lib/code.scm

Fundamental Macros

(include (identifier location)) ; macro

• identifier must be an identifier.
• location must be a location.

If location is a relative location, then the implementation will look for a file named location relative to where the including file's location. If location is a non-relative location, then the implementation will look for a file in an implementation defined manner.

The file will be opened and will be read by the reader in effect for the file. The default reader is the standard read exported from the appropriate library, except it may raise exceptions in situations that are not defined for code, such as circular code.

The introduced identifiers will have the same syntax wrap as that of identifier.

Example:

(include '"lib/code.scm")

Note the use of the quote. That line is equivalent to (include (quote "lib/code.scm"). The quote supplies an identifier with the correct syntax environment.

(make-includer reader) ; procedure

This procedure will make a syntax transformer, suitable for passing as the body of define-syntax or the right hand side of a local syntax binding.

The reader is a procedure of one argument, a textual port. It must return a Scheme datum, an end-of-file object when it is done parsing, or raise a non-continuable exception. It is an error if the dynamic extent of any call to reader made by the macro transformer is re-entered after it has exited.

The returned macro transformer will act like include, but will only use the defined reader to parse the input file.

(make-string-includer reader) ; procedure

This procedure will make a syntax transformer, suitable for passing as the body of define-syntax or the right hand side of a local syntax binding.

The reader procedure is the same as described in make-includer.

The returned syntax transformer will have the same form as include, but will take an arbitrary string that is readable by reader instead of a location. The transformer will act like make-includer, except that it will read the passed string as if it was a file with the same location as the including file.

Derived Macros

(include-r6rs (identifier location)) ; syntax
(r6rs-reader textual-port) ; procedure

Read a source file defined according to the R6RS, as if #!r6rs was prepended to the source.

(include-r7rs (identifier location)) ; syntax
(r7rs-reader textual-port) ; procedure

Read a source file defined according to the R7RS.

(include-r7rs-ci (identifier location)) ; syntax
(r7rs-ci-reader textual-port) ; procedure

Read a source file defined according to the R7RS, as if #!fold-case was prepended to the source.

The R7RS Library Syntax

Extensions to the R7RS library syntax are not supported because one of the features of the R7RS's library syntax is its ability to be reasoned about statically.

To use these macros with R7RS libraries, one could do:

(define-library (my library)
 (import (scheme base) (srfi NNN))
 (begin (include-r6rs '"my-r6rs-file.scm")))

This means that the include-library-declarations form cannot be expressed in non-R7RS syntaxes.

A future standard or proposal should introduce an extensible library syntax that fixes the limitations of the R6RS and R7RS library systems.

Implementation

TODO

The R7RS specific portions of this specification are not portably implementable.

The R6RS can implement portions of this SRFI that do not encompass the R7RS, with the exception of the requirement that the include form be relative to the directory of inclusion.

Acknowledgements

I thank Mark Nieper-Wißkirchen for informing me of Racket's solution to this problem, and to the Racket Project for coming up with the ideas that I have mostly copied.

I also thank John Cowan and Wolfgang Corcoran-Mathe for discussions about other reader macro proposals.

Copyright

© 2026 Peter McGoron.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice (including the next paragraph) shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.