status of cyclic dependency of library imports #277

jobol commented 2025年10月10日 15:23:47 +02:00

It should be specified that cyclic dependencies of libraries is allowed.

In GUILE, when you use use-modules import statement, it accept without complaining cyclic dependencies. But still with GUILE when using import and R7RS layer, it gets stuck!

After thinking about the issue, I believe that cyclic dependencies of imports must be allowed.

I looked in R7RS and R6RS specifications and did not find clear evidence that cyclic dependencies of import is allowed. That is why I am opening that issue, for getting a clear statement of this requirement in future specification.

It should be specified that cyclic dependencies of libraries is allowed. In GUILE, when you use `use-modules` import statement, it accept without complaining cyclic dependencies. But still with GUILE when using `import` and R7RS layer, it gets stuck! After thinking about the issue, I believe that cyclic dependencies of imports must be allowed. I looked in R7RS and R6RS specifications and did not find clear evidence that cyclic dependencies of import is allowed. That is why I am opening that issue, for getting a clear statement of this requirement in future specification.

dpk commented 2025年10月10日 15:40:36 +02:00

@jobol wrote in #277 (comment):

I looked in R7RS and R6RS specifications and did find clear evidence that cyclic dependencies of import is allowed.

Where?

@jobol wrote in https://codeberg.org/scheme/r7rs/issues/277#issue-2505835: > I looked in R7RS and R6RS specifications and did find clear evidence that cyclic dependencies of import is allowed. Where?

mnw commented 2025年10月10日 15:51:05 +02:00

Cyclic imports are not allowed in R6RS. The referenced library must be instantiated before the referencing library is instantiated.

I agree that ruling out cyclic dependencies can sometimes be limiting. In Unsyntax, I had to introduce parameter objects to break an otherwise large-scale cyclic dependency. (The other way out would have been to create one monolithic library.)

If we want to allow cyclic dependencies, we will have to state that the order of visiting or instantiation is undetermined for every cycle. Explicit phasing would have a slight advantage here because it can possibly restrict cycles (and thus this indeterminacy) to certain phases.

A different possibility would be to adopt Chez's module syntax (also implemented in Unsyntax), which is much more flexible than the static library system imposed upon us since R6RS.

Cyclic imports are not allowed in R6RS. The referenced library must be instantiated before the referencing library is instantiated. I agree that ruling out cyclic dependencies can sometimes be limiting. In Unsyntax, I had to introduce parameter objects to break an otherwise large-scale cyclic dependency. (The other way out would have been to create one monolithic library.) If we want to allow cyclic dependencies, we will have to state that the order of visiting or instantiation is undetermined for every cycle. Explicit phasing would have a slight advantage here because it can possibly restrict cycles (and thus this indeterminacy) to certain phases. A different possibility would be to adopt Chez's `module` syntax (also implemented in Unsyntax), which is much more flexible than the static library system imposed upon us since R6RS.

dpk commented 2025年10月10日 16:11:17 +02:00

@mnw wrote in #277 (comment):

Cyclic imports are not allowed in R6RS. The referenced library must be instantiated before the referencing library is instantiated.

My instinct is to leave this as it is, maybe with a more explicit note that cyclical dependencies are not allowed.

A different possibility would be to adopt Chez's module syntax (also implemented in Unsyntax), which is much more flexible than the static library system imposed upon us since R6RS.

This is already noted in the Needed SRFIs list.

@mnw wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7662898: > Cyclic imports are not allowed in R6RS. The referenced library must be instantiated before the referencing library is instantiated. My instinct is to leave this as it is, maybe with a more explicit note that cyclical dependencies are not allowed. > A different possibility would be to adopt Chez's `module` syntax (also implemented in Unsyntax), which is much more flexible than the static library system imposed upon us since R6RS. This is already noted in the Needed SRFIs list.

dpk commented 2025年10月10日 16:17:49 +02:00

Also #120

Also #120

jobol commented 2025年10月12日 10:53:38 +02:00

@dpk wrote in #277 (comment):

@jobol wrote in #277 (commentaire):

I looked in R7RS and R6RS specifications and did find clear evidence that cyclic dependencies of import is allowed.

Where?

I wanted write did NOT find. Fixed now.

@dpk wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7662712: > @jobol wrote in #277 (commentaire): > > > I looked in R7RS and R6RS specifications and did find clear evidence that cyclic dependencies of import is allowed. > > Where? I wanted write `did NOT find`. Fixed now.

readevalprintloop commented 2025年10月17日 14:39:55 +02:00

Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents).

If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking ¹ all of the components into a single binary.

Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior.

(BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only define-module can have cyclic dependencies while library cannot.)

Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents). If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking [^1] all of the components into a single binary. Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior. (BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only `define-module` can have cyclic dependencies while `library` cannot.) [^1]: https://eli.thegreenplace.net/2013/07/09/library-order-in-static-linking

mnw commented 2025年10月17日 14:56:34 +02:00

@readevalprintloop wrote in #277 (comment):

Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents).

If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking 1 all of the components into a single binary.

Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior.

(BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only define-module can have cyclic dependencies while library cannot.)

1. https://eli.thegreenplace.net/2013/07/09/library-order-in-static-linking [↩︎](#fnref:user-content-1)

Please read my earlier post. It is not "implementation-specific" whether library dependencies in R[67]RS can be circular or not. Circular library dependencies are forbidden because they are inconsistent with the rest of the spec. For example, here is the relevant part of R7RS:

When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times.

Guile cannot allow circular dependencies for RnRS libraries and implement their semantics consistently.

(I don't understand the argument about static linking; first, it is an issue of the C tool chain, and Scheme libraries don't necessarily correspond to C object files/library archives. Secondly, Eli actually shows that the problem becomes a non-issue when giving the linker the correct command-line arguments.)

@readevalprintloop wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7773404: > Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents). > > If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking [1](#fn:user-content-1) all of the components into a single binary. > > Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior. > > (BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only `define-module` can have cyclic dependencies while `library` cannot.) > > 1. https://eli.thegreenplace.net/2013/07/09/library-order-in-static-linking [↩︎](#fnref:user-content-1) Please read my earlier post. It is not "implementation-specific" whether library dependencies in R[67]RS can be circular or not. Circular library dependencies are forbidden because they are inconsistent with the rest of the spec. For example, here is the relevant part of R7RS: > When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times. Guile cannot allow circular dependencies for RnRS libraries and implement their semantics consistently. (I don't understand the argument about static linking; first, it is an issue of the C tool chain, and Scheme libraries don't necessarily correspond to C object files/library archives. Secondly, Eli actually shows that the problem becomes a non-issue when giving the linker the correct command-line arguments.)

readevalprintloop commented 2025年10月17日 15:34:12 +02:00

@mnw wrote in #277 (comment):

@readevalprintloop wrote in #277 (评论):

Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents).
If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking 1 all of the components into a single binary.
Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior.
(BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only define-module can have cyclic dependencies while library cannot.)

1. https://eli.thegreenplace.net/2013/07/09/library-order-in-static-linking [↩︎](#fnref:user-content-1)

Please read my earlier post. It is not "implementation-specific" whether library dependencies in R[67]RS can be circular or not. Circular library dependencies are forbidden because they are inconsistent with the rest of the spec. For example, here is the relevant part of R7RS:

Thanks for corrections, it seems that some of my arguments are based on false impression.

I agree with that the need of cyclic dependencies should be fulfilled by other facilities like Chez style modules, or implementation-specific library declaration like define-module.

@mnw wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7774097: > @readevalprintloop wrote in #277 (评论): > > > Although not officially certificated, but the details of library not specified in R6RS/small language can be regarded as "implementation-specified" (that depends on the implementation and that each implementation documents). > > If we place the arguments for and against at opposite ends of the scale, they are almost balanced. Guix is an example that heavily use cyclic import, so packagers can freely organize their package definitions and didn't need to care about their dependents, on the other side, cyclic depdencies does harm for implementations aimed at statically linking [1](#fn:user-content-1) all of the components into a single binary. > > Thus, IMO large language should stay neutral on this, maybe explicit pointing out that is an implementation specified behavior. > > (BTW, I suggest you to file a bug report about Guile doesn't allow cyclic dependencies while using RnRS library declarations, it's inconsistent that only `define-module` can have cyclic dependencies while `library` cannot.) > > ``` > > 1. https://eli.thegreenplace.net/2013/07/09/library-order-in-static-linking [↩︎](#fnref:user-content-1) > > ``` > > Please read my earlier post. It is not "implementation-specific" whether library dependencies in R[67]RS can be circular or not. Circular library dependencies are forbidden because they are inconsistent with the rest of the spec. For example, here is the relevant part of R7RS: Thanks for corrections, it seems that some of my arguments are based on false impression. I agree with that the need of cyclic dependencies should be fulfilled by other facilities like Chez style `modules`, or implementation-specific library declaration like `define-module`.

mnw commented 2025年10月17日 15:51:43 +02:00

To make it work, one could also introduce a concept of deferred import sets. If a library is imported through a deferred import set, initialisation of that library is deferred until after the importing library is initialised. It is an error to evaluate uninitialised variables/transformer procedures.

Example: Library A imports library B and B imports library A through a deferred import set. That breaks the cycle, and the order of initialisation is B, then A.

Existing Scheme systems can implement this by transforming the bodies of A and B into equivalent letrec forms and then composing the clauses of the forms into a single letrec where the clauses of B precede those of A.

To make it work, one could also introduce a concept of deferred import sets. If a library is imported through a deferred import set, initialisation of that library is deferred until after the importing library is initialised. It is an error to evaluate uninitialised variables/transformer procedures. Example: Library `A` imports library `B` and `B` imports library `A` through a deferred import set. That breaks the cycle, and the order of initialisation is `B`, then `A`. Existing Scheme systems can implement this by transforming the bodies of `A` and `B` into equivalent `letrec` forms and then composing the clauses of the forms into a single `letrec` where the clauses of `B` precede those of `A`.

themaniacschemer commented 2025年10月17日 16:19:42 +02:00

Speaking of Guile's module system, can we standardize it as an SRFI or something?

Guile's module system, from my exposure to it, seems to be the most dynamic and flexible, almost Common Lisp-like, and from my survey of other Schemes it seems that it's almost the ideal module system for an implementation targeting image-style interactive programming.

I can't imagine a reimplementation of Emacs in Scheme, where you can easily redefine procedures from other modules (think the core of Emacs's functions for text editing) and even access the private procedures of a certain module as you can in Emacs (which has no modules, funnily enough), using the R6RS library system. In contrast, such an attempt is perfectly possible using Common Lisp, thanks to its package system (see Lem). A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs.

Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, eval is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support?

For reference, Ypsilon Scheme achieves this feature by introducing an LLVM-based runtime that natively, incrementally compiles expressions at runtime. Perhaps the Ypsilon runtime which executes Scheme files could be bundled with (compiled) source files to produce a final executable? Just a suggestion for how to implement this feature.

Thank you for your time.

Speaking of Guile's module system, can we standardize it as an SRFI or something? Guile's module system, from my exposure to it, seems to be the most dynamic and flexible, almost Common Lisp-like, and from my survey of other Schemes it seems that it's almost the ideal module system for an implementation targeting image-style interactive programming. I can't imagine a reimplementation of Emacs in Scheme, where you can easily redefine procedures from other modules (think the core of Emacs's functions for text editing) and even access the private procedures of a certain module as you can in Emacs (which has no modules, funnily enough), using the R6RS library system. In contrast, such an attempt is perfectly possible using Common Lisp, thanks to its package system (see [Lem](https://github.com/lem-project/lem)). A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs. Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, `eval` is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support? For reference, Ypsilon Scheme achieves this feature by introducing an LLVM-based runtime that natively, incrementally compiles expressions at runtime. Perhaps the Ypsilon runtime which executes Scheme files could be bundled with (compiled) source files to produce a final executable? Just a suggestion for how to implement this feature. Thank you for your time.

readevalprintloop commented 2025年10月17日 16:55:19 +02:00

@themaniacschemer wrote in #277 (comment):

Speaking of Guile's module system, can we standardize it as an SRFI or something?

Yeah, everyone can write SRFI, and SRFI editors should not reject SRFI because they think it is a wrong-headed approach to the problem. IMO, the issue of attempts to standardized Guile's module is, it's scope too large to reach consensuse.

I think a Racket-style Unit might be a better compromise can be standardized as a SRFI for those expect module to have dynamic linking behavior.

A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs.

I belive that R6RS like library doesn't prevent you from doing module reflections, but I hadn't actually dive into this.

Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, eval is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support?

There's only eval, no compile (unlike our Common Lisp cousin) in previous Scheme standard. I think it's intended to avoid over-specifying (R6RS implementation may not rely on a compiler, see the pure interpreter mode of Chez, and IIRC there's also "pure interpreter mode" for SBCL). And in a past working group email message, @mnw explained that eval can be treat as a kind of "the JIT compiler in Scheme"

@themaniacschemer wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7775249: > Speaking of Guile's module system, can we standardize it as an SRFI or something? Yeah, everyone can write SRFI, and SRFI editors should not reject SRFI because they think it is a wrong-headed approach to the problem. IMO, the issue of attempts to standardized Guile's module is, it's scope too large to reach consensuse. I think a [Racket-style Unit](https://docs.racket-lang.org/guide/units.html) might be a better compromise can be standardized as a SRFI for those expect module to have dynamic linking behavior. > A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs. I belive that R6RS like library doesn't prevent you from doing module reflections, but I hadn't actually dive into this. > > Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, `eval` is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support? There's only `eval`, no `compile` (unlike our Common Lisp cousin) in previous Scheme standard. I think it's intended to avoid over-specifying (R6RS implementation may not rely on a compiler, see the pure interpreter mode of Chez, and IIRC there's also "pure interpreter mode" for SBCL). And in a past working group email message, @mnw explained that `eval` can be treat as a kind of "the JIT compiler in Scheme"

dpk commented 2025年10月17日 17:24:03 +02:00

@themaniacschemer wrote in #277 (comment):

Speaking of Guile's module system, can we standardize it as an SRFI or something?

1. We have two library syntaxes already, which already one too many, but that’s what history has given us

2. Guile supports both of the RnRS syntaxes already. If my proposal for Guile 4.0 is accepted (and the reaction from Guile users and maintainers is positive so far), it will even offer much more natural support for them out of the box, rather than requiring activation to look in the standard paths used by other Scheme implementations for RnRS libraries

3. Guile’s ‘Common Lisp’-like model, as you call it, is broadly felt by the Guile maintainers to be a mistake. Guile has added declarative modules to get away from its original idea that a module is a mutable hash table; RnRS library declarations have this quality already.

4. The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot). This also means that ‘making the compiler available at run time’ as a standard report feature is a non-starter. We have eval but it might not be anything better than a tree-walking interpreter in such implementations.

5. Guile modules also, by default, don’t start from an empty environment but bring along a large implementation-specific standard library unless this is turned off. I believe Guile’s maintainers now also consider this to be a mistake, because the standard library can’t grow without risking breaking existing modules by defining the same identifier they already use.

6. Guile’s module syntax depends on Guile-specific lexical syntax (for keywords) which, as far as we have reached consensus so far, will not be in the report.

7. Even if we were to get rid of the semantic problems here by supporting only the syntax of Guile’s declarations (including the lexical syntax for keywords), and using the semantics of existing RnRS libraries – even apart from the language bloat of adding a third syntax for the same underlying semantics, it’s just not compelling to support something that doesn’t add any new capabilities to Scheme. We could have new features that aren’t problematic for static implementations, like Racket’s units/Scheme 48’s ML-like structures with their functors (although these introduce phasing nightmares, which is why they aren’t Racket’s standard module system but rather a second level embedded one), or Chez’s modules as @mnw suggests (which would let you generate a whole module from a macro, for example). We could add support for a customizable library-level language, as has also been suggested before. But I’m not even convinced that those would be worth it,* though, considering how complex the library system of R7RS already will be.

(* Since I know you like to take any opportunity where I say I’m not sure about something to try to convince me, @mnw, perhaps put that effort right now into making a SRFI for Chez-style modules instead, so that when we come to discussing the library system in about a year from now, we can have a much more informed discussion about it then :-) )

@themaniacschemer wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7775249: > Speaking of Guile's module system, can we standardize it as an SRFI or something? 1. We have two library syntaxes already, which already one too many, but that’s what history has given us 2. Guile supports both of the RnRS syntaxes already. If my proposal for Guile 4.0 is accepted (and the reaction from Guile users and maintainers is positive so far), it will even offer much more natural support for them out of the box, rather than requiring activation to look in the standard paths used by other Scheme implementations for RnRS libraries 3. Guile’s ‘Common Lisp’-like model, as you call it, is broadly felt by the Guile maintainers to be a mistake. Guile has added declarative modules to get away from its original idea that a module is a mutable hash table; RnRS library declarations have this quality already. 4. The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot). This also means that ‘making the compiler available at run time’ as a standard report feature is a non-starter. We have `eval` but it might not be anything better than a tree-walking interpreter in such implementations. 5. Guile modules also, by default, don’t start from an empty environment but bring along a large implementation-specific standard library unless this is turned off. I believe Guile’s maintainers now also consider this to be a mistake, because the standard library can’t grow without risking breaking existing modules by defining the same identifier they already use. 6. Guile’s module syntax depends on Guile-specific lexical syntax (for keywords) which, as far as we have reached consensus so far, will not be in the report. 7. Even if we were to get rid of the semantic problems here by supporting only the syntax of Guile’s declarations (including the lexical syntax for keywords), and using the semantics of existing RnRS libraries – even apart from the language bloat of adding a third syntax for the same underlying semantics, it’s just not compelling to support something that doesn’t add any new capabilities to Scheme. We could have new features that aren’t problematic for static implementations, like Racket’s units/Scheme 48’s ML-like structures with their functors (although these introduce phasing nightmares, which is why they aren’t Racket’s standard module system but rather a second level embedded one), or Chez’s modules as @mnw suggests (which would let you generate a whole module from a macro, for example). We could add support for a customizable library-level language, as has also been suggested before. But I’m not even convinced that those would be worth it,* though, considering how complex the library system of R7RS already will be. (* Since I know you like to take any opportunity where I say I’m not sure about something to try to convince me, @mnw, perhaps put that effort right now into making a SRFI for Chez-style modules instead, so that when we come to discussing the library system in about a year from now, we can have a much more informed discussion about it then :-) )

dpk commented 2025年10月17日 17:31:20 +02:00

Oh, you should maybe also watch my talk at the Scheme Workshop yesterday, in which I talk a bit about the tension between adopting features like this one from Common Lisp/Guile vs making a report which works for many implementation styles. (Thanks to @readevalprintloop for finding the right timestamp in the long recording of the whole Workshop.)

To be clear, Guile’s Common Lisp-like idea of how modules work, including the ability for runtime reflection, is marvellous for doing the things Guile itself is intended for. Even as it moves towards declarative modules, it still enables some pretty amazing things. I wouldn’t have been able to make extensible-match work on Guile without the ability to inspect other modules’ structure, for example. But not all implementations will be able to offer this.

Oh, you should maybe also [watch my talk at the Scheme Workshop yesterday,](https://www.youtube.com/live/Zl-M3avRPXI?si=hb651QDhylQBlakt&t=25780) in which I talk a bit about the tension between adopting features like this one from Common Lisp/Guile vs making a report which works for many implementation styles. (Thanks to @readevalprintloop for finding the right timestamp in the long recording of the whole Workshop.) To be clear, Guile’s Common Lisp-like idea of how modules work, including the ability for runtime reflection, is marvellous for doing the things Guile itself is intended for. Even as it moves towards declarative modules, it still enables some pretty amazing things. I wouldn’t have been able to make [extensible-match](https://codeberg.org/dpk/extensible-match) work on Guile without the ability to inspect other modules’ structure, for example. But not all implementations will be able to offer this.

themaniacschemer commented 2025年10月17日 18:00:55 +02:00

The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot).

For an implementation like Cyclone, which compiles to C and is statically linked at compile-time (just like C programs), the compiler at runtime model does indeed conflict with this implementation strategy and priorities. I anticipated that the Common Lisp model wouldn't work for some implementations. My hope was that if an implementation tried to adopt such a model, it would follow an already specified version, so that for implementations that support it, they could be compatible. This includes Guile's module system and perhaps first-class environments from MIT Scheme. If you deem it worth it for standardization.

The main feature from Common Lisp (specifically SBCL) to be standardized, I believe, is something like a generate-image procedure that produces a statically linked and complete executable with the runtime and all hitherto loaded source files (where this executable can later be loaded with little startup time, since it's all already compiled and perhaps optimized). This procedure could even be used as the standard API to compile a Scheme program, by loading its source files at the top-level, if one wishes to use the implementation itself for the build process.

This is all, of course, an optional feature that I propose to be considered for standardization. Programs that rely on aggressive optimization will of course favor whole-program optimization, which doesn't play well with such a dynamic model of modules (which the Guile manual states).

I bring this all up because I wish to see the Scheme ecosystem have its own way of supporting the Common Lisp model. Instead of being drawn to Common Lisp for this one feature, for a use case like Lem.

Also, I want to be clear that I am not interested in the specific syntax of Guile's module system. I just am interested in the semantic features. A library system that (optionally) could do what Guile can do would be enough for me.

> The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot). For an implementation like Cyclone, which compiles to C and is statically linked at compile-time (just like C programs), the compiler at runtime model does indeed conflict with this implementation strategy and priorities. I anticipated that the Common Lisp model wouldn't work for some implementations. My hope was that if an implementation tried to adopt such a model, it would follow an already specified version, so that for implementations that support it, they could be compatible. This includes Guile's module system and perhaps first-class environments from MIT Scheme. If you deem it worth it for standardization. The main feature from Common Lisp (specifically SBCL) to be standardized, I believe, is something like a `generate-image` procedure that produces a statically linked and complete executable with the runtime and all hitherto loaded source files (where this executable can later be loaded with little startup time, since it's all already compiled and perhaps optimized). This procedure could even be used as the standard API to compile a Scheme program, by loading its source files at the top-level, if one wishes to use the implementation itself for the build process. This is all, of course, an optional feature that I propose to be considered for standardization. Programs that rely on aggressive optimization will of course favor whole-program optimization, which doesn't play well with such a dynamic model of modules (which the Guile manual states). I bring this all up because I wish to see the Scheme ecosystem have its own way of supporting the Common Lisp model. Instead of being drawn to Common Lisp for this one feature, for a use case like Lem. Also, I want to be clear that I am not interested in the specific syntax of Guile's module system. I just am interested in the semantic features. A library system that (optionally) could do what Guile can do would be enough for me.

dpk commented 2025年10月17日 18:36:16 +02:00

@themaniacschemer wrote in #277 (comment):

The main feature from Common Lisp (specifically SBCL) to be standardized, I believe, is something like a generate-image procedure that produces a statically linked and complete executable with the runtime and all hitherto loaded source files (where this executable can later be loaded with little startup time, since it's all already compiled and perhaps optimized). This procedure could even be used as the standard API to compile a Scheme program, by loading its source files at the top-level, if one wishes to use the implementation itself for the build process.

This could indeed be a viable SRFI along the lines of SRFI 138. (I am a bit doubtful that enough implementations support this exact model for it to be worthwhile, but that is another issue.) But, like SRFI 138, it is probably out of scope for the Scheme report itself, and almost certainly out of scope for the normative part.

@themaniacschemer wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7776644: > The main feature from Common Lisp (specifically SBCL) to be standardized, I believe, is something like a `generate-image` procedure that produces a statically linked and complete executable with the runtime and all hitherto loaded source files (where this executable can later be loaded with little startup time, since it's all already compiled and perhaps optimized). This procedure could even be used as the standard API to compile a Scheme program, by loading its source files at the top-level, if one wishes to use the implementation itself for the build process. This could indeed be a viable SRFI along the lines of SRFI 138. (I am a bit doubtful that enough implementations support this exact model for it to be worthwhile, but that is another issue.) But, like SRFI 138, it is probably out of scope for the Scheme report itself, and almost certainly out of scope for the normative part.

themaniacschemer commented 2025年10月17日 18:46:13 +02:00

@dpk wrote in #277 (comment):

This could indeed be a viable SRFI along the lines of SRFI 138. (I am a bit doubtful that enough implementations support this exact model for it to be worthwhile, but that is another issue.) But, like SRFI 138, it is probably out of scope for the Scheme report itself, and almost certainly out of scope for the normative part.

I take this to mean that my idea of the Common Lisp model and related features (like dynamic modules and images) have no place in the standard (I mean to say, they need not be addressed explicitly or taken into consideration in the specification of Scheme's semantics and features), and they should rather remain in the realm of SRFIs?

@dpk wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7777604: > This could indeed be a viable SRFI along the lines of SRFI 138. (I am a bit doubtful that enough implementations support this exact model for it to be worthwhile, but that is another issue.) But, like SRFI 138, it is probably out of scope for the Scheme report itself, and almost certainly out of scope for the normative part. I take this to mean that my idea of the Common Lisp model and related features (like dynamic modules and images) have no place in the standard (I mean to say, they need not be addressed explicitly or taken into consideration in the specification of Scheme's semantics and features), and they should rather remain in the realm of SRFIs?

jobol commented 2025年10月17日 21:26:41 +02:00

@mnw wrote in #277 (comment):

For example, here is the relevant part of R7RS:

When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times.

Telling that circular dependencies is forbidden is an interpretation of the words.

Here it is said that:

• the order of evaluation of expression within a library is respected
• the library is loaded (not evaluated) before evaluation of its dependencies
• it is not required to load the same library an infinite count of times

So nothing forbids circular dependencies.

@mnw wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7774097: > For example, here is the relevant part of R7RS: > > > When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times. Telling that circular dependencies is forbidden is an interpretation of the words. Here it is said that: * the order of evaluation of expression _within_ a library is respected * the library is loaded (not evaluated) before evaluation of its dependencies * it is not required to load the same library an infinite count of times So nothing forbids circular dependencies.

mnw commented 2025年10月18日 09:32:26 +02:00

@jobol wrote in #277 (comment):

@mnw wrote in #277 (Kommentar):

For example, here is the relevant part of R7RS:

When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times.

Telling that circular dependencies is forbidden is an interpretation of the words.

Here it is said that:

* the order of evaluation of expression _within_ a library is respected
* the library is loaded (not evaluated) before evaluation of its dependencies
* it is not required to load the same library an infinite count of times

So nothing forbids circular dependencies.

Loading of a library means that its expressions (right-hand sides of top-level defines and the top-level commands) are executed (meaning they are evaluated, possibly already in their expanded form). As a library has to be loaded before any dependent library is loaded, and "before" implies a (partial) order, you cannot have a dependency graph with cycles.

R6RS makes no difference. Here is one relevant quote:

If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n.

(Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.)

@jobol wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7779599: > @mnw wrote in #277 (Kommentar): > > > For example, here is the relevant part of R7RS: > > > When a library is loaded, its expressions are executed in textual order. If a library’s definitions are referenced in the expanded form of a program or library body, then that library must be loaded before the expanded program or library body is evaluated. This rule applies transitively. If a library is imported by more than one program or library, it may possibly be loaded additional times. > > Telling that circular dependencies is forbidden is an interpretation of the words. > > Here it is said that: > > * the order of evaluation of expression _within_ a library is respected > > * the library is loaded (not evaluated) before evaluation of its dependencies > > * it is not required to load the same library an infinite count of times > > > So nothing forbids circular dependencies. Loading of a library means that its expressions (right-hand sides of top-level defines and the top-level commands) are executed (meaning they are evaluated, possibly already in their expanded form). As a library has to be loaded before any dependent library is loaded, and "before" implies a (partial) order, you cannot have a dependency graph with cycles. R6RS makes no difference. Here is one relevant quote: > If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n. (Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.)

jobol commented 2025年10月18日 10:03:44 +02:00

@mnw wrote in #277 (comment):

(Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.)

That is great and I still think that standard is ambiguous enough to allow circular dependencies.

R6RS makes no difference. Here is one relevant quote:

If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n.

As long as there is no real execution of code, instanciation can deal correctly with circular dependencies.

I would not say that circular dependency is common but splitting a large program in separate component often lead to mutual dependencies.

Using the R7RS definition of library is compatible with circular dependencies. I agree that R7RS small implementations are free to deal with it or not but I'm in favor of requiring it for larger language.

@mnw wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7784696: > (Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.) That is great and I still think that standard is ambiguous enough to allow circular dependencies. > R6RS makes no difference. Here is one relevant quote: > >> If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n. As long as there is no real execution of code, instanciation can deal correctly with circular dependencies. I would not say that circular dependency is common but splitting a large program in separate component often lead to mutual dependencies. Using the R7RS definition of library is compatible with circular dependencies. I agree that R7RS small implementations are free to deal with it or not but I'm in favor of requiring it for larger language.

mnw commented 2025年10月18日 13:20:16 +02:00

@jobol wrote in #277 (comment):

@mnw wrote in #277 (Kommentar):

(Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.)

That is great and I still think that standard is ambiguous enough to allow circular dependencies.

The latter cannot be; the existing standards (R6RS, R7RS small) would be inconsistent if they allowed circular dependencies. (And this is not just some theoretical inconsistency; you simply cannot implement circular dependencies and be true to what the standard demands.

R6RS makes no difference. Here is one relevant quote:

If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n.

As long as there is no real execution of code, instanciation can deal correctly with circular dependencies.

I think this is where your misunderstanding lies. "Instantiation" means evaluating code. Again, quoting from R6RS:

An instance of a library corresponds to an evaluation of its variable definitions and expressions in a particular phase relative to another library—a process called instantiation.

R7RS (small), which doesn't have the concept of phases, calls this loading (which also corresponds to evaluation of code, see my earlier quotation from R7RS).

I would not say that circular dependency is common but splitting a large program in separate component often lead to mutual dependencies.

I agree with you here. But sole wishful thinking does not change what's in a standard. What you can do is to lobby for R7RS large to amend the library system to allow circular dependencies, or to lobby for R7RS small to be revised to allow the same.

Using the R7RS definition of library is compatible with circular dependencies. I agree that R7RS small implementations are free to deal with it or not but I'm in favor of requiring it for larger language.

Consider the following libraries:

(define-library (symbol-table)
 (export getprop setprop)
 (import (scheme base))
 (begin
 (define table '())
 (define (getprop sym) (let ((e (assq table sym)))) (and e (cdr e))))
 (define (setprop sym val) (set! table (cons (cons sym val) table))))

(define-library (somelib)
 (export get-answer)
 (import (scheme base) (symbol-table))
 (begin
 (setprop 'foo 42)
 (define (get-answer) (getprop 'foo))))

Here, the standards guarantee that symbol-table is instantiated/loaded (its top-level definitions and commands evaluated) before somelib is instantiated/loaded. If that were not the case, the use of (symbol-table) by (somelib) would be broken. Even more, it would not be possible for the initialization code of any library to use bindings exported by other libraries if the order of initialization were not guaranteed.

Of course, for R7RS small (in contrast to R6RS), there is always the loophole that an implementation is allowed to do anything (undefined behaviour) in an error situation, for example, if there is a circular dependency. So an implementation is consistent with R7RS small if it maintains the prescribed initialization order when there is no circular dependency. In case of some circular dependency anywhere in the program, however, the user has no guarantee that the implementation is doing something sensible or something the user would expect.

@jobol wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7784864: > @mnw wrote in #277 (Kommentar): > > > (Remember that I don't say one shouldn't consider allowing circular references, but for that, one has to amend either standard.) > > That is great and I still think that standard is ambiguous enough to allow circular dependencies. The latter cannot be; the existing standards (R6RS, R7RS small) would be inconsistent if they allowed circular dependencies. (And this is not just some theoretical inconsistency; you simply cannot implement circular dependencies and be true to what the standard demands. > > R6RS makes no difference. Here is one relevant quote: > > > If the expanded form of one library references at phase 0 an identifier from another library, then before the referencing library is instantiated at phase n, the referenced library must be instantiated at phase n. > > As long as there is no real execution of code, instanciation can deal correctly with circular dependencies. I think this is where your misunderstanding lies. "Instantiation" means evaluating code. Again, quoting from R6RS: > An instance of a library corresponds to an evaluation of its variable definitions and expressions in a particular phase relative to another library—a process called instantiation. R7RS (small), which doesn't have the concept of phases, calls this loading (which also corresponds to evaluation of code, see my earlier quotation from R7RS). > I would not say that circular dependency is common but splitting a large program in separate component often lead to mutual dependencies. I agree with you here. But sole wishful thinking does not change what's in a standard. What you can do is to lobby for R7RS large to amend the library system to allow circular dependencies, or to lobby for R7RS small to be revised to allow the same. > Using the R7RS definition of library is compatible with circular dependencies. I agree that R7RS small implementations are free to deal with it or not but I'm in favor of requiring it for larger language. Consider the following libraries: ```scheme (define-library (symbol-table) (export getprop setprop) (import (scheme base)) (begin (define table '()) (define (getprop sym) (let ((e (assq table sym)))) (and e (cdr e)))) (define (setprop sym val) (set! table (cons (cons sym val) table)))) ``` ```scheme (define-library (somelib) (export get-answer) (import (scheme base) (symbol-table)) (begin (setprop 'foo 42) (define (get-answer) (getprop 'foo)))) ``` Here, the standards guarantee that `symbol-table` is instantiated/loaded (its top-level definitions and commands evaluated) before `somelib` is instantiated/loaded. If that were not the case, the use of `(symbol-table)` by `(somelib)` would be broken. Even more, it would not be possible for the initialization code of any library to use bindings exported by other libraries if the order of initialization were not guaranteed. Of course, for R7RS small (in contrast to R6RS), there is always the loophole that an implementation is allowed to do anything (undefined behaviour) in an error situation, for example, if there is a circular dependency. So an implementation is consistent with R7RS small if it maintains the prescribed initialization order when there is no circular dependency. In case of some circular dependency anywhere in the program, however, the user has no guarantee that the implementation is doing something sensible or something the user would expect.

mnw commented 2025年10月18日 13:58:50 +02:00

@themaniacschemer wrote in #277 (comment):

Speaking of Guile's module system, can we standardize it as an SRFI or something?

Guile's module system, from my exposure to it, seems to be the most dynamic and flexible, almost Common Lisp-like, and from my survey of other Schemes it seems that it's almost the ideal module system for an implementation targeting image-style interactive programming.

I can't imagine a reimplementation of Emacs in Scheme, where you can easily redefine procedures from other modules (think the core of Emacs's functions for text editing) and even access the private procedures of a certain module as you can in Emacs (which has no modules, funnily enough), using the R6RS library system. In contrast, such an attempt is perfectly possible using Common Lisp, thanks to its package system (see Lem). A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs.

Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, eval is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support?

For reference, Ypsilon Scheme achieves this feature by introducing an LLVM-based runtime that natively, incrementally compiles expressions at runtime. Perhaps the Ypsilon runtime which executes Scheme files could be bundled with (compiled) source files to produce a final executable? Just a suggestion for how to implement this feature.

Thank you for your time.

It seems to me that R7RS could have become a language like the one you have in mind (and R7RS-large a language like Common Lisp but with guaranteed tail calls, hygienic macros, and a single binding space).

Let me quote people who voted against R6RS (and were responsible for starting the R7RS project as an alternative to R6RS):

Marc Feeley:

One of Scheme's strengths is its dynamic nature and in particular the ability to be used interactively through a REPL. The draft [R6RS] moves away from this by removing ``load'' and offering only a static linking model for programs. The draft does not explain how this affects program development and debugging using a REPL, which is the bread-and-butter of a large number of Scheme users.

The draft puts too much emphasis on performance. Is R5RS Scheme so slow that we need to introduce into the standard immutable variables, fields, and pairs? I am all for speed, but not at the expense of inconsistencies in the language design. Immutability hinders debugging and ``live-repair'', and goes against the basic design principles of Scheme:

The syntax for libraries is just too complex. [...] This is another unfortunate instance of feature creep. Scheme needs a simple module system that can be grasped easily. [...]

Will Clinger:

The R6RS has given us the option of writing programs in the new and more static language it describes. Two implementations of the dynamic language most recently described by the R5RS have added support for this new mode of execution, and four brand new implementations of R6 Scheme are well under way.

That represents a partial success, but this process has disappointed users of the language formerly known as Scheme. Many had hoped that this process would give them standard facilities for defining libraries and records while offering at least some
common direction on matters such as Unicode, IEEE floating-point arithmetic, and exception handling.
Instead, the R6RS made those facilities available only to programmers who adopt the new language.

If you read posts from that time, you will find a lot of people who put these hopes into the R7RS language. If you ask me, the coexistence of R6RS and R7RS would have made complete sense if that vision had come true. However, as things have developed under WG2 and are being steered, R7RS-large seems to become an R6RS at its heart. (And that's why I think it would be better to see R6RS developed further, and make R7RS a true alternative, instead of making it an R6RS with superficial R7RS elements. But I seem to be alone in this opinion in the WG.)

@themaniacschemer wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7775249: > Speaking of Guile's module system, can we standardize it as an SRFI or something? > > Guile's module system, from my exposure to it, seems to be the most dynamic and flexible, almost Common Lisp-like, and from my survey of other Schemes it seems that it's almost the ideal module system for an implementation targeting image-style interactive programming. > > I can't imagine a reimplementation of Emacs in Scheme, where you can easily redefine procedures from other modules (think the core of Emacs's functions for text editing) and even access the private procedures of a certain module as you can in Emacs (which has no modules, funnily enough), using the R6RS library system. In contrast, such an attempt is perfectly possible using Common Lisp, thanks to its package system (see [Lem](https://github.com/lem-project/lem)). A reimplementation effort of an Emacs-like editor in Scheme, using only R6RS Scheme, would mean just a static program that can't be hacked on as you can with GNU Emacs. > > Also, speaking of Emacs, I would like to politely bring to your attention the problem of making sure the compiler is available at runtime for Scheme implementations. For an implementation like SBCL and CCL, `eval` is implemented using the compiler, which is available at runtime. For a Schemey Emacs-like, user extensions should ideally also be compiled. This entails the compiler being available at runtime (because recompiling the editor with extensions is out of the question), which usually isn't the case for Scheme. How should this be handled with current Schemes, and should this be treated in the specification? Perhaps an optional feature some Schemes can support? > > For reference, Ypsilon Scheme achieves this feature by introducing an LLVM-based runtime that natively, incrementally compiles expressions at runtime. Perhaps the Ypsilon runtime which executes Scheme files could be bundled with (compiled) source files to produce a final executable? Just a suggestion for how to implement this feature. > > Thank you for your time. It seems to me that R7RS could have become a language like the one you have in mind (and R7RS-large a language like Common Lisp but with guaranteed tail calls, hygienic macros, and a single binding space). Let me quote people who voted against R6RS (and were responsible for starting the R7RS project as an alternative to R6RS): _Marc Feeley:_ > One of Scheme's strengths is its dynamic nature and in particular the ability to be used interactively through a REPL. The draft [R6RS] moves away from this by removing ``load'' and offering only a static linking model for programs. The draft does not explain how this affects program development and debugging using a REPL, which is the bread-and-butter of a large number of Scheme users. > The draft puts too much emphasis on performance. Is R5RS Scheme so slow that we need to introduce into the standard immutable variables, fields, and pairs? I am all for speed, but not at the expense of inconsistencies in the language design. Immutability hinders debugging and ``live-repair'', and goes against the basic design principles of Scheme: > The syntax for libraries is just too complex. [...] This is another unfortunate instance of feature creep. Scheme needs a simple module system that can be grasped easily. [...] _Will Clinger:_ > The R6RS has given us the option of writing programs in the new and more static language it describes. Two implementations of the dynamic language most recently described by the R5RS have added support for this new mode of execution, and four brand new implementations of R6 Scheme are well under way. > That represents a partial success, but this process has disappointed users of the language formerly known as Scheme. Many had hoped that this process would give them standard facilities for defining libraries and records while offering at least some common direction on matters such as Unicode, IEEE floating-point arithmetic, and exception handling. Instead, the R6RS made those facilities available only to programmers who adopt the new language. If you read posts from that time, you will find a lot of people who put these hopes into the R7RS language. If you ask me, the coexistence of R6RS and R7RS would have made complete sense if that vision had come true. However, as things have developed under WG2 and are being steered, R7RS-large seems to become an R6RS at its heart. (And that's why I think it would be better to see R6RS developed further, and make R7RS a true alternative, instead of making it an R6RS with superficial R7RS elements. But I seem to be alone in this opinion in the WG.)

dpk commented 2025年10月18日 14:20:19 +02:00

I would say that the fact WG1 decided to adapt basically the same model as R6RS for R7RS small, with programs distinct from ‘the REPL’ (even if they have to pretend to be the REPL in one way) and libraries as fundamentally static entities, that had the result you describe. If WG1 had wanted to go that way, they would have created a report that had the kind of Guile-style library system @themaniacschemer described, or they might have expanded on the system R5RS seemed to intend to create with its first-class environments.

But the benefit of the R6RS/R7RS small model is that it prescribes a static minimum to which dynamic features can be added by implementations which want to go that way. Guile is in fact an example of this: R6RS libraries loaded into Guile can be interacted with just like Guile’s own, dynamic modules.

I would say that the fact WG1 decided to adapt basically the same model as R6RS for R7RS small, with programs distinct from ‘the REPL’ (even if they have to pretend to be the REPL in one way) and libraries as fundamentally static entities, that had the result you describe. If WG1 had wanted to go that way, they would have created a report that had the kind of Guile-style library system @themaniacschemer described, or they might have expanded on the system R5RS seemed to intend to create with its first-class environments. But the benefit of the R6RS/R7RS small model is that it prescribes a static minimum to which dynamic features can be added by implementations which want to go that way. Guile is in fact an example of this: R6RS libraries loaded into Guile can be interacted with just like Guile’s own, dynamic modules.

mnw commented 2025年10月18日 14:34:36 +02:00

I should also add that you can add hot-patching to R6RS-style libraries (although only in cooperation with the library author). Consider:

(define-syntax define-patchable
 (syntax-rules ()
 ((_ id expr)
 (begin
 (define v (vector expr))
 (define-syntax id
 (identifier-syntax
 (_ (vector-ref v 0))
 ((set! _ e) (vector-set! v 0 e))))))))

A "variable" defined by (define-patchable VAR INIT) and exported by the library can be re-assigned at any time, thus providing a hook.

I should also add that you can add hot-patching to R6RS-style libraries (although only in cooperation with the library author). Consider: ```scheme (define-syntax define-patchable (syntax-rules () ((_ id expr) (begin (define v (vector expr)) (define-syntax id (identifier-syntax (_ (vector-ref v 0)) ((set! _ e) (vector-set! v 0 e)))))))) ``` A "variable" defined by `(define-patchable VAR INIT)` and exported by the library can be re-assigned at any time, thus providing a hook.

mnw commented 2025年10月18日 15:05:25 +02:00

@dpk wrote in #277 (comment):

I would say that the fact WG1 decided to adapt basically the same model as R6RS for R7RS small, with programs distinct from ‘the REPL’ (even if they have to pretend to be the REPL in one way) and libraries as fundamentally static entities, that had the result you describe. If WG1 had wanted to go that way, they would have created a report that had the kind of Guile-style library system @themaniacschemer described, or they might have expanded on the system R5RS seemed to intend to create with its first-class environments.

But the benefit of the R6RS/R7RS small model is that it prescribes a static minimum to which dynamic features can be added by implementations which want to go that way. Guile is in fact an example of this: R6RS libraries loaded into Guile can be interacted with just like Guile’s own, dynamic modules.

I agree with you that WG1 had already steered in that direction (if you ask me - personal opinion -, in fact, so much that it would have made more sense to standardise a subset of R6RS as the small language instead of breaking compatibility with the previous R6RS). However, R7RS-small is also indecisive at the same time. It offers the load procedure, but it doesn't offer a way to use it in (top-level) programs, only in REPL scripts and through eval.

NB If one reads old posts on the WG1 mailing list, one sees that Aaron W. Hsu proposed a system like Chez's modules, which is more local and doesn't build on the R6RS system, which seems to be influenced mostly by PLT Scheme. The dynamic nature and the REPL and load semantics, etc., could have been fully preserved with the module system (Chez itself had module before adopting R6RS's library).

@dpk wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7787564: > I would say that the fact WG1 decided to adapt basically the same model as R6RS for R7RS small, with programs distinct from ‘the REPL’ (even if they have to pretend to be the REPL in one way) and libraries as fundamentally static entities, that had the result you describe. If WG1 had wanted to go that way, they would have created a report that had the kind of Guile-style library system @themaniacschemer described, or they might have expanded on the system R5RS seemed to intend to create with its first-class environments. > > But the benefit of the R6RS/R7RS small model is that it prescribes a static minimum to which dynamic features can be added by implementations which want to go that way. Guile is in fact an example of this: R6RS libraries loaded into Guile can be interacted with just like Guile’s own, dynamic modules. I agree with you that WG1 had already steered in that direction (if you ask me - personal opinion -, in fact, so much that it would have made more sense to standardise a subset of R6RS as the small language instead of breaking compatibility with the previous R6RS). However, R7RS-small is also indecisive at the same time. It offers the `load` procedure, but it doesn't offer a way to use it in (top-level) programs, only in REPL scripts and through `eval`. NB If one reads old posts on the WG1 mailing list, one sees that Aaron W. Hsu proposed a system like Chez's modules, which is more local and doesn't build on the R6RS system, which seems to be influenced mostly by PLT Scheme. The dynamic nature and the REPL and `load` semantics, etc., could have been fully preserved with the `module` system (Chez itself had `module` before adopting R6RS's `library`).

mnw commented 2025年10月18日 19:43:38 +02:00

@dpk wrote in #277 (comment):

4. The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot). This also means that ‘making the compiler available at run time’ as a standard report feature is a non-starter. We have `eval` but it might not be anything better than a tree-walking interpreter in such implementations.

What is an example of an environment where a compiler at runtime is not possible? (I am not speaking of embedded systems, which are likely more a target for the small and not for the large language.)

@dpk wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7776182: > 4. The ‘Common Lisp’-like model suits only implementations like Guile which load a live runtime incrementally. It’s not suited to implementations which do separate compilation of libraries nor whole-program compilation ahead of time (including, for example, Guile’s new sister implementation, Hoot). This also means that ‘making the compiler available at run time’ as a standard report feature is a non-starter. We have `eval` but it might not be anything better than a tree-walking interpreter in such implementations. What is an example of an environment where a compiler at runtime is not possible? (I am not speaking of embedded systems, which are likely more a target for the small and not for the large language.)

dpk commented 2025年10月18日 19:47:08 +02:00

It depends what you expect the compiler to do. If you want it to make bytecode, the only restriction is that including a whole compiler might unreasonably swell the object code size in an environment like the browser. If you want it to make native code, there are many platforms where that isn’t possible, including (I think) WebAssembly in the browser, the Nintendo Switch ...

It depends what you expect the compiler to do. If you want it to make bytecode, the only restriction is that including a whole compiler might unreasonably swell the object code size in an environment like the browser. If you want it to make native code, there are many platforms where that isn’t possible, including (I think) WebAssembly in the browser, the [Nintendo Switch](https://www.european-lisp-symposium.org/static/proceedings/2025.pdf#page=44) ...

mnw commented 2025年10月18日 20:05:31 +02:00

@dpk wrote in #277 (comment):

It depends what you expect the compiler to do. If you want it to make bytecode, the only restriction is that including a whole compiler might unreasonably swell the object code size in an environment like the browser.

This would only be the case if the library (scheme eval) or (rnrs eval) were loaded. But even then, it is customary to download code modules on demand in a browser environment, which can be done here as well.

If you want it to make native code, there are many platforms where that isn’t possible, including (I think) WebAssembly in the browser, the Nintendo Switch ...

A Wasm module can call JavaScript, and JavaScript can create Wasm modules, can't it? So you can instantiate new native code modules.

But even if there is an important platform where JIT compiling to native code is not possible (and I wouldn't call Nintendo Switch an important platform for R7RS-large), a Common Lisp-like model built on an imaginary save-image command would still work: You develop the program on the host, which is different from the target (obviously, because we assume that the target cannot produce native code). At some point, you call save-image on the host but with a flag indicating that the image is for the target. And the host will produce native code for the target (and not for itself) when dumping procedures and will arrange that the resulting image file can be loaded by the target.

@dpk wrote in https://codeberg.org/scheme/r7rs/issues/277#issuecomment-7791563: > It depends what you expect the compiler to do. If you want it to make bytecode, the only restriction is that including a whole compiler might unreasonably swell the object code size in an environment like the browser. This would only be the case if the library `(scheme eval)` or `(rnrs eval)` were loaded. But even then, it is customary to download code modules on demand in a browser environment, which can be done here as well. > If you want it to make native code, there are many platforms where that isn’t possible, including (I think) WebAssembly in the browser, the [Nintendo Switch](https://www.european-lisp-symposium.org/static/proceedings/2025.pdf#page=44) ... A Wasm module can call JavaScript, and JavaScript can create Wasm modules, can't it? So you can instantiate new native code modules. But even if there is an important platform where JIT compiling to native code is not possible (and I wouldn't call Nintendo Switch an important platform for R7RS-large), a Common Lisp-like model built on an imaginary `save-image` command would still work: You develop the program on the host, which is different from the target (obviously, because we assume that the target cannot produce native code). At some point, you call `save-image` on the host but with a flag indicating that the image is for the target. And the host will produce native code for the target (and not for itself) when dumping procedures and will arrange that the resulting image file can be loaded by the target.