Should we allow quasiquotation of homogeneous vectors from SRFIs 4 and 160? They would look like #f64(1.1 ,x), where x must be bound to an inexact number.
No.
What would the syntax object representation be? To support this it must be possible for an @vector to store and retrieve values of at least any datum type other than @, which would defeat their objective
@dpk wrote in #312 (comment):
To support this it must be possible for an
@vectorto store and retrieve values of at least any datum type
Can you explain why?
An @vector syntax object need not be a numeric @vector. It is perfectly possible (and sensible) that any syntax datum can be put into a @vector syntax object. Only when that syntax object is read as an expression by the evaluator, it would be an error if its stripped version contained a non-number.
So what @johnwcowan wants (and users probably expect to be supported) can work.
@johnwcowan wrote in #312 (comment):
To support this it must be possible for an
@vectorto store and retrieve values of at least any datum typeCan you explain why?
Consider a vector with quasiquote `#(1 2 3 ,(+ 1 3)), in many implementations, it would be read as (quasiquote #(1 2 3 (unquote (+ 1 3)))).
The part #(1 2 3 (unquote (+ 1 3))), if we treat it independently, is a valid Scheme object and also a valid datum.
However, if we write things like `#vu8(1 2 3 ,(+ 1 3)), #vu8(1 2 3 (unquote (+ 1 3))) is not a valid object. Every sane Scheme implementation will reject it at read time.
Copyright: IIRC, this example was given by @dpk in IRC.
@readevalprintloop wrote in #312 (comment):
@johnwcowan wrote in #312 (Kommentar):
To support this it must be possible for an
@vectorto store and retrieve values of at least any datum typeCan you explain why?
Consider a vector with quasiquote
`#(1 2 3 ,(+ 1 3)), in many implementations, it would be read as(quasiquote #(1 2 3 (unquote (+ 1 3)))).The part
#(1 2 3 (unquote (+ 1 3))), if we treat it independently, is a valid Scheme object and also a valid datum.However, if we write things like
`#vu8(1 2 3 ,(+ 1 3)),#vu8(1 2 3 (unquote (+ 1 3)))is not a valid object. Every sane Scheme implementation will reject it at read time.
No, not necessarily. See my previous post. One lesson R6RS told us is that the hygienic macro model of Scheme forces us to distinguish between datum objects and syntax objects.
There is no datum value represented by the example you gave. But it makes sense to have a syntax object type to represent it.
@mnw wrote in #312 (comment):
@readevalprintloop wrote in #312 (评论):
@johnwcowan wrote in #312 (Kommentar):
To support this it must be possible for an
@vectorto store and retrieve values of at least any datum typeCan you explain why?
Consider a vector with quasiquote
`#(1 2 3 ,(+ 1 3)), in many implementations, it would be read as(quasiquote #(1 2 3 (unquote (+ 1 3)))).
The part#(1 2 3 (unquote (+ 1 3))), if we treat it independently, is a valid Scheme object and also a valid datum.
However, if we write things like`#vu8(1 2 3 ,(+ 1 3)),#vu8(1 2 3 (unquote (+ 1 3)))is not a valid object. Every sane Scheme implementation will reject it at read time.No, not necessarily. See my previous post. One lesson R6RS told us is that the hygienic macro model of Scheme forces us to distinguish between datum objects and syntax objects.
There is no datum value represented by the example you gave. But it makes sense to have a syntax object type to represent it.
But your solution seems to imply that there's another procedure read-syntax, which is different from regular read and can be used to read an homogeneous vector with unquote or unquote-splicing inside (e.g. `#u8(1 2 ,(+ 1 2))) as a syntax object.
In my view, this could lead following issues:
read. We may even unable to create such syntax objects at runtime.eval. Though in the program that uses eval does not, in principle, require such construct. But it does harm the meta-circularity of the language.syntax->datum on it?R6RS introduce a conceptual type "syntax object". However, these syntax objects can always derived from data.
read-syntax. Ordinary read does not collect line number information, for example, and when evaluating/compiling a Scheme program, users expect source location information. It makes sense to standardise an interface to such a procedure.syntax form can be used to create syntax at runtime, including quasi-homogeneous vectors.eval interface is partly from an ancient time when Scheme had no hygienic vectors and when datums represented Scheme expressions. eval-syntax should accompany read-syntax.syntax->datum would raise a violation. But that's okay. It is already a lossy conversion. In some sense, identifier->symbol is all that is really needed. Every other use of syntax->datum can be arguably solved differently in a more elegant way.@mnw wrote in #312 (comment):
- Any good Scheme system has a procedure that may be called
read-syntax. Ordinaryreaddoes not collect line number information, for example, and when evaluating/compiling a Scheme program, users expect source location information.
Chez does and so does Racket, but I think they are the only ones.
syntax->datum is one thing, but what does unwrap-syntax (syntax-e) return? That is needed to implement syntax-case, and for syntax-case to be able to match within a numeric vector to implement quasiquote, which is the point here.
@johnwcowan wrote in #312 (comment):
@mnw wrote in #312 (Kommentar):
- Any good Scheme system has a procedure that may be called
read-syntax. Ordinaryreaddoes not collect line number information, for example, and when evaluating/compiling a Scheme program, users expect source location information.Chez does and so does Racket, but I think they are the only ones.
Internally, most Schemes likely have such a thing. The difference is that Chez and Racket export it under non-standardised names. In any case, it is useful to have for anyone wanting to read Scheme lexical syntax and be able to report source locations, for example.
@dpk wrote in #312 (comment):
syntax->datumis one thing, but what doesunwrap-syntax(syntax-e) return? That is needed to implementsyntax-case, and forsyntax-caseto be able to match within a numeric vector to implement quasiquote, which is the point here.
Indeed, syntax-case is the tool provided to destruct syntax objects. A syntax object is like an object of an algebraic data type, and syntax-case is the corresponding matcher/destructor.
syntax-case can be implemented in various ways. If you want to be able to implement it as library syntax with something like unwrap-syntax, you will have to add a type that unwrap-syntax returns as a container for vector-like types (such a container can consist of a tag-field together with element-fields). But this is not particularly important for the user of the language, only for writers of the standard library. Exposing something like unwrap-syntax became an idea during R7RS-large (and R6RS works very well without it). It has its value for academic discussions (and for the appendix to R4RS), but neither is it needed for nor should it dictate the surface language a user of R7RS sees.
I should make it clear that what I said applies to that when a new vector lexical syntax is added to the language, no fundamental reason speaks against quasi-quotation in source code, and users of the language can rightly expect that it should work.
However, I don't think that any such new lexical syntax should be added to the language. People always praise Scheme for having such a simple syntax, while, in fact, the actual lexical syntax has become much more complicated than the basic syntax of s-exprs. Indeed, there is already no fundamental reason why we need special syntax for vectors (or bytevectors starting with R6RS). String syntax is needed, but for a very particular reason: No one wants to write strings in source code as a list of characters.
One of the benefits of an entity being having lexical syntax is that it is a datum and can be passed to and returned from syntax transformers (a syntax transformer can manipulate a string to implement string interpolation, for example).
Syntax transformers could be extended to return non-datums (or even to all types), but specifying that would be trickier. And you'd have a "syntax" object with no lexical syntax.
@phm wrote in #312 (comment):
One of the benefits of an entity being having lexical syntax is that it is a datum and can be passed to and returned from syntax transformers (a syntax transformer can manipulate a string to implement string interpolation, for example).
Why should there be a problem encoding some object (like a multidimensional array) as a syntax object (that roughly corresponds to a syntax object for an s-expr) for communication between syntax transformers? They already have to do such an encoding/decoding for record types/hashtables/whatever.
@mnw wrote in #312 (comment):
Why should there be a problem encoding some object (like a multidimensional array) as a syntax object (that roughly corresponds to a syntax object for an s-expr) for communication between syntax transformers? They already have to do such an encoding/decoding for record types/hashtables/whatever.
This argument can be used to advocate for getting rid of record types entirely. Why not just always operate on trees of cons cells?
Serializing a non-datum compound type into a macro transformer means you a) have to pay the cost to serialize/desrialize an object constantly to operate on it, or b) redefine all of your operations on the serialized representation. Do I want to define inheritable subtyping predicates on serialized records?
@phm wrote in #312 (comment):
@mnw wrote in #312 (Kommentar):
Why should there be a problem encoding some object (like a multidimensional array) as a syntax object (that roughly corresponds to a syntax object for an s-expr) for communication between syntax transformers? They already have to do such an encoding/decoding for record types/hashtables/whatever.
This argument can be used to advocate for getting rid of record types entirely. Why not just always operate on trees of cons cells?
No. Record types are currently not supported as syntax objects, nor do they have read/write invariance. Yet, they have great use in actual code that is executed at runtime. You wouldn't gain much by allowing record types in syntax objects (or being able to read them).
Serializing a non-datum compound type into a macro transformer means you a) have to pay the cost to serialize/desrialize an object constantly to operate on it, or b) redefine all of your operations on the serialized representation. Do I want to define inheritable subtyping predicates on serialized records?
At runtime, no cost is paid at all. Syntax transformer procedures that do lots of work by calling helper procedures, whatever, ... also do not have to serialise/deserialise. The only case when this has to be done is when two macro invocations by the expander have to interact with each other. This is not a bottleneck at all.
For AOT expansion of libraries, it is important that syntax objects can be serialised, so the output of a syntax transformer (as part of a macro invocation) must not be arbitrary data, anyway.
unwrap-syntax is the wrong abstraction when the goal is to find fundamental procedures/syntax for syntax objects. Syntax objects make up an algebraic data type, so the fundamental procedure/syntax for it is a matcher (if we want syntax) or a suitable fold (if we want procedures). unwrap-syntax is just a partial converter and not as fundamental. It is a different question whether the fundamental matcher should be as full-blown as syntax-case or whether it should be something basic.I had to come up with a practical example of @dpk & @readevalprintloop's point. I think it captures why this is a weird idea in practice. Here it is: If I read
`#u16(,(+ 1 0))
then I get a list (quasiquote <something>) where <something> is, I guess, an invalid u16-vector.* You could extract that weird u16-vector with cadr and pass it to things that expect u16-vectors, which is not so good.
How do you deal with this? Is read supposed to allow this syntax and "pass the buck" on error detection? Then how do you prevent people from creating heterogeneous homogeneous vector-things by this kind of read hack?
*: It's invalid, of course, because the first element is (unquote (+ 1 0)).
There is nothing weird about it. The correct theoretical model behind this is that a procedure like read-syntax is the fundamental procedure. Its task is to turn a textual representation into a Scheme program (or fragment thereof), and as we know, since the R4RS-macro appendix, Scheme programs are no longer datums but syntax objects. The procedure read is conceptually the composition of read-syntax and syntax->datum. Its main use is to read data files encoded in Scheme datum syntax, but when used to read Scheme source code, it only gives an approximation.
As the example given does not represent a datum value, syntax->datum will signal an error and read will propagate this error. And that's completely fine and shouldn't come unexpected. Already in existing standards, syntax objects on one hand and datum values on the other hand do not form isomorphic families (think of identifiers).
To give you another example: if you use (quote #u16(,(+ 1 0))) in your code, the implementation's reader (read-syntax) won't have any problem with this expression and the expander of quote will be given a syntax object representing the example you gave. It will be that expander that signals an error because it will conceptually call syntax->datum by definition of quote.
I saw @johnwcowan wondering how Common Lisp handling the unquote{,-splicing} in #S (struct) and other lexical syntax in IRC. I think it's better to discuss it here instead of IRC.
IIUC, Hyperspec doesn't require a backquote form must be interpreted as a S-exp representation. Quoting from Hyperspec
An implementation is free to interpret a backquoted form F1 as any form F2 that, when evaluated, will produce a result that is the same under equal as the result implied by the above definition, provided that the side-effect behavior of the substitute form F2 is also consistent with the description given above. The constructed copy of the template might or might not share list structure with the template itself. As an example, the above definition implies that
`((,a b) ,c ,@d)
will be interpreted as if it were
(append (list (append (list a) (list 'b) 'nil)) (list c) d 'nil)
but it could also be legitimately interpreted to mean any of the following:
(append (list (append (list a) (list 'b))) (list c) d)
(append (list (append (list a) '(b))) (list c) d)
(list* (cons a '(b)) c d)
(list* (cons a (list 'b)) c d)
(append (list (cons a '(b))) (list c) d)
(list* (cons a '(b)) c (copy-list d))
Therefore, CL implementations may freely extend their backquote to support interpolation in other lexical syntax, without worrying introducing malformed syntax object.
This stands in stark contrast to the wording in R6RS/R7RS-Small, which states:
'
<datum> ,<datum> ,@<datum> #'<datum> #
#,
#,@
Each of these is an abbreviation:
' for (quote ),
<datum> for (quasiquote <datum>), ,<datum> for (unquote <datum>), ,@<datum> for (unquote-splicing <datum>), #'<datum> for (syntax <datum>), #for (quasisyntax ),
#, for (unsyntax ), and
#,@ for (unsyntax-splicing
R7RS 4.2.8
The two notations `〈qq template〉 and (quasiquote
〈qq template〉) are identical in all respects. ,〈expression〉
is identical to (unquote 〈expression〉), and ,@〈expression〉
is identical to (unquote-splicing 〈expression〉). The
write procedure may output either format.
@readevalprintloop wrote in #312 (comment):
Therefore, CL implementations may freely extend their backquote to support interpolation in other lexical syntax, without worrying introducing malformed syntax object.
This is the behavior of TR7 (an R7RS implementation):
(define x 5)
(define rest '(100 200))
'`#u8(1 2 ,x ,@rest) ; => (apply bytevector `(1 2 ,x ,@rest))
(eval '`#u8(1 2 ,x ,@rest)) ; => #u8(1 2 5 100 200)
For TR7 behaviour, I swear that it is accidental. Just for fun I checked a more complex expression:
'`(list x #u8(1 2 ,x ,@rest) #(1 ,@rest x))
; => `(list x ,(apply bytevector `(1 2 ,x ,@rest)) ,(apply vector `(1 ,@rest x)))
The transformation is made during lexical read. At lexical level occurencies #( and #u8 appearing inside a quasi-quote are replaced by (unquote (apply ?vector (quasi-quote ...
Trying to perform the transformation during lexical reading is doomed to fail.
@mnw wrote in #312 (comment):
Trying to perform the transformation during lexical reading is doomed to fail.
I agree. Introducing special construct to handle manifest vector or bytevector as syntaxic constructs was a longer path in TR7. That explains but not excuses.
No due date set.
No dependencies set.
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?