-
Notifications
You must be signed in to change notification settings - Fork 131
Coupling of Tokens
and Scanners
#462
-
I was hoping to add a feature to TokenParsers
that allows users to provide a string that represents a single token and get a parser that recognizes that token (this sort of interaction comes essentially for free in a scannerless parser); however, I was only able to make it work in my own code because my TokenParsers.lexical
member was a Scanners
object with Tokens
mixed in. It doesn't work in general because of the separation of Tokens
from Scanners
(which is what StdLexical
does).
Since the whole point of TokenParsers
is to read tokens, it seems logical to me that Tokens
should be combined into Scanners
instead of decoupled; after all, Tokens
doesn't provide an interface for accessing tokens, so it can't stand alone. The current design seems to assume that the authors of the TokenParsers
subclass will know how TokenParsers.lexical
is intended to be used, presumably because the same authors will have written it. Incorporating Tokens
into Scanners
and requiring TokenParsers.lexical
to be a Scanners
object would provide a defined interface that would allow for features like the one I described above.
I'm guessing, however, that this has been considered and that the authors of this library decoupled the two classes for a reason. If you're willing, I would love to better understand the rationale behind this design choice.
(My agenda here is that I would like for this library to be a viable option for my undergraduate compilers students. A few improvements, mostly to documentation, will suffice. I'm happy to contribute code and docs, but I want to make sure I understand before I write much of either.)
Beta Was this translation helpful? Give feedback.