Useless Factor

Yesterday didn't feel like much of an eventful day until the end. As I was taking the bus home, I realized there was no more high school, ever, for the rest of my life. It's amazing. I just have one final (Calc III) and then I graduate. Now, when you deride my opinions based on my age, you can't say "He's just a little high schooler." You have to say "He's just a little college student." The downside is, if you ever felt like talking about how amazing it is that "a high schooler can do that," (if I ever did anything amazing, which I didn't) it just doesn't work the same with a college student, because college students are generally recognized as full citizens who are capable of doing cool things.

Anyway, I've been inactive for the past week or so because I've had final projects to do. But here's the latest stuff I've been coming up with. Remember all that qualified naming code I posted here before? Well, I have a third version, which is probably much better. It's based on Slava's original idea for qualified naming. Instead of having a word like math or a prefix like << to mark what vocab something is in, what if you prepend it onto the word name itself? That is, what if you used math:+ instead of math: + or << math +>> The advantage of that, besides saving one or four characters, is that it is integrated into the language better. For example, you can do \ math:+ and all will work properly; the word + from the vocab math will be pushed on the stack. The other solutions don't do this right. So, here's the surprisingly short code needed to achieve this. Note that this code only works with the latest CVS because it uses the new module system, which treats vocabs differently.

USING: kernel sequences assocs parser vocabs namespaces ;

: define-qualified ( vocab-name -- )
 [ CHAR: : add ] keep vocab vocab-words
 [>r append r> ] assoc-map-with use get push ;


: QUALIFIED:
 scan define-qualified ; parsing

So the things I'm working on now are (a) Unicode, which should be ready for .90, or if not, .91, and (b) improvements on libs/xml. You probably thought libs/xml was done after so much work. Well, Matthew Willis is writing a Jabber client in Factor. To do that, he has to parse XML without exhausting the stream. Now, because the XML library is stream-based, it can handle streams just fine. The only problem is that, with the DOM-style view (which is reached using the words read-xml or xml-chunk, among others), the stream is exhausted. So, in their current state, it is unusable for this purpose. These structures are strict, not lazy (a path I may explore in the future, along the lines of HaXML, which I still have to read about).

The solution is to extend the pull-xml view. When I implemented it, I never thought it would be that useful, but now it turns out to be very useful for these incremental situations. To extend the DOM view to pull-xml, all I have to do is write a word which will pull the next DOM-view element--a tag, string, or other similar thing. Except, if the next element is a closing tag, I have to return that. The implementation of this is almost done, and when it is, Yuuki should be able to make the Jabber client relatively easily, without having to reinvent the wheel of the tree structure that XML follows.

In my previous take at qualified naming in Factor I used an awkward syntax where you might see << math +>> to access the word + in math. I was discussing this on #concatenative, the Factor IRC channel, and Slava proposed something like math-+ for qualified naming. This would need to be prefixed by something like QUALIFIED: math to bring all of those math words in, under a qualified name.

I thought about it, and realized that it's possible to have pretty syntax without all of that renaming. Instead, the syntax math: + could be used, where math: is a parsing word defined by QUALIFIED:. Think of it like a macro generating a macro, something possible in both Scheme and Lisp. Eager to stop what I was working on and do something useless, I wrote the following code, which implements this syntax. Again, it is amazing how easy this is to do in Factor. I don't know of another language that is so flexible yet structured with respect to naming.

Note that this qualified naming solution has the same problem as the previous one I discussed--it has the potential to break homoiconic syntax.

USING: kernel parser words hashtables assocs quotations sequences namespaces ;
IN: qualified

: qual-quot ( vocab-name -- quot )
 [ vocab scan swap at parsed ] curry ;

: qual-name ( vocab-name -- word-name )
 CHAR: : add ;

: qual-word ( vocab-name -- quot )
 [ qual-quot ] keep qual-name "" 
 [ swap define-compound ] keep
 dup t "parsing" set-word-prop ; 

: qual-vocab ( vocab-name -- vocab )
 [ qual-word ] keep qual-name associate ;

: define-qual ( vocab-name -- )
 qual-vocab use get push ;

: QUALIFIED: 
 scan define-qual ; parsing

For a while, I was getting off the mission of this blog--to do useless exercises in Factor which show how cool it is. Now, I think I'm getting back on track.

One problem that occasionally comes up in Factor is overlapping word names in different vocabularies. Factor's vocab system allows for these overlaps, and whichever vocab is used last has the definition that is used. For example, if you have the following code:

IN: foo
: x 1 ;
IN: bar
: x 2 ;

USING: foo bar ;
IN: baz
: y x ;

Then y would return 2 from bar's x, rather than 1 from foo's x. bar is used after foo, so it overrides foo's definitions. For this reason, it is always appropriate to put the IN: line after the USING: line. If IN: comes before, then the USE:'d vocabularies could potentially override the the vocabulary that you're actually in!

Inevitably (this may have already happened; I'm not sure) there will be a case of conflict so bad that names need to be changed. They would need be changed because they overlap, but both need to be used in the same third piece of code. But this is terrible for modularity. The authors of the first two pieces of code shouldn't have to care about each other, or about the ways they might be combined.

Most programming languages solve this using qualified names for modules. That is, instead of just writing x and hoping it comes from the right module that you've previously imported, you could write Foo.x or Bar.x. Until 15 minutes ago, this was impossible in Factor, but now you can, using the syntax << foo x>> or << bar x>>.

I thought this would be hard, but it was actually extremely easy. The entire code is below. It's so small, it doesn't even need to use kernel. It's great how easy Factor makes it to arbitrarily extend the parser for little hacks like this.

USING: parser sequences namespaces ;
IN: qualified

: use- ( -- )
 use get pop* ;

DEFER:>>

: <<
 scan use+ \>> parse-until
 [ parsed ] each use- ; parsing

The code is up in my darcs repository.

By the way, there's one major flaw with this code (as Slava will be quick to point out): it doesn't interact with the prettyprinter properly. The prettyprinter would need major changes to deal with printing something like [ << foo x>> << bar x>> ]. Probably, in that case, both xs should be printed as qualified. However, this occurs very rarely, and unless two words with the same name from different vocabularies are used, the prettyprinter acts appropriately.

Update: I should note that this construct is a bit more general than it looks at first. If you want to include a vocabulary for the duration of one word definition, you can do the following.

<< foo
 : bar ( a -- )
 word1-from-foo swap [ word2-from-foo ] each-with ;
>>

The words from foo will be taken from the appropriate vocab, while swap and each-with will also function (assuming they're already in the use path). A weird side effect of this is that << foo swap>> works the same as swap, probably providing the kernel definition in both cases. The qualified naming system could disallow this, but I don't see the point.