Premise
At the moment, name resolution hackily uses syn for parsing. This prevents the use of the local lexer and token buffer representation, both of which are more efficient. syn itself is quite slow and performs a lot of unnecessary allocations. It does not support Rust's nightly features, used by the standard library. We need a custom parser.
Design Decisions
The most important design questions involve the AST, the output of the parser, and how it is consumed by name resolution. There are two primary proposals:
-
Build an AST that supports random access and let name resolution use it to resolve out-of-order item references. The parser would produce the full AST, the name resolver would use it to resolve some references, and then would lower the AST into partially-resolved HIR items.
-
Build a streaming AST (which can only be traversed once, in order) and let name resolution build a scope tree out of it. During that single traversal, declarations in the AST would be used to build a file-local scope tree, and items would be lowered to unresolved HIR simultaneously. Afterwards, a local name resolution step would resolve references in the local scope tree, and look for declarations outside the file.
The latter is preferable. The AST would just be the output of a pull parser, and would be very straightforward to implement. The scope tree would be a better data structure for resolving lookups. The HIR can be built directly from a streamed AST (most importantly, within function bodies). Fewer allocations would be needed.
Architecture
The parser will be a streaming parser; it will return individual items as soon as they are parsed. The API can be structured in two ways -- fn parse() -> Iterator<Item = ParseItem>, or fn ModuleParser::next() -> Option<ItemParser<'_>>. The former is more common, but the latter may be easier to implement (e.g. could rely on the caller's stack).