TextMate grammars support a comment field on rules for documentation purposes. This field was being parsed but not exposed to consumers, making it inaccessible to tools like VS Code's scope inspector.

Changes

Type System

• Added optional comment field to IRawRule interface
• Added scopeComments: (string | null)[] to IToken interface, parallel to existing scopes array

Rule Storage

• Updated Rule base class and all subclasses (CaptureRule, MatchRule, IncludeOnlyRule, BeginEndRule, BeginWhileRule) to store and expose comments via getComment() method
• Modified RuleFactory to extract comment field from grammar rules during compilation

Scope Stack

• Extended ScopeStack to track scopeComment alongside scopeName, added getComments() method
• Updated AttributedScopeStack.pushAttributed() to accept and propagate comment parameter
• For multi-scope pushes (space-separated scopes), comment is assigned to first scope only

Tokenization

• Modified tokenization logic to retrieve comments from rules and pass through pushAttributed() calls
• Token creation now includes scope comments array matching scope hierarchy

Example

// Grammar with comments
{
 "match": "\\b(hello)\\b",
 "name": "keyword.test",
 "comment": "Matches the hello keyword"
}
// Resulting token
{
 scopes: ["source.test", "keyword.test"],
 scopeComments: [null, "Matches the hello keyword"]
}

Comments default to null when not present, maintaining backward compatibility.

• Fixes Expose scope comment in IToken #227

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.