Python ANTLR4 example - Parser doesn't seem to parse correctly

Question

To demonstrate the problem, I'm going to create a simple grammar to merely detect Python-like variables.

I create a virtual environment and install antlr4-python3-runtime in it, as mentioned in "Where can I get the runtime?":

Install antlr4-python3-runtime

Then, I create a PyVar.g4 file with the following content:

grammar PyVar;
program: IDENTIFIER+;
IDENTIFIER: [a-zA-Z_][a-zA-Z0-9_]*;
NEWLINE: '\n' | '\r\n';
WHITESPACE: [ ]+ -> skip;

Now if I test the grammar with grun, I can see that the grammar detects the variables just fine:

Testing the grammar with grun

Now I'm trying to write a parser in Python to do just that. I generate the Lexer and Parser, using this command:

antlr4 -Dlanguage=Python3 PyVar.g4

And they're generated with no errors:

antlr4 -Dlanguage=Python3 PyVar.g4

But when I use the example provided in "How do I run the generated lexer and/or parser?", I get no output:

Python parser produces nothing

What am I not doing right?

Answer 1

There are two problems here.

1. The grammar:

In the line where I had,

program: IDENTIFIER+;

the parser will only detect one or more variables, and it will not detect any newline. The output you see when running grun is the output created by the lexer, that's why newlines are present in the tokens. So I had to replace it with something like this, for the parser to detect newlines.

program: (IDENTIFIER | NEWLINE)+;

2. Printing the output of parser

In PyVar.py file, I created a tree with this line:

tree = parser.program()

But it didn't print its output, nor did I know how to, but the OP's comment on this accepted answer suggests using tree.toStringTree().

Now if we fix those, we can see that it works:

Works!