Generate parser for Python3 in python, using ANTLR 4.6

Question

I'm using the ANTLRv4 Python3 grammar from here:

https://github.com/antlr/grammars-v4/blob/master/python3/Python3.g4

and running:

java -jar antlr-4.6-complete.jar -Dlanguage=Python2 Python3.g4

to generate Python3Lexer.py + some other files.

However, Python3Lexer.py contains code which is not python! For eg.

 def __init__(self, input=None):
 super(Python3Lexer, self).__init__(input)
 self.checkVersion("4.6")
 self._interp = LexerATNSimulator(self, self.atn, self.decisionsToDFA, PredictionContextCache())
 self._actions = None
 self._predicates = None
 // A queue where extra tokens are pushed on (see the NEWLINE lexer rule).
 private java.util.LinkedList<Token> tokens = new java.util.LinkedList<>();
 // The stack that keeps track of the indentation level.
 private java.util.Stack<Integer> indents = new java.util.Stack<>();

Its unusable because of this. Does anyone know why this is happening and how I can fix it? Thanks!

Answer 1

This grammar is full of action code written in Java to deal with specialities of Python. You have to port that manually to python to make the grammar usuable for you. This is why grammar writers are encouraged to move out action code into base classes or listener code.

• greenberet123: Thanks! I moved to Java as the target language to avoid this now.