Hi Pablo, could you clarify please? Is that on the main branch, or would
you be willing to share the code?
On 30/05/2022 16:23, Pablo Galindo Salgado wrote:
There is no *public* one but there is a private one accesible from
Python I added for testing purposes.
On 2022年5月30日, 15:17 Victor Stinner, <[email protected]> wrote:
On Mon, May 30, 2022 at 1:40 AM Eric V. Smith <[email protected]>
wrote:
> python -m tokenize < file-to-parse.py
>
> See the comment at the top of tokenize.py. IIRC, it
re-implements the
> tokenizer, it does not call the one used for python code.
Ah right, I would be surprised that there would be a public Python API
to get the tokenizer output, since there is no public C API for that
:-)
I just removed <token.h> header file since it was never usable outside
Python C internals: there is no public C API to just run the tokenizer
and gets its output.
Victor
_______________________________________________
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/CT3YSWSPMJ5DLUCVBX3AAPRWOUOXYWEL/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/D3EMINLPE6TXEENR27LOVNN7TTYM2GNE/
Code of Conduct: http://python.org/psf/codeofconduct/