0

I can find a load of information on the reverse, not so much this way around :)

So, the summary is that I want to write some Python code-completion stuff in C++, but I can't figure out the best way of tokenizing the Python code.

Are there any libraries out there that will do this?

I'm leaning towards calling Python's tokenize.tokenize directly from C++... but whenever I look at calling Python code from C++ I go cross-eyed.

asked Oct 16, 2012 at 11:01
2
  • 1
    Have you checked out Boost.python? Commented Oct 16, 2012 at 11:06
  • 1
    Python code completion support is very limited due to its dynamic nature. Add to that trying to do it in C++, you are setting yourself up for a world of pain. Commented Oct 16, 2012 at 11:55

1 Answer 1

3

Using regular parser-generators to generate parsers from the grammar is usually complicated with Python (for example due to its significant whitespace and difficult line-continuation rules).

I am not sure about your experience with Python, but my recommendation would be to parse the Python file from Python, and do as much of the processing as possible in Python, then return the result to the C++ code using well-defined data types (such as the stdc++ ones) and using Boost.python for the bindings.

answered Oct 16, 2012 at 11:06
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.