-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Hello, Arrow community!
Firstly, let me thank you for the great work you've done on this project so far, it's been tremendous help for my activities, so I really appreciate that :)
Secondly, after reading this Pre-PEP, I started wondering where could Rust fall within existing PyArrow implementation, either currently, given that arrow-rs is quire mature already, or in future, if the aforementioned Pre-PEP ever gets accepted and Rust starts being an integral part of CPython. Given all the reasons that were stated in the Pre-PEP, it does seem like a good addition, although all the drawbacks are valid as well.
So, the question is - does it seem like a good idea to include arrow-rs as a foundation for PyArrow? Forgive me if this question seems silly, I just want to hear out what community members think on this matter :)
All reactions
Replies: 2 comments 2 replies
There is no relationship between Rust being used in CPython and Rust being used in PyArrow.
PyArrow wraps Arrow C++, with most of the wrapping code using Cython, and therefore I'm not sure how Rust could be useful for us. Perhaps it might be useful for Arrow C++ itself, though that sounds unlikely at the moment.
All reactions
Ok, I see, I've mixed those up, thank you!
All reactions
I wonder if https://github.com/kylebarron/arro3/ is the kind of project you had mind? It leverages pyO3/arrow-rs and splits up functionality a bit more aggressively for a much smaller binary size.
All reactions
Yeah, exactly something like this :) I assumed that the Rust binary would be a bit smaller in size, but what they achieved there is impressive, thanks!