Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Using borrowed ArrayView across threads #394

Answered by mejrs
edmnbv asked this question in Q&A
Discussion options

I have a FFI program that reads in a numpy array and polars DataFrame, does some aggregations in parallel and then returns a DataFrame. As part of this I need to supply the borrowed ArrayView into a closure which needs to be 'static + Send + Sync. Since this ArrayView is not 'static, it is not considered safe. However in this use case it seems justifiable. Note that scoped threads are not an option.

Yesterday I posted this on StackOverflow. If you replace the Foo type with an ArrayView, this is closer to my problem.

Asking here because the situation seems more likely to happen using the numpy crate as opposed to ndarray alone (It wouldn't be a problem if we had an owned Array). I also tried using a RawArrayView which does not carry a lifetime, however, it is not Send + Sync and so will not work.

First thing to answer is: Is this a justifiable thing to do? From my understanding the ArrayView can be treated as ``static` in this particular usage of the FFI. Moving forward, two solutions I can see is via a transmute (see StackOverflow answer), or to wrap up the RawArrayView in a new type and unsafely implement Send + Sync.

You must be logged in to vote

I'm not that familiar with polars but it looks like you can avoid this by just not using a lazy dataframe (which in your code there is no reason to anyway).

two solutions I can see is via a transmute

This is a bad idea; don't do this.

Replies: 1 comment 1 reply

Comment options

I'm not that familiar with polars but it looks like you can avoid this by just not using a lazy dataframe (which in your code there is no reason to anyway).

two solutions I can see is via a transmute

This is a bad idea; don't do this.

You must be logged in to vote
1 reply
Comment options

Thanks for the response. You're right that this would work without the lazy API and may be the only good solution. The reason why I used the lazy API was because the eager method is deprecated.
https://docs.rs/polars/latest/polars/frame/group_by/struct.GroupBy.html#method.par_apply

two solutions I can see is via a transmute

This is a bad idea; don't do this.

Well noted 😄

Answer selected by edmnbv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants

AltStyle によって変換されたページ (->オリジナル) /