-
Notifications
You must be signed in to change notification settings - Fork 733
Support for spark 3.5+ #14678
-
Good day,
When can we expect support for higher versions of spark?
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions
Hi @MartinLisy,
Making Spark NLP compatible with higher Spark versions is definitely a high priority for us, and we are working on it!
Spark 3.5.* works, but Spark 4 and up requires some substantial changes in the library.
Replies: 1 comment 3 replies
-
Hi @MartinLisy,
Making Spark NLP compatible with higher Spark versions is definitely a high priority for us, and we are working on it!
Spark 3.5.* works, but Spark 4 and up requires some substantial changes in the library.
Beta Was this translation helpful? Give feedback.
All reactions
-
Thank you @DevinTDHa for the quick response. I can confirm that Spark NLP works on Spark 3.5.0. However, I have ran into issues when running Spark 3.5.2, for instance "Py4JError: com.johnsnowlabs.nlp.DocumentAssembler does not exist in the JVM". Is this something You are aware of or does it recquire an issue thread?
Beta Was this translation helpful? Give feedback.
All reactions
-
Hi @MartinLisy,
We are not actually aware of issues there. It'd very be helpful for us if you can open an issue thread with precise reconstruction steps. That way, we can help you the most effectively. Thanks a lot!
Beta Was this translation helpful? Give feedback.
All reactions
-
Hi @DevinTDHa,
My issues are explained by the closed issue #14203. Thanks for the support!
Best Regards.
Beta Was this translation helpful? Give feedback.