Integrating peroxide #240

Now, I will answer the next two questions you asked.

First, I created DataFrame struct and Series struct, inspired by pandas, a popular dataframe library used in Python. Next, I wanted to make it simple to read an write to multiple file formats. So, I choose csv, NetCDF and parquet.

The implementation itself is not particularly special, there are already a great Rust libraries georust/netcdf and arrow2, so I implemented only the frontend to use it easily in the peroxide's DataFrame using these libraries as backends.

Of course, I also considered implementing HDFS, but Rust's excellent hdf5 library hdf5-rust requires ndarray, which overlaps a lot with peroxide, as default, so I didn't use that. Fortunately, the hdf5-rust team provides hdf5-sys, which is a binding for the HDF5 C library, so I want to using it, but it seems that it will take some time because I'm not familiar with HDF5.

I wonder what the questions about gradient descent and gradient boost are asking. Could you give me a little more detail?

Thanks for the great presentation of peroxide, there are so many good points, in particular the Dataframe and Series implementation that we don't have yet in smartcore

I wonder what the questions about gradient descent and gradient boost are asking. Could you give me a little more detail?

Sure. as XGBoost is the most popular algorithm for gradient descent, we are considering having it implemented in smartcore, see this for a reference.
What are in your opinion the most relevant use cases for a linear algebra user to use XGBoost? How would you envision a good interface to allow usage of gradient descent in a linear algebra library (not involving Neural Nets)?

Well... first of all, I don't have a special idea about XGBoost because I don't use Boosted Decision Tree methods well.
However, in the case of gradient descent, there are a few things to say about the necessity and implementation.

In my case, there were quite a few cases where gradient calculation was needed for arbitrary functions when performing optimization (e.g. Newton's method, etc.). Therefore, in peroxide, there were some automatic differentiation tools - peroxide-ad.
I use this to compute exact jacobian or hessian and use these in newton's method. Here is the example - Root finding via AD. Or even I wrote a skeleton multi-layer perceptron with it - MLP (from scratch), but that's probably not what you're interested in.

Since I'm not familiar with gradient boosting, this is all my opinion. I apologize if the focus is off 😥

Thanks, this is a good answer that clears some of my doubts.

Is there something you would like to be helped with to improve peroxide?

EDIT: it would be possible to use peroxide to implement RL algorithms (see #244 )? How would you proceed?