The goal of this project is to implement a tiny inference only library for running ML models. I want this to be something like ggml and tinygrad.

The idea is to support different optimization backends like:

• Accelerate
• AVX
• openblas
• cuBLAS (not sure about cuBLAS)
• naive CPU only (fallback)
• etc

The roadmap right now is more or less like this:

• implement a minimal tensor class with support for broadcasting and dynamic shapes
• implement a CPU only backend and write tests for different ops
• write other backends
• support fp16, int8 and quantization
• a demo of the lib using llama or something similar
• would also like this to work on vision models like segment anything, resnet, etc