Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
A flexible, high-performance serving system for machine learning models
A Cloud Native Batch System (Project under CNCF)
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
Serve, optimize and scale PyTorch models in production
Build custom inference engines for models, agents, multi-modal systems, RAG, pipelines and more.
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
Database system for AI-powered apps
TensorFlow template application for deep learning
A comprehensive guide to building RAG-based LLM applications for production.
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.
DELTA is a deep learning based natural language and speech processing platform. LF AI & DATA Projects: https://lfaidata.foundation/projects/delta/
A flexible, high-performance carrier for machine learning models(『飞桨』服务化部署框架)
A scalable inference server for models optimized with OpenVINOTM
Generic and easy-to-use serving service for machine learning models
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
A unified end-to-end machine intelligence platform
A high-performance inference system for large language models, designed for production environments.
Add a description, image, and links to the serving topic page so that developers can more easily learn about it.
To associate your repository with the serving topic, visit your repo's landing page and select "manage topics."