inference-engine

Star

Here are 277 public repositories matching this topic...

Language: All

Filter by language

All 277 Python 90 C++ 73 Jupyter Notebook 23 C# 14 Java 10 JavaScript 9 Rust 6 TypeScript 6 Go 5 C 4

Sort: Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

FedML-AI / FedML

Star 4k

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

machine-learning deep-learning inference-engine model-deployment model-serving distributed-training federated-learning mlops edge-ai ai-agent on-device-training

Updated Aug 11, 2025
Python

zjhellofss / KuiperInfer

Star 3.2k

校招、秋招、春招、实习好项目!带你从零实现一个高性能的深度学习推理库,支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step

deep-neural-networks caffe deep-learning graph-algorithms inference pytorch yolo convolution diy resnet maxpooling sigmoid inference-engine ncnn relu yolov5 pnnx

Updated Jun 22, 2025
C++

hyperjumptech / grule-rule-engine

Star 2.4k

Rule engine implementation in Golang

golang rule-engine rule hacktoberfest inference-engine rule-based rule-based-engine hacktoberfest2021

Updated Aug 30, 2025
Go

onediff

siliconflow / onediff

Star 1.9k

OneDiff: An out-of-the-box acceleration library for diffusion models.

cuda pytorch lora lcm performance-optimization inference-engine diffusion-models stable-diffusion diffusers sd-webui comfyui sdxl aigc-serving lcm-lora stable-video-diffusion sdxl-turbo comfyui-workflow

Updated May 8, 2025
Jupyter Notebook

aphrodite-engine / aphrodite-engine

Star 1.6k

Large-scale LLM inference engine

machine-learning cuda intel api-rest lora rocm inference-engine tpu inferentia speculative-decoding

Updated Oct 12, 2025
C++

Tencent / FeatherCNN

Star 1.2k

FeatherCNN is a high performance inference engine for convolutional neural networks.

android ios caffe convolutional-neural-networks arm-neon inference-engine

Updated Sep 24, 2019
C++

PaddlePaddle / Paddle.js

Star 1.1k

Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also...

webgl ocr deep-learning model webassembly webgpu inference-engine paddlepaddle

Updated Apr 3, 2024
JavaScript

zhihu / ZhiLight

Star 901

A highly optimized LLM inference acceleration engine for Llama and its variants.

cuda pytorch llama gpt inference-engine model-serving llm llm-serving llm-inference deepseek-r1

Updated Jul 10, 2025
C++

quic / ai-hub-models

Star 814

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.