ggml

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

machine-learning deployment inference pytorch artificial-intelligence llama whisper gemma mistral openai-api llm flan-t5 chatglm llamacpp vllm ggml wizardlm qwen llama3 glm4

Updated Oct 23, 2025
Python

rustformers / llm

Star 6.1k

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

rust ai ml llm ggml

Updated Jun 24, 2024
Rust

leejet / stable-diffusion.cpp

Star 4.5k

Diffusion model(SD,Flux,Wan,Qwen Image,...) inference in pure C/C++

flux ai cplusplus image-generation wan diffusion text2image image2image img2img txt2img latent-diffusion stable-diffusion ggml videogeneration flux-dev flux-schnell qwen-image

Updated Oct 23, 2025
C++

guinmoon / LLMFarm

Star 1.9k

llama and other large language models on iOS and MacOS offline using GGML library.

macos swift ios ai llama gpt-2 rwkv ggml gptneox starcoder

Updated Sep 19, 2025
C

RWKV / rwkv.cpp

Star 1.5k

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

machine-learning deep-learning quantization language-model llm rwkv ggml

Updated Mar 23, 2025
C++

gollama

sammcj / gollama

Sponsor

Star 1.5k

Go manage your Ollama models

macos linux ai models tui llm ggml ollama gguf

Updated Sep 29, 2025
Go

RahulSChand / gpu_poor

Star 1.4k

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

gpu pytorch llama quantization language-model huggingface llm llamacpp ggml llama2

Updated Dec 3, 2024
JavaScript

PABannier / bark.cpp

Star 840

Suno AI's Bark model in C/C++ for fast text-to-speech generation

machine-learning text-to-speech inference tts ggml

Updated Nov 16, 2024
C++

the-crypt-keeper / can-ai-code

Star 596

Self-evaluating interview for AI coders

ai transformers humaneval llm langchain llama-cpp ggml

Updated Jun 21, 2025
Python

pollockjj / ComfyUI-MultiGPU

Star 583

This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose

pytorch unet-pytorch stable-diffusion comfyui ggml comfyui-workflow comfyui-nodes gguf-models wanvideowrapper