This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose
A quant trading system platform based on FinGPT, demonstrating new applications of large pre-trained Language Models in quantitative finance.
Experimental interface environment for open source LLM, designed to democratize the use of AI. Powered by llama-cpp, llama-cpp-python and Gradio.
Discover the Best AI Models for Your PC
Run Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1, and other state-of-the-art language models locally with scorching-fast performance. Inferno provides an intuitive CLI and an OpenAI/Ollama-compatible API, putting the inferno of AI innovation directly in your hands.
Simple Python Script that converts the Weight of a GGUF Model to a PyTorch Tensor
π€ Serverless AI on Hedera! Smart contracts run LLMs βοΈ Distilled GGUF on IPFS. Pay crypto πͺ β get replies π¬. 99.9% cheaper πΈ than big-tech APIs. Decentralized βοΈ no censorship. "Gov canβt stop it. Nobody can." π§ β Sir Charles Spikes , Cincinnati,Ohio| TELEGRAM: @SirGODSATANAGIβοΈ SirCharlesspikes5@gmail.com #AGI #NoPC
This is a Bash script to automatically launch llama-server, detects available .gguf models, and selects GPU layers based on your free VRAM.
A high-performance local service engine for large language models, supporting various open-source models and providing OpenAI-compatible API interfaces with streaming output support.
LLMBuilder is a production-ready framework for training and fine-tuning Large Language Models (LLMs) β not a model itself. Designed for developers, researchers, and AI engineers, LLMBuilder provides a full pipeline to go from raw text data to deployable, optimized LLMs, all running locally on CPUs or GPUs.
Automate bulk downloads of Hugging Face LLMs with retry logic, manifest export, checksum validation, and usage reporting. Ideal for managing GGUF models at scale.
A modular CLI chatbot with Discord webhook integration optimized for edge devices using GGUF quantized models.
A web UI for managing multiple models with llama-server.exe on windows
A fine-tuned legal QA system using Zephyr or LLaMA with GGUF support. Real-time inference via Hugging Face API or local model hosting.
This repository demonstrates how to use outlines and llama-cpp-python for structured JSON generation with streaming output, integrating llama.cpp for local model inference and outlines for schema-based text generation.
A fast and reliable model downloader written in Go for downloading HuggingFace models.
π€ Harness AI reasoning on blockchain with serverless architecture, utilizing HBAR tokens for on-demand, scalable intelligence without central servers.
Add a description, image, and links to the gguf-models topic page so that developers can more easily learn about it.
To associate your repository with the gguf-models topic, visit your repo's landing page and select "manage topics."