GLM-5.2-NVFP4-REAP-469B serving on SM120 (×ばつ RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Attention + MTP speculative decode
-
Updated
Jun 19, 2026 - Shell
#
rtx-pro-6000
Here are 32 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
multimodal long-context vllm local-llm speculative-decoding nvidia-blackwell rtx-pro-6000 exl3 glm-5-3
-
Updated
Sep 4, 2026 - Python
An LLM server for a single RTX 5090, built for agent workloads: tool calls, long conversations, reasoning, and many requests at once. One of the fastest engines on this card, at batch 1 and at dozens of concurrent streams, with the numbers in the repo.
cpp cuda quantization inference-engine mixture-of-experts blackwell llm safetensors llama-cpp vllm local-llm llm-inference qwen gguf fp4 rtx-5090 sm120 nvfp4 rtx-pro-6000 gated-deltanet
-
Updated
Sep 7, 2026 - Cuda
Reproducible SGLang recipe + public prebuilt image (ghcr.io) for DeepSeek-V4-Flash-0731 on 4x RTX PRO 6000 Blackwell (SM120): TP4/DP4/EP4, 1M ctx, benchmarks, and the DSPARK draft-depth corruption boundary
blackwell llm-inference speculative-decoding deepseek sglang flashinfer sm120 rtx-pro-6000 deepseek-v4
-
Updated
Aug 9, 2026 - Dockerfile
Rust + CUDA LLM inference engine for Blackwell (Tuned specifically on RTX PRO 6000, RTX 5090, B200): OpenAI-compatible (+converse and ant) serving, per-model X hardware exactness gates. NVFP4/mixed (fp8 hybrid, 4o6, etc - correctness, performance, hardware specific adapted) main quant support.
rust cuda inference glm gemma inference-engine b200 mixture-of-experts blackwell openai-api llm llm-serving qwen gguf speculative-decoding deepseek rtx-5090 nvfp4 rtx-pro-6000 deepseek-v4
-
Updated
Sep 7, 2026 - OpenEdge ABL
180-226 tok/s single-stream decode for Qwen3.8-Flash-Next NVFP4 on one RTX PRO 6000 Blackwell, at full 262K context with unchanged quantization. Config, benchmark harness, and the FlashInfer autotune correctness bug that silently corrupts output.
-
Updated
Aug 27, 2026 - Python
Systematic 24-hour benchmark study of Qwen3.6-27B inference on dual NVIDIA RTX PRO 6000 Blackwell SM120 (TP=2). 8 experiments comparing repne/vllm fork vs upstream vLLM across FP8/BF16/NVFP4/Q8_0 quants and MTP/DFlash speculative decoding. Peak: 2,083 tok/s at c=32. Quality: KLD vs BF16 = 0.0018 (noise floor).
-
Updated
Jun 3, 2026 - Python
Fish Audio OpenAudio S2-Pro on vLLM-Omni. low-latency ~100ms TTFA, OpenAI-compatible, runs on NVIDIA Blackwell (RTX 5090 / RTX PRO 6000). Self-hosted streaming TTS & voice cloning.
text-to-speech streaming real-time cuda self-hosted tts speech-synthesis low-latency openaudio voice-cloning fastapi blackwell openai-api vllm fish-speech rtx-5090 rtx-pro-6000 elevenlabs-alternative
-
Updated
Jun 27, 2026 - Python
MiniMax-M3 (428B MoE) running on ×ばつ RTX PRO 6000 Blackwell at TP=3 with 240K context, FP8 KV cache, and working multimodal vision input. Includes dist_utils.py patch for non-divisible attention heads.
vision multimodal inference-optimization blackwell tensor-parallelism vllm nvfp4 rtx-pro-6000 minimax-m3
-
Updated
Jul 15, 2026 - Shell
Image-to-3D-Video-Asset-Generator is an all-in-one generative 3D pipeline that transitions smoothly from textual concepts or reference images into fully realized 3D mesh assets (.glb), dynamic camera movements in 5-second MP4 videos, and clean bundle exports (.zip).
voxel transformers torch trellis gradio rerun torchvision trimesh image-to-3d pyrender huggingface-spaces flash-attention-2 rerun-sdk rtx-pro-6000 nvdiffrast
-
Updated
Jul 30, 2026 - Python
Production-grade FlashAttention FP8 e4m3 forward kernel for NVIDIA Blackwell consumer GPUs (sm_120a, e.g. RTX PRO 6000). 647–652 TFLOPS at hd=128, sl=8192. Multi-kernel dispatcher, C library with Go and Python bindings
cuda transformer attention gpu-kernels tensor-cores blackwell fp8 flash-attention fp8e4m3 sm120 rtx-pro-6000 sm-120a
-
Updated
Aug 3, 2026 - Cuda
Hub for ongoing Qwen inference benchmarks on NVIDIA Blackwell. Indexes all studies, hosts the rolling SOTA leaderboard, points to the toolchain.
-
Updated
May 15, 2026 - Python
Prolepsis is a speculative decoding implementation for Qwen3 draft-target models with Hugging Face and vLLM backends. On an RTX PRO 6000 at batch size 1, it measured 1.72x throughput with vLLM FP8 and 1.32x with Hugging Face BF16, with complete latency and response artifacts.
pytorch triton gpu-optimization inference-optimization huggingface llm vllm speculative-decoding rtx-pro-6000
-
Updated
Jul 14, 2026 - Python
GLM-5.2-504B NVFP4 at 250K context on 4x RTX PRO 6000 Blackwell (sm_120) using STOCK vLLM — no fork, no Docker, no CUDA 13.2. One ~126-line patch. Documents the 3 upstream bugs that block you, with exact error strings and fixes.
-
Updated
Sep 1, 2026 - Python
Deploy the GLM-5.2-469B model on four RTX PRO 6000 Blackwell GPUs using a turnkey vLLM Docker configuration to enable high-speed sparse attention and inference.
android open-source benchmark ocr smartthings smartapp moe glm zwave gles2 reap vllm chatglm-6b long-horizon rtx-pro-6000
-
Updated
Sep 7, 2026 - Shell
Prebuilt spconv v2.3.8 wheels for CUDA 12.8 / 13.0 with native Blackwell (RTX 50-series, sm_120) kernels — for default PyPI torch (cu130) or torch +cu128
cuda pytorch lidar wheels pointcloud b200 lidar-point-cloud 3dvision blackwell spconv b100 rtx-5090 rtx5080 blackwell-gpu rtx5060 blackwell-architecture rtx-pro-6000 gb200 rtx5070
-
Updated
Sep 2, 2026 - Shell
Add this topic to your repo
To associate your repository with the rtx-pro-6000 topic, visit your repo's landing page and select "manage topics."