vLLM with sm_70 (Volta) Support

⚠️ This project is archived and no longer maintained. For Volta (SM70) vLLM — including Qwen3.5 / Qwen3.6 — use 1CatAI/1Cat-vLLM instead: a more complete V100 fork with a native FLASH_ATTN_V100 kernel, AWQ support, and validated multi-GPU Qwen3.5 deployments.

docker pull ghcr.io/jajmangold/vllm-sm70:latest

A Docker image for running the latest vLLM on older NVIDIA GPUs with sm_70 compute capability (Volta architecture), including:

• Tesla V100
• Titan V
• Quadro GV100
• NVIDIA CMP 100-210 (mining GPUs)

This image is built to be feature-complete for inference on Volta, not a crippled fallback.

Despite running on Volta, this image includes the modern inference stack you care about:

• ✅ Triton attention
• ✅ PyTorch SDPA (scaled dot-product attention fallback)
• ✅ bitsandbytes (bnb) for efficient quantized weights
• ✅ AutoRound for W4A16 / low-bit quantization workflows
• ✅ CUDA graphs (enabled by default in vLLM)
• ✅ Continuous batching and KV cache reuse (vLLM core features)

What you don’t get (hardware limits, not software):

• ❌ FlashAttention v2 (requires sm_80+)
• ❌ FP8 / Hopper-only kernels
• ❌ Marlin (Ampere+)

Newer official vLLM images and recent PyTorch releases increasingly drop or de-prioritize Volta (sm_70) support.

This project takes the pragmatic route:

• Use a known-good prebuilt PyTorch image that still includes sm_70
• Include AutoRound for SOTA modern quantized inference
• Avoid PyTorch source builds
• Focus on running inference on Volta, not fighting packaging

• Base image: pytorch/pytorch:2.7.1-cuda12.8-cudnn9-runtime

• CUDA: 12.8

• cuDNN: 9

• PyTorch: 2.7.1 (prebuilt, includes sm_70)

• vLLM: latest (auto-built from upstream releases)

• Attention backends:

  • Triton
  • xFormers
  • PyTorch SDPA

• Quantization tooling:

  • bitsandbytes
  • AutoRound

• Python: from base image

The latest version of vLLM checked nightly . Builds happen when upstream releases happen.

Pull the pre-built image from GitHub Container Registry:

docker pull ghcr.io/jajmangold/vllm-sm70:latest

This tag always tracks:

• the newest upstream vLLM release
• a Volta-compatible PyTorch base
• a full inference feature set ( bnb, AutoRound)