AI & ML 1d ago
llama.cpp adds experimental SM120 CUTLASS MoE prefill
New CUDA kernels target MXFP4 and NVFP4 on SM120 GPUs, while maintainers push for MMVQ refactoring before deeper integration.
By tensorAI & ML
AI & ML 1d ago
PyTorch adds experimental torch.compile for complex tensors
Complex dtypes can now go through the compiler via a real-valued decomposition path that must be opted in explicitly.
By tensorAI & ML 1d ago
llama.cpp server gains image-text embeddings for Qwen3-VL
Multimodal embedding and rerank endpoints now support Qwen3-VL models for RAG pipelines that mix pictures and prose.
By tensorAI & ML 1d ago
PyTorch 2.14 RFC would make CUDA 13.2 the default PyPI build
The proposal keeps CUDA 13.0 as a stable alternate and CUDA 12.6 as legacy while pushing Blackwell-capable 13.2 to pip install torch.
By tensorAI & ML 1d ago
llama.cpp MoE GPU cache nearly doubles decode on dual Vulkan cards
A host-offloaded expert-weight LRU cache kept hot experts in VRAM and lifted Qwen MoE decode from about 8 to nearly 19 tokens per second on two RX 6950 XTs.
By tensorAI & ML 2d ago
llama.cpp doubles cold prefill with direct PLE table reads
A direct-read path for lazy PLE tables roughly halves cold-cache prefill on Windows and keeps memory flat on Apple Silicon.
By tensorAI & ML 2d ago
PyTorch device capability API breaks non-CUDA heuristics
CUDA returns a version tuple while XPU returns a dict, so shared autoheuristic metadata never matches on those backends.
By tensorAI & ML 2d ago
llama.cpp Docker images lack SemVer tags, miss stable builds
Maintainers will change release packaging so GHCR containers ship with proper version tags when a stable build is cut.
By tensorAI & ML 2d ago
PyTorch fixes silent out-of-bounds read in compiled kernels
Strided and offset tensor paths in the compiler could read past valid memory without raising an error.
By tensorAI & ML 2d ago
torch.compile over-reads offset views in Triton kernels
PyTorch can silently return wrong results when compiled kernels skip writing masked output regions.
By tensorAI & ML 2d ago
PyTorch MPS causal attention leaks future tokens in half precision
On Apple Silicon, scaled_dot_product_attention with is_causal=True can silently ignore the causal mask for float16 and bfloat16.
By tensor