Skip to content

ggml-org llama.cpp Q-a Discussions

Pinned Discussions

ggerganov
Inference at the edge
Announcements ggerganov
ggerganov
llama.cpp supports the new gpt-oss model in native MXFP4 format
Announcements ggerganov
ggerganov
guide : running gpt-oss with llama.cpp
Show and tell ggerganov

Sort by: Latest activity

Categories, most helpful, and community links

Categories

Loading

Community links

Community guidelines

Q&A Discussions

Ask the community for help

You must be logged in to vote

Can madlad400 gguf models from huggingface be used?

OkLunixBSD asked Jul 4, 2024 in Q&A · Answered

@OkLunixBSD @brauliobo @egorsmkv

@OkLunixBSD @brauliobo @egorsmkv

29
You must be logged in to vote

Parallelization / Batching Explanation

lapp0 asked Nov 19, 2023 in Q&A · Unanswered

@lapp0 @Artefact2 @ggerganov

@lapp0 @Artefact2 @ggerganov

46
You must be logged in to vote

compute types per platform

okuvshynov asked Oct 23, 2025 in Q&A · Unanswered

@okuvshynov

@okuvshynov

0
You must be logged in to vote

how can I use rvv in riscv?

cyk2018 asked Oct 22, 2025 in Q&A · Unanswered

@cyk2018

@cyk2018

0
You must be logged in to vote

GGML philosophy question...

cptspacemanspiff asked Oct 21, 2025 in Q&A · Unanswered

@cptspacemanspiff @ggerganov

@cptspacemanspiff @ggerganov

1
You must be logged in to vote

Making the output of LLAMA Server exact the same like LLAMA in the instruct mode

alexcardo asked Feb 5, 2024 in Q&A · Unanswered

@alexcardo @wtarreau @qnixsynapse

@alexcardo @wtarreau @qnixsynapse

4
You must be logged in to vote

shifting or self extending hybrid memory?

leok7v asked Oct 21, 2025 in Q&A · Unanswered

@leok7v

@leok7v

0
You must be logged in to vote

How to add a new backend for llama.cpp

yuenyu1 asked Jun 24, 2025 in Q&A · Answered

@yuenyu1 @taronaeo

@yuenyu1 @taronaeo

2
You must be logged in to vote

llama-server Web UI - Is there a setting to make GPT-OSS render new lines in markdown tables instead of <br>?

TFWol asked Oct 19, 2025 in Q&A · Unanswered

@TFWol

@TFWol

0
You must be logged in to vote

Does reasoning_budget work for any model?

Som-anon asked Oct 19, 2025 in Q&A · Unanswered

@Som-anon

@Som-anon

0
You must be logged in to vote

Really struggling, latest version of llama-server, Mistral Nemo Instruct 2407, adding tool capability

linuxmagic-mp asked Oct 18, 2025 in Q&A · Unanswered

@linuxmagic-mp

@linuxmagic-mp

0
You must be logged in to vote

How to specify device order when using RPC? Getting worse performance than expected on multiGPU + RPC.

Panchovix asked Oct 17, 2025 in Q&A · Answered

@Panchovix @rgerganov @slaren

@Panchovix @rgerganov @slaren

20
You must be logged in to vote

Llama.cpp RPC over Ethernet strangely slow

becky-soda asked Aug 22, 2024 in Q&A · Unanswered

@becky-soda @ggerganov @Bikkies

@becky-soda @ggerganov @Bikkies

6
You must be logged in to vote

Boost performance by using idle GPU periods

karambaso asked Oct 16, 2025 in Q&A · Unanswered

@karambaso

@karambaso

0
You must be logged in to vote

RAM speed vs cores count

LordKAS asked Oct 14, 2025 in Q&A · Unanswered

@LordKAS @kirnat @Rotatingxenomorph

@LordKAS @kirnat @Rotatingxenomorph

4
You must be logged in to vote

is it possible to convert Ovi model (audio+video) via llama.cpp?

mariambriccola-eng asked Oct 14, 2025 in Q&A · Unanswered

@mariambriccola-eng

@mariambriccola-eng

0
You must be logged in to vote

Runtime Overhead on Quantization Group Size

kimjoohyungsd asked Sep 18, 2025 in Q&A · Unanswered

@kimjoohyungsd

@kimjoohyungsd

0
You must be logged in to vote

Is it possible to run on ubuntu 18.04?

canh25xp asked Jun 26, 2025 in Q&A · Answered

@canh25xp @misutoneko

@canh25xp @misutoneko

2
You must be logged in to vote

Why any model running with llama server behave differently?

alexcardo asked Sep 26, 2024 in Q&A · Unanswered

@alexcardo @mirek190 @stephen-lev

@alexcardo @mirek190 @stephen-lev

6
You must be logged in to vote

No pre-built containers for ROCm?

stormenergy91 asked Oct 11, 2025 in Q&A · Answered

@stormenergy91 @slaren

@stormenergy91 @slaren

2
You must be logged in to vote

RAM <-> VRAM paging?

Hexorg asked Oct 7, 2025 in Q&A · Unanswered

@Hexorg @Deltrego @0cc4m

@Hexorg @Deltrego @0cc4m

3
You must be logged in to vote

Converting model not working due to Failed to detect model architecure

whackl asked Oct 8, 2025 in Q&A · Unanswered

@whackl

@whackl

0
You must be logged in to vote

Llama-batched-bench shows nice speedsup increasing the number of parallel requests, but doesn't translate to llama-server

Mushoz asked Aug 13, 2025 in Q&A · Unanswered

@Mushoz @ggerganov @Djip007

@Mushoz @ggerganov @Djip007

5
You must be logged in to vote

llama-server Web UI - How do I disable unsafe code from rendering?

TFWol asked Oct 7, 2025 in Q&A · Unanswered

@TFWol

@TFWol

0
You must be logged in to vote

tps slower and slower

sl0sh asked Sep 25, 2025 in Q&A · Unanswered

@sl0sh @JohannesGaessler @abc-nix

@sl0sh @JohannesGaessler @abc-nix

7