Pinned Discussions

• ggerganov

  Inference at the edge

  📣 Announcements · ggerganov
• ggerganov

  llama.cpp supports the new gpt-oss model in native MXFP4 format

  📣 Announcements · ggerganov
• ggerganov

  guide : running gpt-oss with llama.cpp

  🙌 Show and tell · ggerganov

Pinned to Ideas

• 13 You must be logged in to vote

  💡

  GPU optimization across different cards

  performance Speed related topics hardware Hardware related

  JohannesGaessler started May 13, 2023 in Ideas

  16

Discussions

• 1 You must be logged in to vote

  💡

  I hope the upcoming new version can support images, audio, and even video.

  python156 started Oct 26, 2025 in Ideas

  @python156

  @python156

  0
• 9 You must be logged in to vote

  💡

  Add support for different reasoning fields

  aldehir started Aug 16, 2025 in Ideas

  @aldehir @mostlygeek @ggerganov

  @aldehir @mostlygeek @ggerganov

  13
• 3 You must be logged in to vote

  💡

  Expand user message textarea height while editing in Web UI

  nihilfacit started Oct 23, 2025 in Ideas

  @nihilfacit @dwrz

  @nihilfacit @dwrz

  1
• 61 You must be logged in to vote

  💡

  [FEEDBACK] Better packaging for llama.cpp to support downstream consumers 🤗

  Vaibhavs10 started Aug 14, 2025 in Ideas

  @Vaibhavs10 @deadprogram @simonw

  @Vaibhavs10 @deadprogram @simonw

  104
• 5 You must be logged in to vote

  💡

  Speculative decoding potential for running big LLMs on consumer grade GPUs efficiently

  steampunque started Nov 23, 2024 in Ideas

  @steampunque @ggerganov @Djip007

  @steampunque @ggerganov @Djip007

  69
• 1 You must be logged in to vote

  💡

  Automated conversion from HF model definition?

  am17an started Oct 24, 2025 in Ideas

  @am17an

  @am17an

  0
• 8 You must be logged in to vote

  💡

  Split model between NUMA nodes for CPU inference

  CommanderLake started Mar 10, 2025 in Ideas

  @CommanderLake @usrlocalben @ggerganov

  @CommanderLake @usrlocalben @ggerganov

  34
• 1 You must be logged in to vote

  💡

  extremely lightweight version | just the main interference + server

  dlmsf started Oct 23, 2025 in Ideas

  @dlmsf

  @dlmsf

  0
• 1 You must be logged in to vote

  💡

  Proposal: a single Zip in Releases

  saltyJeff started Oct 23, 2025 in Ideas

  @saltyJeff

  @saltyJeff

  0
• 3 You must be logged in to vote

  💡

  Any hope for DeepSeek v3 MTP support?

  Green0-0 started Jan 27, 2025 in Ideas

  @Green0-0 @lippman1125 @BarfingLemurs

  @Green0-0 @lippman1125 @BarfingLemurs

  2
• 1 You must be logged in to vote

  💡

  idea: nanochat support

  twobombs started Oct 22, 2025 in Ideas

  @twobombs @am17an

  @twobombs @am17an

  1
• 1 You must be logged in to vote

  💡

  Improve DLL/shared library stability by replacing static initializers

  gianmichelebertoni started Oct 22, 2025 in Ideas

  @gianmichelebertoni

  @gianmichelebertoni

  0
• 1 You must be logged in to vote

  💡

  DJGPP port?

  DartPower started Oct 21, 2025 in Ideas

  @DartPower

  @DartPower

  0
• 0 You must be logged in to vote

  💡

  I’ve been working on a Unity-based AI avatar project that uses a local inference backend on mobile devices. Here’s a short demo:

  hazama714 started Oct 17, 2025 in Ideas

  @hazama714

  @hazama714

  0
• 1 You must be logged in to vote

  💡

  Discuss how to design omni model

  tc-mb started Oct 13, 2025 in Ideas

  @tc-mb @ggerganov @ngxson

  @tc-mb @ggerganov @ngxson

  7
• 1 You must be logged in to vote

  💡

  Q4_0 quantization support for mmproj

  tdakhran started Aug 20, 2025 in Ideas

  @tdakhran @CISC @ngxson

  @tdakhran @CISC @ngxson

  11
• 1 You must be logged in to vote

  💡

  Round-robin model layer loading when only performing prefill on a given node

  mrebersv started Oct 18, 2025 in Ideas

  @mrebersv

  @mrebersv

  0
• 1 You must be logged in to vote

  💡

  Suggestion: Clean up /image output in llama-mtmd-cli.exe

  apipino started Oct 16, 2025 in Ideas

  @apipino

  @apipino

  0
• 4 You must be logged in to vote

  💡

  [CANN] Any possibility for the Unsloth dynamic quant of R1 to work on Ascend cards?

  Dango233 started Feb 6, 2025 in Ideas

  @Dango233 @hipudding @cecuca

  @Dango233 @hipudding @cecuca

  2
• 1 You must be logged in to vote

  💡

  do you can add type format file AI?

  Ricookies707 started Oct 14, 2025 in Ideas

  @Ricookies707

  @Ricookies707

  0
• 1 You must be logged in to vote

  💡

  Llama multi-node through custom thunderbolt4 rdma driver

  Geramy started Oct 11, 2025 in Ideas

  @Geramy

  @Geramy

  0
• 1 You must be logged in to vote

  💡

  wstring_convert is deprecated in C++17, and will be removed in C++26

  clin1234 started Oct 10, 2025 in Ideas

  @clin1234

  @clin1234

  0
• 1 You must be logged in to vote

  💡

  Add Q4_0 and Q8_0 quantization support for 310p npu Ascend CANN

  cecuca started Oct 9, 2025 in Ideas

  @cecuca

  @cecuca

  0
• 1 You must be logged in to vote

  💡

  Can we get one RPC server per node instead of one RPC server per GPU

  segmond started Oct 6, 2025 in Ideas

  @segmond @rgerganov @abc-nix

  @segmond @rgerganov @abc-nix

  3
• 1 You must be logged in to vote

  💡

  Add timestamps to llama-server logging

  nialllo started Oct 5, 2025 in Ideas

  @nialllo @ggerganov

  @nialllo @ggerganov

  1