llama.cpp usage · ggml-org/llama.cpp · Discussion #7942

nmandic78
Jun 14, 2024

I was wondering how most people use llama.cpp. I mostly use server's OpenAI (semi)compatible endpoint for own projects.

How do you primarily use the llama.cpp?

main (cli) application running in the terminal

40%

server application with a included browser-based UI

20%

server application with own custom-built frontend

40%

server application with a third-party UI

both the CLI and server applications in various configurations

5 votes