local-llms

A small project for a quickstart into running local LLMs.

Client

Prerequisites

• flatpak
• flatpak-builder

Building

$ cd client/
$ ./build.sh

Running

$ flatpak run ai.lmstudio.LMStudio

Server

For this example, podman is being used. docker can be used in place if you wish to use it instead.

Prerequisites

• podman
• podman-compose

Or:

• docker
• docker-compose

Running & Configuring SearXNG

Firstly, start the container.

$ cd servers/
$ podman compose up -d

Then enter it and edit /etc/searxng/settings.yml

$ podman compose exec search /bin/sh
(container) $ vi /etc/searxng/settings.yml

Make sure json is a permitted format

search:formats:- html- json

Restart the container

$ podman compose restart

Important

You may have to configure your firewall to permit access to port 8080 from the LM Studio flatpak.

Configuring LM Studio

In LM Studio go to the Developer tab (or press ctrl+2) and open mcp.json. Paste the following in:

{
 "mcpServers": {
 "searxng": {
 "command": "npx",
 "args": [
 "-y",
 "mcp-searxng"
 ],
 "env": {
 "SEARXNG_URL": "http://localhost:8080"
 }
 }
 }
}

You can enable the SearXNG integration by clicking on the "🔨" icon at the bottom left of the input area.