- To run, first set variable for the remote location of the offline host path, like this:
export HOST_OFFLINE_PATH="/Volumes/Offline LLM/.ollama/models" # or whatever the external drive location might be.
- After that, issue
docker compose up -d --build
- Then go to localhost:8080, go to settings and set the connection to
http://localhost:11434/v1.