Ollama MCP Agent allows you to use LLM models locally on your PC for free. Using Ollama's locally installed LLM models along with MCP (Model Context Protocol) additional features, you can easily extend LLM functionality.

• contains Ollama(main.py), Gemini(gemini.py) example
• Inspired by: Teddynote-lab's mcp agents, langchain mcp adapters
• Contributor: odeothx

• Run LLM models locally on your PC (no additional costs)
• Extend LLM capabilities through MCP
• Streaming response output
• Tool call information monitoring

• Python 3.12 or higher
• Ollama installation
• uv - Fast Python package installer and resolver
• MCP server (optional)

1. Clone repository

git clone https://github.com/godstale/ollama-mcp-agent
cd ollama-mcp-agent

2. Install uv (if not installed)

# Using pip
pip install uv
# Or using curl (Unix-like systems)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Or using PowerShell (Windows)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

3. Create virtual environment and install dependencies

# Create and activate virtual environment
uv venv
source .venv/bin/activate # For Unix-like systems
# Or
.venv\Scripts\activate # For Windows
# Install dependencies
uv sync

4. Install Ollama and download model

# Install Ollama (refer to https://ollama.ai for platform-specific installation)
# Download LLM model which supports Tool calling feature
ollama pull MFDoom/deepseek-r1-tool-calling:14b

You can extend LLM functionality through the MCP configuration file. You can implement MCP servers directly in Python or add MCP servers found on smithery.ai. Add settings to the mcp_config.json file:

{
 "mcpServers": {
 "weather": {
 "command": "python",
 "args": ["./mcp_server/mcp_server_weather.py"],
 "transport": "stdio"
 },
 "fetch": {
 "command": "npx",
 "args": [
 "-y",
 "@smithery/cli@latest",
 "run",
 "@smithery-ai/fetch",
 "--key",
 "your_unique_uuid"
 ]
 }
 }
}

Basic execution:

uv run main.py

With options:

uv run main.py --temp 0.7 --timeout 300 --show-tools

Ollama MCP Agent now supports Google's Gemini model as an alternative to Ollama. To use Gemini:

1. Set up Google API Key

# Create .env file and add your Google API key
echo GOOGLE_API_KEY=your_google_api_key_here > .env
# Or set environment variable directly
export GOOGLE_API_KEY=your_google_api_key_here # For Unix-like systems
# Or
set GOOGLE_API_KEY=your_google_api_key_here # For Windows

2. Run with Gemini

uv run gemini.py

• --temp: Set temperature value (0.0 ~ 1.0, default: 0.5)
• --system-prompt: Set custom system prompt
• --timeout: Response generation timeout (seconds, default: 300)
• --show-tools: Display tool call information

• Requires valid Google API key
• Uses Gemini 1.5 Flash model by default
• Supports all MCP tools like the Ollama version
• Streaming responses are enabled by default

• --temp: Set temperature value (0.0 ~ 1.0, default: 1.0)
• --system-prompt: Set system prompt
• --timeout: Response generation timeout (seconds, default: 300)
• --show-tools: Display tool call information

• main.py: Main application file
• mcp_manager.py: MCP client management
• mcp_config.json: MCP server configuration file

1. Add new MCP server and tools to mcp_config.json
2. Implement and run MCP server
3. Restart application

Refer to smithery.ai to add and use various MCP servers.

Enter one of the following commands to exit the program:

• quit
• exit
• bye

• Basic LLM functionality works even without MCP server configuration
• Response speed may vary depending on local PC performance
• Be mindful of memory usage (especially when using large LLM models)

MIT License