Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 2e775d2

Browse files
init
0 parents commit 2e775d2

File tree

7 files changed

+731
-0
lines changed

7 files changed

+731
-0
lines changed

‎README.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# 🧠 MindsDB KB Manager
2+
3+
A simple and extensible project to manage MindsDB Knowledge Bases — create, ingest, query, summarize, and interact via agents. Includes a CLI and optional terminal-style UI demo.
4+
5+
---
6+
7+
## 📌 Overview
8+
9+
This project streamlines core operations on MindsDB Knowledge Bases:
10+
11+
- 📁 **Create** KBs with embedding and metadata configurations
12+
- 📊 **Ingest** CSV/text data into KBs
13+
- 🔍 **Query** KBs using natural language, with or without metadata filters
14+
- 🧠 **Summarize** KB content via GPT-3.5
15+
- 🤖 **Create AI agents** connected to specific KBs
16+
- 💬 **Chat** with agents using natural questions
17+
18+
MindsDB acts as the orchestration engine for vector indexing, OpenAI integration, and agent execution.
19+
20+
---
21+
22+
## YouTube Demo : https://www.youtube.com/watch?v=K-cufogVz0Q
23+
## Blog Post :
24+
25+
---
26+
27+
## ⚙️ Setup
28+
29+
### 0. Clone the Repository
30+
31+
```bash
32+
git clone github.com/pheonix-coder/kb-manager.git
33+
cd mindsdb-kb-manager
34+
```
35+
36+
### 1. MindsDB and Ollama Setup
37+
38+
Setup MindsDB and Ollama with Docker (docker-compose.yml is provided):
39+
40+
```bash
41+
docker-compose up -d
42+
```
43+
44+
Install nomic-embed-text or any other embedding model in Ollama container:
45+
46+
```bash
47+
docker exec ollama ollama pull nomic-embed-text
48+
```
49+
50+
### 2. MindsDB SQL Prerequisites
51+
52+
Run these in the MindsDB SQL editor:
53+
54+
```sql
55+
-- OpenAI Engine
56+
CREATE ML_ENGINE openai_engine
57+
FROM openai
58+
USING
59+
openai_api_key = "your_openai_api_key";
60+
61+
-- Summarizer model
62+
CREATE MODEL kb_summarizer
63+
PREDICT summary
64+
USING
65+
engine = 'openai_engine',
66+
model_name = 'gpt-3.5-turbo',
67+
prompt_template = 'Provide a concise summary of the following knowledge base content and highlight the main insights:\n\n{{kb_content}}\n\nSummary:';
68+
69+
-- Vector backend
70+
CREATE DATABASE pvec
71+
WITH
72+
ENGINE = 'pgvector',
73+
PARAMETERS = {
74+
"host": "pgvector",
75+
"port": 5432,
76+
"database": "ai",
77+
"user": "ai",
78+
"password": "ai",
79+
"distance": "cosine"
80+
};
81+
````
82+
83+
### 2. Python Setup
84+
85+
```bash
86+
# Create virtual environment
87+
uv venv
88+
source .venv/bin/activate
89+
90+
# Install dependencies
91+
uv pip install -r requirements.txt
92+
```
93+
94+
---
95+
96+
## 🧪 Usage
97+
98+
### CLI
99+
100+
```bash
101+
python main.py --help
102+
```
103+
104+
Example:
105+
106+
```bash
107+
# Create a new KB
108+
python main.py init-kb --name quotes_kb --model-name nomic-embed-text ...
109+
110+
# Ingest data
111+
python main.py ingest-kb --kb-name quotes_kb --file-path "data/quotes.csv"
112+
113+
# Query KB
114+
python main.py query-kb --kb-name quotes_kb --query "inspiration" --relevance 0.3
115+
116+
# Summarize KB
117+
python main.py summarize-kb --kb-name quotes_kb --query "life insights"
118+
119+
# Create & chat with agent
120+
python main.py create-agent --agent-name quote_bot --knowledge-bases quotes_kb ...
121+
python main.py chat-agent --agent-name quote_bot --question "What did Einstein say about logic?"
122+
```
123+
124+
---
125+
126+
## 💡 Use Cases
127+
128+
* ✅ Quickly prototype KBs and agents without building full UI
129+
* ✅ Run automated ingestion and querying pipelines for AI use cases
130+
* ✅ Summarize large KBs into digestible formats
131+
* ✅ Validate MindsDB functionality in real-world agent workflows
132+
133+
---
134+
135+
## 🧠 Powered By
136+
137+
* [MindsDB](https://mindsdb.com/)
138+
* [OpenAI GPT-3.5](https://platform.openai.com/)
139+
* [pgvector](https://github.com/pgvector/pgvector)
140+
* [Typer](https://typer.tiangolo.com/)

‎core/agent_ops.py‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
"""
2+
Agent Operations for KB manager.
3+
4+
This module provides functionality to manage agents in MindsDB, including creation, deletion, and querying.
5+
"""
6+
7+
from typing import Literal, Optional
8+
from pydantic import BaseModel
9+
from mindsdb_sdk import connect
10+
11+
class ModelConfig(BaseModel):
12+
provider: Literal["openai"] | Literal["anthropic"] | Literal["google"]
13+
model_name: str
14+
api_key: str
15+
system_prompt: str
16+
17+
class AgentOperations:
18+
def __init__(self):
19+
"""Initialize connection to MindsDB server."""
20+
self.server = connect()
21+
22+
def create_agent(self, agent_name: str, model_config: ModelConfig, knowledge_bases: Optional[list[str]] = None, project_name: str = "mindsdb") -> None:
23+
"""
24+
Create an agent with the specified configuration and associated knowledge bases.
25+
26+
Args:
27+
agent_name: The name of the agent to create.
28+
model_config: Configuration for the model including provider, model name, API key, and system prompt.
29+
knowledge_bases: Optional list of knowledge base names to associate with the agent.
30+
project_name: The name of the project associated with the agent. Defaults to 'mindsdb'.
31+
"""
32+
if not knowledge_bases:
33+
raise Exception("Can't create agent without knowledge bases")
34+
35+
query = f"""
36+
CREATE AGENT {project_name}.{agent_name}
37+
USING
38+
model = '{model_config.model_name}',
39+
{model_config.provider}_api_key = '{model_config.api_key}',
40+
include_knowledge_bases = {knowledge_bases},
41+
prompt_template = '{model_config.system_prompt}';
42+
"""
43+
self.server.query(query).fetch()
44+
45+
def delete_agent(self, agent_name: str, project_name: str = "mindsdb") -> None:
46+
"""
47+
Delete an agent by its name.
48+
49+
Args:
50+
agent_name: The name of the agent to delete.
51+
project_name: The name of the project associated with the agent. Defaults to 'mindsdb'.
52+
"""
53+
query = f"DROP AGENT {project_name}.{agent_name};"
54+
self.server.query(query).fetch()
55+
56+
def query_agent(self, agent_name: str, question: str, project_name: str = "mindsdb") -> str:
57+
"""
58+
Query an agent with a specific question and return the answer.
59+
60+
Args:
61+
agent_name: The name of the agent to query.
62+
question: The question to ask the agent.
63+
project_name: The name of the project associated with the agent. Defaults to 'mindsdb'.
64+
65+
Returns:
66+
The answer from the agent as a string.
67+
"""
68+
query = f"""
69+
SELECT answer
70+
FROM {project_name}.{agent_name}
71+
WHERE question = '{question.replace("'", "\\'")}';
72+
"""
73+
try:
74+
result = self.server.query(query).fetch()
75+
return result["answer"].iloc[0] if not result.empty else ""
76+
except RuntimeError as e:
77+
if "Event loop is closed" in str(e):
78+
# Retry once if event loop is closed
79+
result = self.server.query(query).fetch()
80+
return result["answer"].iloc[0] if not result.empty else ""
81+
else:
82+
raise

‎core/file_ops.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
"""
2+
File Operations for KB manager.
3+
4+
This module provides functionality to interact with files in MindsDB, including uploading, listing, deleting, and retrieving files.
5+
"""
6+
7+
from pandas import DataFrame
8+
from mindsdb_sdk import connect
9+
10+
class FileOperations:
11+
def __init__(self):
12+
"""Initialize connection to MindsDB server and access files database."""
13+
self.server = connect()
14+
self.files_db = self.server.get_database("files")
15+
16+
def upload_file(self, file_name: str, df: DataFrame, replace: bool = False) -> None:
17+
"""
18+
Upload a DataFrame as a file to the 'files' database in MindsDB.
19+
20+
Args:
21+
file_name: The name of the file to create in the database.
22+
df: The pandas DataFrame containing the data to store as a file.
23+
replace: If True, replaces an existing file with the same name. Defaults to False.
24+
"""
25+
self.files_db.create_table(name=file_name, query=df, replace=replace)
26+
27+
def list_files(self, name: str = "") -> list:
28+
"""
29+
List all files in the 'files' database that start with the given name.
30+
31+
Args:
32+
name: The prefix to filter files by name. If empty, lists all files.
33+
34+
Returns:
35+
A list of file names from the 'files' database matching the given prefix.
36+
"""
37+
query = f"""
38+
SHOW TABLES FROM files
39+
WHERE Tables_in_files LIKE '{name}%'
40+
"""
41+
result = self.server.query(query).fetch()
42+
return list(result["Tables_in_files"])
43+
44+
def get_file(self, file_name: str) -> DataFrame:
45+
"""
46+
Retrieve a file from the 'files' database by its name.
47+
48+
Args:
49+
file_name: The name of the file to retrieve from the database.
50+
51+
Returns:
52+
A pandas DataFrame containing the data of the specified file.
53+
"""
54+
query = f"SELECT * FROM files.`{file_name}`"
55+
return self.server.query(query).fetch()
56+
57+
def delete_file(self, file_name: str) -> None:
58+
"""
59+
Delete a file from the 'files' database by its name.
60+
61+
Args:
62+
file_name: The name of the file to delete from the database.
63+
"""
64+
query = f"DROP TABLE files.`{file_name}`"
65+
self.server.query(query).fetch()

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /