MAITRI is a fully local Retrieval-Augmented Generation (RAG) AI companion designed for emotionally intelligent, context-aware conversations while preserving user privacy.

The project combines a local Large Language Model (Qwen), semantic retrieval using ChromaDB, long-term conversational memory, and structured prompt engineering to create an AI assistant capable of maintaining context across conversations without relying on cloud services.

Status: 🚧 Active Development

• 🔒 Fully Offline AI Assistant
• 🧠 Retrieval-Augmented Generation (RAG)
• 💬 Long-Term Conversational Memory
• 📚 Semantic Search using ChromaDB
• ⚡ Streaming Token Generation
• 🚀 GPU Accelerated Embeddings (CUDA)
• 📖 Token-Aware Document Chunking
• 📝 Session Logging using JSONL
• 🧩 Modular Pipeline Architecture
• ❤️ Emotion-Aware Prompt Engineering
• 🔍 Semantic Memory Retrieval
• 🛡️ Crisis Detection & Safety Handling

User
 │
 ▼
Maitri Chat
 │
 ▼
Analyst Prompt
 │
 ▼
Retriever
 │
 ├──────────────► ChromaDB
 │ │
 │ ▼
 │ Relevant Memories
 │
 ▼
Context Builder
 │
 ▼
Qwen Local LLM
 │
 ▼
Streaming Response
 │
 ▼
Conversation Logger
 │
 ├────────► JSONL History
 │
 └────────► Session Summary
 │
 ▼
 Memory Indexer
 │
 ▼
 ChromaDB

.
├── builder.py
├── clean_memory.py
├── config.py
├── data
├── embedding
│ ├── embedder.py
│ └── model.py
├── history
├── LLM
│ └── llm_engine.py
├── loader.py
├── maitri_chat.py
├── models.py
├── pipeline
│ ├── indexing.py
│ ├── querying.py
│ └── rag_pipeline.py
├── retrival
│ ├── context_builder.py
│ ├── router.py
│ └── search_service.py
├── src
│ └── chunker.py
├── system_prompt.py
├── vectordb
│ ├── base.py
│ ├── chroma_store.py
│ └── retriever.py
└── vector_db
 └── chroma.sqlite3

1. User sends a message.
2. The Analyst prompt classifies the conversation.
3. Relevant memories are retrieved from ChromaDB.
4. Context Builder prepares the final prompt.
5. Qwen generates a streamed response.
6. Conversation is logged.
7. Session summaries are periodically indexed into the vector database for future retrieval.

Responses are streamed token-by-token instead of waiting for the entire generation to finish.

Past conversations are summarized, embedded, and stored inside ChromaDB, allowing MAITRI to retrieve relevant memories using semantic similarity.

Documents are split according to tokenizer limits instead of arbitrary word counts, improving retrieval quality.

Conversation history uses append-only JSONL storage for scalable and efficient logging.

Sentence embeddings automatically utilize CUDA when available.

• High inference latency depending on hardware
• Memory summarization can still be improved
• Prompt engineering is continuously evolving
• Multi-user support is not yet implemented
• No graphical user interface (CLI only)

• Reduce inference latency
• Hybrid retrieval (BM25 + Vector Search)
• Memory ranking and decay
• Function calling
• Voice interaction
• Web interface
• Agentic workflow
• Multi-modal support
• Quantized model benchmarking
• Docker deployment

Clone the repository

git clone https://github.com/<your_username>/<repository>.git
cd <repository>

Create a virtual environment

python -m venv venv

Activate it

Linux

source venv/bin/activate

Install dependencies

pip install -r requirements.txt

Start Ollama

ollama serve

Pull Qwen if necessary

ollama pull qwen2.5:7b

Run the chatbot

python maitri_chat.py

Current observations

This project was created to explore Retrieval-Augmented Generation, prompt engineering, semantic memory, and local Large Language Models while building an AI companion that prioritizes privacy and contextual understanding.

The primary objective is educational: to gain hands-on experience designing scalable AI systems rather than relying solely on existing frameworks.

This project is licensed under the MIT License.

Vicky

Bachelor's Student — Artificial Intelligence & Machine Learning

Interested in:

• Artificial Intelligence
• Retrieval-Augmented Generation
• Robotics
• Computer Vision
• Deep Learning
• Large Language Models