TravelAI

Transform YouTube travel vlogs into personalized itineraries. TravelAI extracts structured travel data from YouTube videos and generates customized travel plans using AI-powered RAG (Retrieval Augmented Generation).

• YouTube Content Intelligence: Automatically crawls and transcribes travel vlogs using Whisper AI
• Entity Extraction: Extracts places, restaurants, hotels, and activities from travel-video transcripts
• Smart Deduplication: 4-tier canonicalization with geocoding and enrichment
• Semantic Search: FREE local embeddings with ChromaDB vector database
• RAG Itinerary Generation: Natural language queries → personalized day-by-day itineraries
• Cost-Effective: ~0ドル.01-0.02 per itinerary using optimized LLM providers
• Operational Pipeline: S3 data lake, metadata tracking, monitoring, and error handling

# Clone and install
git clone https://github.com/Divish1032/travel-ai-core.git
cd travel-ai-core
./setup.sh
# Configure credentials
cp .env.example .env
# Edit .env with your AWS, YouTube, and LLM API keys

# Stage 1: Crawl YouTube videos
./crawl.sh youtube --input urls.txt
# Stage 2: Extract entities
./crawl.sh process-stage2
# Stage 3: Deduplicate & enrich
./crawl.sh process-stage3
# Stage 4: Generate embeddings
./crawl.sh process-stage4 --embedding-types all
# Stage 5: Generate itinerary
./crawl.sh generate-itinerary -q "5 days Bangkok solo budget party"

# View pipeline status
./crawl.sh status
# Launch dashboard at http://localhost:8501
./crawl.sh dashboard
# Search entities
./crawl.sh search --query "best street food" --city Bangkok

• Python 3.11+
• AWS Account with S3 access
• API Keys:
  • YouTube Data API (crawling)
  • Gemini / OpenAI / DeepSeek (entity extraction, RAG)
  • Optional: Google Maps Geocoding (enrichment fallback)

Interactive Streamlit dashboard for exploring entities and monitoring pipeline:

./crawl.sh dashboard

URL: http://localhost:8501

Features: Video explorer, entity search, analytics, metadata viewer

📚 For complete documentation, see docs/

Quick Links:

• Installation Guide
• Quickstart Tutorial
• System Architecture
• Pipeline Stages
• CLI Commands Reference
• Configuration Guide

• Videos Processed: 117 (all 5 stages complete)
• Entities Extracted: 5,000+ raw → 1,200+ canonical
• Vector Embeddings: 13,000+ indexed in ChromaDB
• Pipeline Cost: ~0ドル.30 total for all data
• Per-Itinerary Cost: ~0ドル.01-0.02

# Generate itinerary from natural language
./crawl.sh generate-itinerary -q "3 days Phuket couple mid-range beach relaxation"
# Output: Day-by-day itinerary with:
# - Personalized recommendations
# - Cost breakdown
# - Travel tips
# - Validated against 1,200+ entities

MIT License - See LICENSE file for details

• Issues: GitHub Issues
• Documentation: docs/
• CLI Reference: COMMANDS.md