Yunjin (云锦)

RSS Topic Aggregator and Feed Reader

Cloud Brocade - Weaving together similar news topics from multiple RSS feeds into coherent, related aggregates.

Project Abandoned

This project is no longer maintained.

After reaching v0.4.0, the decision was made to abandon Yunjin. The core goal — producing high-quality topic aggregation from RSS feeds — proved elusive given the resources (time, compute, and complexity) that I was willing to invest in a hobby project. The clustering and embedding approach works at a basic level, but delivering genuinely useful, high-quality aggregation requires significantly more effort than is practical here.

The code is left here for reference. Feel free to fork it, learn from it, or build on it, but no further development or maintenance is planned.

📦 Version 0.4.0

Phase 1, 2, 3 & 4 Complete - Full web reader interface now available!

Yunjin now has a complete web-based news reader interface with Flask backend and responsive design! View aggregated articles in an organized feed, manage your RSS subscriptions, and track read status through an intuitive browser UI. Feed your RSS subscriptions and let Yunjin group related articles together, then browse them with a modern, colorblind-friendly interface.

Current Capabilities:

• ✅ Multiple RSS feed support with management UI
• ✅ Automatic article clustering by topic
• ✅ Configurable clustering parameters
• ✅ Automatic tag extraction (hashtags & named entities)
• ✅ Media RSS support (automatic image extraction from feeds)
• ✅ Intelligent image/media extraction from articles
• ✅ Tag ignore list management
• ✅ SQLite-based storage
• ✅ Web-based news reader interface
• ✅ Read/unread status tracking
• ✅ Feed management UI
• ✅ Responsive, colorblind-friendly design

What is Yunjin?

Yunjin is an RSS feed aggregator designed to solve information overload. Instead of reading articles individually from multiple feeds, Yunjin automatically groups similar articles together using machine learning embeddings and clustering algorithms.

Key Features

• Automatic Topic Clustering ✅: Groups similar articles using TF-IDF embeddings and DBSCAN
• Multiple Feed Support ✅: Handle unlimited RSS/Atom feeds
• Smart Tags ✅: Extract hashtags and named entities, manage ignore lists
• Media Integration ✅: Captures images from Media RSS feeds and article content
• Minimal JavaScript ✅: Server-side rendering for simplicity
• Pure SQLite ✅: No external dependencies
• Free & Open Source ✅: AGPL-3.0-or-later

Getting Started

Prerequisites

• Python 3.12+
• uv package manager

Installation

# Clone the repository
git clone https://codeberg.org/marvin8/yunjin.git
cd yunjin
# Create virtual environment and install dependencies
uv sync
# Run database initialization
python -m yunjin init
# Add your first RSS feed
python -m yunjin add-feed https://xkcd.com/rss.xml --title "XKCD"
# Fetch articles from all feeds
python -m yunjin fetch
# Cluster articles into aggregates
python -m yunjin aggregate

Usage Examples

Add more feeds:

python -m yunjin add-feed https://news.ycombinator.com/rss --title "Hacker News"
python -m yunjin add-feed https://lwn.net/headlines/rss --title "LWN"

List configured feeds:

python -m yunjin list-feeds

Extract tags and media:

# Extract tags from all articles
python -m yunjin extract-tags
# Extract images from all articles
python -m yunjin extract-media

Cluster with custom parameters:

# Tighter clustering (more clusters, more specific)
python -m yunjin aggregate --eps=0.2
# Looser clustering (fewer clusters, broader topics)
python -m yunjin aggregate --eps=0.5
# Full recalculation from scratch
python -m yunjin aggregate --recalculate

Web Interface

Start the web server:

python -m yunjin.web.wsgi

Then open your browser to http://127.0.0.1:5000

Features:

• Reader: Browse aggregated articles in an organized grid
• Article View: Read full article content with images and tags
• Feed Management: Add/remove RSS feeds through the web UI
• Settings: Manage tag ignore lists
• Read Tracking: Mark articles as read/unread

Development

Quick Start

# Install with development dependencies
uv sync --all-groups
# Run tests
pytest
# Check code quality
ruff check
ty check
# Format code
ruff format

Project Structure

yunjin/
├── src/yunjin/
│ ├── __main__.py # CLI entry point
│ ├── db/ # Database layer (SQLite)
│ ├── models/ # Data models (frozen dataclasses)
│ └── services/ # Business logic (fetcher, aggregator, etc.)
├── tests/ # Test suite
├── README.md # This file
├── Roadmap.md # Detailed implementation roadmap
├── LICENSE # AGPL-3.0-or-later
└── pyproject.toml # Project configuration

Code Standards

• Type Hints: Required for all public functions/methods
• Testing: Comprehensive test coverage with pytest
• Linting: All code must pass ruff check with no warnings
• Type Checking: Must pass ty check validation
• Python Version: 3.12+ features supported

Roadmap

Yunjin is built in phases. See Roadmap.md for detailed implementation plan:

• Phase 1: Foundation & Data Infrastructure ✅ Complete (v0.1.0)
• Phase 2: Aggregation Engine ✅ Complete (v0.2.0)
• Phase 3: Tag & Media Management ✅ Complete (v0.3.0)
• Phase 4: Web Frontend & Reader ✅ Complete (v0.4.0)
• Phase 5: Polish & Optimization (Planned)
• Phase 6: Stretch Goals (Future)

Documentation

• Roadmap.md - Detailed implementation phases and architecture
• docs/PHASE2.md - Phase 2 architecture, usage guide, and tuning
• PHASE2_SUMMARY.md - Phase 2 implementation summary
• PHASE3_SUMMARY.md - Phase 3 implementation summary
• PHASE4_SUMMARY.md - Phase 4 implementation summary
• RELEASE_NOTES_0.2.0.md - Version 0.2.0 release notes

Quality Metrics

• Tests: 141/141 passing (100% pass rate)
• Code Quality: ✅ ruff check (all files), ✅ ty check (all files), ✅ ruff format
• Type Hints: Required on all public functions
• Test Coverage: 25 new tests for Phase 4 web interface

Test Breakdown:

• Phase 1: 40 tests
• Phase 2: 33 tests
• Phase 3: 43 tests
• Phase 4: 25 tests

License

This project is licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later).

See LICENSE for the full license text.

Contributing

This project is abandoned and not accepting contributions.

Repository

• Source Code: https://codeberg.org/marvin8/yunjin
• Issues: https://codeberg.org/marvin8/yunjin/issues
• Documentation: https://marvin8.codeberg.page/yunjin

Yunjin is no longer maintained. The repository is archived for reference.