HybridTM - Semantic Translation Memory Engine

Highlights

• Imports XLIFF 2.x, TMX 1.4b, and SDLTM files, preserving metadata, notes, and custom properties
• Generates semantic vectors with any Hugging Face Transformers.js-compatible text model — ships three ready-to-use presets (compact, standard, large) and also accepts any other Hugging Face feature-extraction model id; default is HybridTM.LARGE_MODEL (gte-multilingual-base)
• Provides semanticTranslationSearch, semanticSearch, and concordanceSearch APIs with metadata-aware filtering
• Streams data into LanceDB through a JSONL-based batch importer to keep memory usage predictable
• Prevents duplicate segments by rewriting entries with deterministic IDs (fileId:unitId:segmentIndex:lang)
• Backs up any instance to a single format-agnostic XML file and restores it into an existing or brand-new instance — optionally under a different embedding model — making it easy to move data to another computer, switch models, or simply keep a backup
• Ships a hybridtm command-line tool for creating instances, importing files, backing up and restoring data, and enriching XLIFF files with TM match candidates from the shell
• Runs as a JSON-over-HTTP server for CAT tools and editor integrations, keeping an instance and its loaded embedding model open across many requests with ticketed handling for long-running imports and searches

Models download automatically the first time you initialize an instance and are cached in the standard Hugging Face directory.

The three embedding presets trade off footprint against accuracy differently depending on language: standard is a good default for Western European languages and matches or beats large on accuracy while being faster; large is worth its extra cost for morphologically rich languages such as Finnish; compact has the smallest footprint but should be treated as a deliberate trade-off rather than a safe default.

Requirements

• Node.js 24 LTS or later
• npm 11+
• Disk space for both the LanceDB directory you choose and the embedding model cache