Tests Coverage Version Python License
Extensible Machine Translation Providers Framework
- MT Providers
MT Providers is a unified framework for machine translation services that allows easy integration of different translation providers through a consistent interface. Whether you're building a small application or a large-scale translation service, MT Providers provides the flexibility and reliability you need.
- π Plugin Architecture: Easy integration of new translation providers
- π Async Support: Full async/await support with sync fallback
- π Batch Processing: Efficient bulk translation capabilities
- π‘οΈ Rate Limiting: Built-in rate limiting and retry logic
- π― Type Safety: Complete type annotations with mypy support
- π₯ Health Checks: Provider health monitoring and status checking
- π·οΈ Smart User-Agent: Automatic provider identification with custom override
- π Comprehensive Logging: Structured logging throughout
- π§ͺ Well Tested: High test coverage and quality assurance
# Install the framework pip install mt-providers # Install providers (example with Microsoft Translator) pip install mt_provider_microsoft
from mt_providers import get_provider from mt_providers.types import TranslationConfig # Configure your provider config = TranslationConfig( api_key="your-api-key", region="westus2" # Required for some providers like Microsoft ) # Get a provider instance translator = get_provider("microsoft")(config) # Translate text result = translator.translate("Hello world", "en", "es") print(result["translated_text"]) # Output: Β‘Hola mundo! # Batch translation texts = ["Hello", "World", "How are you?"] results = translator.bulk_translate(texts, "en", "es") for result in results: print(result["translated_text"])
import asyncio async def async_translation(): result = await translator.translate_async("Hello world", "en", "es") print(result["translated_text"]) asyncio.run(async_translation())
| Provider | Package | Status | Features |
|---|---|---|---|
| Microsoft Translator | mt_provider_microsoft |
β Available | Sync/Async, Batch, Rate Limiting |
| DeepL | mt_provider_deepl |
β Available | Sync/Async, Batch, Rate Limiting |
from mt_providers import list_providers, discover_providers # List all available providers providers = list_providers() print(f"Available providers: {providers}") # ['microsoft'] # Discover providers from entry points discovered = discover_providers() print(f"Discovered providers: {discovered}") # Check provider health from mt_providers.registry import check_provider_health is_healthy = await check_provider_health("microsoft", config) print(f"Provider healthy: {is_healthy}")
from mt_providers.types import TranslationConfig config = TranslationConfig( api_key="your-api-key", region="your-region", # Provider-specific timeout=30, # Request timeout in seconds rate_limit=10, # Requests per second retry_attempts=3, # Number of retries retry_backoff=1.0, # Backoff multiplier user_agent_name="MyApp", # Optional: Custom UA name user_agent_version="1.0" # Optional: Custom UA version (required if name is set) )
import os config = TranslationConfig( api_key=os.getenv("TRANSLATION_API_KEY"), region=os.getenv("TRANSLATION_REGION", "westus2"), rate_limit=int(os.getenv("TRANSLATION_RATE_LIMIT", "10")), user_agent_name=os.getenv("APP_NAME"), # Optional: For application identification user_agent_version=os.getenv("APP_VERSION") # Optional: Application version )
User-Agent Notes:
- If not specified, providers automatically use their package name and version
- Set both
user_agent_nameanduser_agent_versionfor custom identification - Useful for partner integrations or application-specific tracking
from mt_providers.exceptions import ProviderNotFoundError, ConfigurationError from mt_providers.types import TranslationStatus try: translator = get_provider("microsoft")(config) result = translator.translate("Hello", "en", "es") # Check result status - all errors are returned in the response if result["status"] == TranslationStatus.SUCCESS: print(f"Translation: {result['translated_text']}") else: print(f"Translation failed: {result['error']}") except ProviderNotFoundError: print("Provider not found in registry") except ConfigurationError: print("Provider configuration error")
# Configure rate limiting config = TranslationConfig( api_key="your-key", region="westus2", rate_limit=100 # 100 requests per second ) # Rate limiting is automatically applied results = translator.bulk_translate(large_text_list, "en", "es")
from mt_providers import BaseTranslationProvider from mt_providers.types import TranslationResponse class MyProvider(BaseTranslationProvider): name = "my_provider" def translate(self, text: str, source_lang: str, target_lang: str) -> TranslationResponse: # Your implementation here return self._create_response( translated_text="translated text", source_lang=source_lang, target_lang=target_lang, char_count=len(text) )
Register via entry points in pyproject.toml:
[project.entry-points."mt_providers"] my_provider = "my_package.module:MyProvider"
Comprehensive documentation is available to help you get started and master the framework:
- π API Reference - Complete API documentation for all classes and methods
- π§ Provider Integration Guide - Step-by-step guide for creating custom providers
- π Usage Examples - Practical examples and tutorials for common use cases
- βοΈ Configuration Guide - Detailed configuration options and best practices
- ποΈ Architectural Decisions - Design decisions and framework architecture
- Getting Started Tutorial
- Creating Your First Provider
- Configuration Examples
- Error Handling Patterns
- Performance Optimization
The framework includes comprehensive test coverage ensuring reliability and stability:
# Run all tests pytest # Run with coverage report pytest --cov=mt_providers --cov-report=html # Run specific test categories pytest tests/test_integration.py # Integration tests pytest tests/test_provider.py # Provider tests pytest tests/test_registry.py # Registry tests # Type checking mypy mt_providers # Code formatting and linting black mt_providers isort mt_providers
- Total Tests: 27 tests covering all major functionality
- Coverage: 77% code coverage (207 statements, 48 uncovered)
- Test Categories: Unit, Integration, Performance, and Regression tests
- Continuous Integration: Automated testing on multiple Python versions
MT Providers is optimized for high-performance translation workloads with enterprise-grade capabilities:
- β‘ Fast Provider Discovery: Sub-millisecond provider lookup and instantiation
- π¦ Efficient Batch Processing: Automatic request chunking and parallel processing
- π Connection Pooling: HTTP connection reuse for reduced latency
- β‘ Async Support: Non-blocking operations with asyncio for maximum concurrency
- π‘οΈ Smart Rate Limiting: Configurable rate limiting that respects API quotas
- π Performance Monitoring: Built-in benchmarking and health check capabilities
- π Memory Efficient: Optimized memory usage for large-scale deployments
- Provider discovery: ~0.005 seconds for full registry scan
- Provider instantiation: <0.001 seconds average
- Memory footprint: Minimal overhead per provider instance
- Concurrent operations: Scales linearly with available resources
- Python: 3.8+ (3.10+ recommended for optimal performance)
- Type Checking: Full mypy support with comprehensive type annotations
- Async: Native asyncio compatibility with automatic sync fallback
- Frameworks: Seamless integration with Flask, Django, FastAPI, Starlette
- Platforms: Cross-platform support (Linux, macOS, Windows)
- Packaging: Standard Python packaging with wheel and source distributions
We welcome contributions! The MT Providers framework is designed to be extensible and community-driven.
- Fork the repository and create your feature branch
- Add comprehensive tests for new functionality
- Update documentation for any API changes
- Ensure all tests pass and maintain code coverage
- Follow code style guidelines (black, isort, mypy)
- Submit a pull request with a clear description
- π New translation provider implementations
- π Documentation improvements and examples
- π§ͺ Additional test coverage and edge cases
- β‘ Performance optimizations and benchmarks
- π Bug fixes and issue resolution
# Clone the repository git clone https://github.com/assystant/mt-providers.git cd mt-providers # Create virtual environment python -m venv .venv source .venv/bin/activate # or `.venv\Scripts\activate` on Windows # Install development dependencies pip install -e .[test,dev] # Run tests pytest # Run type checking mypy mt_providers # Format code black mt_providers isort mt_providers
mt_providers/
βββ mt_providers/ # Core framework code
β βββ base.py # Base provider class
β βββ registry.py # Provider discovery and registration
β βββ types.py # Type definitions
β βββ exceptions.py # Custom exceptions
β βββ utils.py # Utility functions
βββ tests/ # Comprehensive test suite
βββ docs/ # Documentation
βββ examples/ # Usage examples
This project is licensed under the MIT License - see the LICENSE file for details.
- π§ Email: support@assystant.com
- π Issues: GitHub Issues - Report bugs and request features
- π¬ Discussions: GitHub Discussions - Ask questions and share ideas
- π Documentation: Comprehensive guides available in the docs/ directory
- π― Examples: Practical examples in docs/usage_examples.md
See CHANGELOG.md for detailed information about changes in each version.
- β Core Framework: Complete with plugin architecture
- β Microsoft Translator: Full implementation with async support
- β Documentation: Comprehensive guides and API reference
- β Testing: 77% coverage with 27 tests
- β Type Safety: Full mypy compliance
- β Performance: Optimized for production workloads
- π§ Caching Layer: Redis/Memcached response caching
- π§ Translation Memory: Integration with TMX files
- π§ Streaming Support: Real-time translation capabilities
- π§ Metrics & Observability: Detailed performance monitoring
- π§ CLI Tools: Command-line interface for batch operations
- π§ Additional Providers: Google Translate, Amazon Translate, and other services