A high-performance request router for vLLM — smart load balancing, response caching, prefill/decode disaggregation, semantic routing, Anthropic/OpenAI API translation, and operational tooling out of the box.
-
Updated
Aug 1, 2026 - Rust
#
inferencing
Here are 12 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
Testing Function Calling while running llama3.1 locally using ollama
-
Updated
Oct 13, 2024 - Python
Fuzzy Logic Toolbox with GUI and Files Analysis.
python programming gui-application problem-solving artficial-intelligence fuzzy-logic fuzzy-sets pyqt5-desktop-application fuzziness fuzzification inferencing partial-truth degree-of-membership
-
Updated
Feb 9, 2023 - Python
-
Updated
Dec 11, 2021 - Python
Local-first control platform for running, managing, chatting with, monitoring, and benchmarking GGUF language models on your own hardware.
-
Updated
Aug 21, 2026 - Python
Python codes generation from latex expressions. Using synthetic dataset and CodeT5-base model.
transformers postprocessing synthetic-data fine-tuning transformer-models t5-model inferencing large-language-models llm t5-base llm-deployment llm-models latex-to-python
-
Updated
Oct 16, 2024 - Jupyter Notebook
-
Updated
Oct 3, 2025 - Python
Classification of sperm heads based on its morphological quality via Mask-RCNN
deep-learning image-annotation image-processing image-classification image-recognition image-segmentation mask-rcnn sperm-detection inferencing
-
Updated
Sep 29, 2023 - Jupyter Notebook
An LLM inferencing benchmark tool focusing on device-specific latency and memory usage
-
Updated
Jan 24, 2025 - Python
Add this topic to your repo
To associate your repository with the inferencing topic, visit your repo's landing page and select "manage topics."