Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

v3.5.0 #6195

mudler announced in Announcements
v3.5.0 #6195
Sep 3, 2025 · 0 comments
Discussion options




🚀 LocalAI 3.5.0

Welcome to LocalAI 3.5.0! This release focuses on expanding backend support, improving usability, refining the overall experience, and keeping reducing footprint of LocalAI, to make it a truly portable, privacy-focused AI stack. We’ve added several new backends, enhanced the WebUI with new features, made significant performance improvements under the hood, and simplified LocalAI management with a new Launcher app (Alpha) available for Linux and MacOS.

TL;DR – What’s New in LocalAI 3.5.0 🎉

  • 🖼️ Expanded Backend Support: Welcome to MLX! mlx, mlx-audio, mlx-vlm are now all available in LocalAI. We also added support to WAN for video generation, and a CPU and MPS version of the diffusers backend! Now you can generate and edit images from MacOS or if you don't have any GPU (albeit slow).
  • ✨ WebUI Enhancements: Download model configurations, a manual model refresh button, streamlined error streaming during SSE events, and a stop button for running backends. Models now can also be imported and edited via the WebUI.
  • 🚀 Performance & Architecture: Whisper backend has been rewritten in Purego with integrated Voice Activity Detection (VAD) for improved efficiency and stability. Stablediffusion also benefits from the Purego conversion.
  • 🛠️ Simplified Management: New LocalAI Launcher App (Alpha) for easy installation, startup, updates, and access to the WebUI.
  • ✅ Bug Fixes & Stability: Resolutions to AMD RX 9060XT ROCm errors, libomp linking issues, model loading problems on macOS, CUDA device detection improvements, and more.
  • Enhanced support for MacOS: whisper, diffusers, llama.cpp, MLX (VLM, Audio, LLM), stable-diffusion.cpp will now work on MacOS!

What’s New in Detail

🚀 New Backends and Model Support

We've significantly expanded the range of models you can run with LocalAI!

  • mlx-audio: Bring text to life with Kokoro’s voice models on MacOS with the power of MLX!. Install with the mlx-audio backend. Example configuration: 
    backend: mlx-audio
name: kokoro-mlx
parameters:
 model: prince-canuma/Kokoro-82M
 voice: "af_heart"
 known_usecases:
 - tts
  • mlx-vlm: Experiment with the latest VLM models. While we don't have any models in the gallery, it's really easy to configure, see Add mlx-vlm #6119 for more details. 
    name: mlx-gemma
backend: mlx-vlm
parameters:
 model: "mlx-community/gemma-3n-E2B-it-4bit"
template:
 use_tokenizer_template: true
known_usecases:
- chat
  • WAN: Generate videos with Wan2.1 or Wan 2.2 models using the diffusers backend, supporting both I2V and T2V. Example configuration: 
    name: wan21
f16: true
backend: diffusers
known_usecases:
 - video
parameters:
 model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
diffusers:
 cuda: true
 pipeline_type: WanPipeline
 step: 40
options:
 - guidance_scale:5.0
 - num_frames:81
 - torch_dtype:bf16
  • Diffusers CPU and MacOS Support: Run diffusers models directly on your CPU without a GPU or with a Mac! This opens up LocalAI to a wider range of hardware configurations.

✨ WebUI Improvements

We've added several new features to make using LocalAI even easier:

  • Download Model Config: A "Get Config" button in the model gallery lets you download a model’s configuration file without installing the full model. This is perfect for custom setups and easier integration.
  • Manual Model Refresh: A new button allows you to manually refresh the on-disk YAML configuration, ensuring the WebUI always has the latest model information.
  • Streamlined Error Handling: Errors during SSE streaming events are now displayed directly to the user, providing better visibility and debugging information.
  • Backend Stop Button: Quickly stop running backends directly from the WebUI.
Screenshot From 2025年08月15日 22-25-52
  • Model import and edit: Now models can be edited and imported directly from the WebUI.
Screenshot 2025年08月14日 at 22-28-59 LocalAI - Import Model Screenshot 2025年08月14日 at 22-28-47 LocalAI - Edit Model gpt-oss-20b
  • Installed Backend List: Now displays installed backends in the WebUI for easier access and management.

🚀 Performance & Architecture Improvements

  • Purego Whisper Backend: The Whisper backend has been rewritten in Purego for increased performance and stability. This also includes integrated Voice Activity Detection (VAD) for detecting speech.
  • Purego Stablediffusion: Similar to Whisper, Stablediffusion has been converted to Purego, improving its overall architecture and enabling better compatibility.

🛠️ Simplified Management – Introducing the LocalAI Launcher (Alpha)

We're excited to introduce the first version of the LocalAI Launcher! This application simplifies:

  • Installation
  • Startup/Shutdown
  • Updates
  • Access to the WebUI and Application Folder
Screenshot From 2025年08月26日 11-46-35 Screenshot From 2025年08月26日 11-46-50 Screenshot From 2025年08月26日 11-46-21

Please note: The launcher is in Alpha and may have bugs. The macOS build requires workarounds to run due to binaries not yet signed, and specific steps for running it are needed: https://discussions.apple.com/thread/253714860?answerId=257037956022#257037956022.

✅ Bug Fixes & Stability Improvements

  • AMD RX 9060XT ROCm Error: Fixed an issue causing errors with AMD RX 9060XT GPUs when using ROCm. This error, "ROCm error: invalid device function", occurred because of device function incompatibility. The fix involves updating the ROCm image and ensuring the correct GPU targets are specified during compilation. Recommended kernel versions and verification steps for GPU detection are available [here](link to troubleshooting doc if created).
  • libomp Linking: Resolved a missing libomp.so issue on macOS Docker containers.
  • macOS Model Loading: Addressed a problem where models could not be loaded on macOS. This was resolved by bundling necessary libutf8 libraries.
  • CUDA Device Detection: Improved detection of available GPU resources.
  • Flash Attention: Set auto for flash_attention in llama.cpp, allowing the system to optimize performance.

Additional Improvements

  • System Backend: Added a new "system" backend path (LOCALAI_BACKENDS_SYSTEM_PATH or via command-line arguments) defaulting to /usr/share/localai/backends. This allows specifying a read-only directory for backends, useful for package management and system-wide installations.
  • P2P Model Sync: Implemented automatic synchronization of installed models between LocalAI instances within a federation. Currently limited to models installed through the gallery, and configuration changes are not synced. Future improvements will address these limitations.
  • Diffusers Image Source Handling: Enhanced image source selection in the diffusers backend, prioritizing ref_images over src for more robust loading behavior.
  • Darwin CI Builds: Added support for building some Go-based backends (Stablediffusion and Whisper) on Darwin (macOS) in the CI pipeline.

🚨 Important Notes

  • Launcher (Alpha): The LocalAI Launcher is in its early stages of development. Please report any issues you encounter. The MacOS build requires additional steps due to code signing.
  • Model Configuration Updates: Changes to model configuration files are not currently synchronized when using P2P model sync.

The Complete Local Stack for Privacy-First AI

LocalAI Logo

LocalAI

The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required.

Link: https://github.com/mudler/LocalAI

LocalAGI Logo

LocalAGI

A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI.

Link: https://github.com/mudler/LocalAGI

LocalRecall Logo

LocalRecall

A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI.

Link: https://github.com/mudler/LocalRecall

Thank you! ❤️

A massive THANK YOU to our incredible community and our sponsors! LocalAI has over 35.000 stars, and LocalAGI has already rocketed past 1100+ stars!

As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repository, contributing code, reporting bugs, or spreading the word!

Full changelog 👇

👉 Click to expand 👈

What's Changed

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Full Changelog: v3.4.0...v3.5.0

This discussion was created from the release v3.5.0.
You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /