Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

localai 2.26.0 #207814

Merged
merged 2 commits into from
Feb 15, 2025
Merged

localai 2.26.0 #207814

merged 2 commits into from
Feb 15, 2025
+10 −10

Conversation

BrewTestBot
Copy link
Member

@BrewTestBot BrewTestBot commented Feb 15, 2025 •
edited by nandahkrishna
Loading

Created by brew bump

Created with brew bump-formula-pr.

  • resource blocks have been checked for updates.
release notes 

  





:llama:  LocalAI v2.26.0!


Hey everyone - very excited about this release!


It contains several cleanups, performance improvements and few breaking changes: old backends that are now superseded have been removed (for example, vall-e-x), while new backends have been added to expand the range of model architectures that LocalAI can support. While most of the changes are tested, if you encounter issues with the new backends or migrated ones please file a new issue.


We also now have support for Nvidia L4T devices (for example, Nvidia AGX Orin) with specific container images. See the documentation for more details.


:warning: Breaking Changes :warning:


    

  • Several backends have been dropped and replaced for improved performance and compatibility.
    • 

  • Vall-e-x and Openvoice were deprecated and dropped.
    • 

  • The stablediffusion-NCN backend was replaced with the stablediffusion-ggml implementation.
    • 

  • Deprecated llama-ggml backend has been dropped in favor of GGUF support.
    • 





Check all details!


Backends that were dropped:


    

  • Vall-e-x and Openvoice: These projects went silent, and there are better alternatives now. They have been completely superseded by the CoquiTTS community fork, Kokoro, and OutelTTS.
    • 

  • Stablediffusion-NCN: This was the first variant shipped with LocalAI based on the ONNX runtime. It has now been superseded by the stablediffusion-ggml backend, which offers similar capabilities and wider support across more architectures.
    • 

  • Llama-ggml backend: This was the pre-GGUF backend, which is now deprecated. Moving forward, LocalAI will support only GGUF models.
    • 



Notable Backend Changes:


    

  • Mamba has moved to the transformers backend.
    • 

  • Transformers-Musicgen has moved to the transformers backend.
    • 

  • Sentencetransformers has moved to the transformers backend.
    • 



While LocalAI will try to alias to the transformers backend automatically when using these backends, there might be incompatibilies with your configuration files. Please open an issue if you face any problem!


New Backends:


    

  • Kokoro (TTS): A new backend for text-to-speech.
    • 

  • OuteTTS: A TTS backend with voice cloning capabilities.
    • 

  • Fast-Whisper: A backend designed for faster whisper model inference.
    • 





New Features  🎉


    

  • Lazy grammars (llama.cpp): Added grammar triggers for llama.cpp: this allow models trained with specific tokens to enable grammar generation when such tokens are seen: this allows precise JSON generation but also consistent output when the model does not need to answer with a tool. For example, in the config file of the model triggers can be specified as such:
    • 



  function:
    grammar:
      triggers:
        word: "<tool_call>"
        at_start: true


    

  • Function Argument Parsing Using Named Regex: A new feature that allows parsing function arguments with named regular expressions, simplifying function calls.
    • 

  • Support for New Backends: Added Kokoro, OutelTTS, and Fast-Whisper backends.
    • 

  • Diffusers Update: Added support for Sana pipelines and image generation option overrides.
    • 

  • Machine Tag and Inference Timing: Allows tracking machine performance during inference.
    • 

  • Tokenization: Introduced tokenization support for llama.cpp to improve text processing.
    • 

  • AVX512: There is now bundled support for CPUs supporting AVX512 instruction set
    • 

  • Nvidia L4T: Support for Nvidia devices on arm64, for example Nvidia AGX Orin and alikes.  See the documentation. TLDR; You can start container images ready to go with:
    • 



docker run -e DEBUG=true \
                    -p 8080:8080 \
                    -v $PWD/models:/build/models  \
                   -ti --restart=always --name local-ai \
                   --runtime nvidia --gpus all quay.io/go-skynet/local-ai:master-nvidia-l4t-arm64-core


Bug Fixes 🐛


    

  • Multiple fixes to improve stability, including enabling SYCL support for stablediffusion-ggml and consistent OpenAI stop reason returns.
    • 

  • Improved context shift handling for llama.cpp and fixed gallery store overrides.
    • 



🧠 Models:



  






  





I've fine-tuned a family of models based on o1-cot and function call datasets to work closely with all LocalAI features regarding function calling. The models are tailored to be conversational and execute function calls:



Enjoy! All the models are available in the LocalAI gallery:


local-ai run LocalAI-functioncall-phi-4-v0.3
local-ai run LocalAI-functioncall-llama3.2-1b-v0.4
local-ai run LocalAI-functioncall-llama3.2-3b-v0.5
local-ai run localai-functioncall-qwen2.5-7b-v0.5


Other models


Numerous model updates and additions:


    

  • New models like nightwing3-10b, rombos-qwen2.5-writer, and negative_llama_70b.
    • 

  • Updated checksum for model galleries.
    • 

  • Added icons and improved prompt templates for various models.
    • 

  • Expanded model gallery with new additions like DeepSeek-R1, Mistral-small-24b, and more.
    • 




Full changelog :point_down:




:point_right: Click to expand :point_left: 


Breaking Changes 🛠



Bug fixes :bug:



Exciting New Features 🎉



🧠 Models



📖 Documentation and examples



👒 Dependencies



Other Changes





New Contributors



Full Changelog: mudler/LocalAI@v2.25.0...v2.26.0

@github-actions github-actions bot added go Go use is a significant feature of the PR or issue bump-formula-pr PR was created using `brew bump-formula-pr` labels Feb 15, 2025
@BrewTestBot @nandahkrishna
localai 2.26.0
5cc973a 
Updated `grpcio-tools` resource.

Co-authored-by: Nanda H Krishna <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

🤖 An automated task has requested bottles to be published to this PR.

@github-actions github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label Feb 15, 2025
@BrewTestBot BrewTestBot added this pull request to the merge queue Feb 15, 2025
Merged via the queue into master with commit df8b5ef Feb 15, 2025
15 checks passed
@BrewTestBot BrewTestBot deleted the bump-localai-2.26.0 branch February 15, 2025 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carlocab carlocab carlocab approved these changes

@github-actions github-actions[bot] github-actions[bot] approved these changes

Assignees
No one assigned
Labels
bump-formula-pr PR was created using `brew bump-formula-pr` CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. go Go use is a significant feature of the PR or issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@BrewTestBot @carlocab