·
103 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
Farewell to 2025. Thank you to all contributors and supporters. We will continue to deliver an easy and efficient LLM fine-tuning framework to the community in 2026. Stay tuned.
Breaking
- Repository name updated: LLaMA-Factory → LlamaFactory
- Python 3.9–3.10 have been deprecated; LlamaFactory now requires Python 3.11–3.13
- Migrated from pip to uv; use
uv pip install llamafactory - The official LlamaFactory blog is now live: https://blog.llamafactory.net/en/
New features
- 🔥 Support Orthogononal Fine-Tuning (OFT) by @zqiu24 in #8623
- 🔥 Support Semantic Initialization for new added tokens by @ximinng in #9267
- 🔥 Support Megatron-LM training via MCoreAdapter by @Kuangdd01 in #9237
- 🔥 Support KTransformers backend by @JimmyPeilinLi in #9400
- Support MPO algorithm by @Kuangdd01 in #8930
- Support FP8 training by @penfever in #8960
- Support Transformers v5 by @tangefly in #9569
- Support reasoning and plaintext in function call message by @tangefly in #9610
- Support DeepSpeed AutoTP by @sunyi0505 in #9602
- Support efficient NPU fused kernels by @frozenleaves in #9520
- Support TRL 0.24 by @UsernameFull in #9617
Models
- Falcon H1 by @dhiaEddineRhaiem in #8403
- Kimi-VL and GLM-4.5V by @Kuangdd01 in #8462
- Gemma3n by @Kuangdd01 in #8509
- Granite4 by @Tuyohai in #8680
- Qwen3-2507 by @hiyouga in #8750
- MiniCPM-V 4.0 by @ZMXJJ in #8813
- Intern-S1-mini by @hhaAndroid in #8976
- Seed-OSS by @Kuangdd01 in #8992
- MiniCPM-V 4.5 by @tc-mb in #9022
- InternVL-3.5 by @Kuangdd01 in #9028
- ERNIE-4.5-Text and ERNIE-4.5-VL by @isLinXu in #9165
- Ling-V2 by @wangsff in #9188
- Qwen3-VL and Qwen3-Omni by @xvxuopop and @Kuangdd01 in #9196
- Hunyuan-mt by @wyfdgg in #9284
- GLM-4.6V by @isLinXu in #9586
- Ministral 3 by @tangefly in #9582
- VibeThinker by @isLinXu in #9616
- MiMo-V2-Flash by @isLinXu in #9637
- MiniMax-M1 and MiniMax-M2 by @isLinXu in #9680
Thanks to teams collaborating with LlamaFactory in 2025
- NPU Team: @jiaqiw09 @frozenleaves @xvxuopop @UsernameFull @codemayq
- KTransformers Team: @JimmyPeilinLi @poryfly @mrhaoxx
- ROLL Team
And to individuals who made significant contributions
Full Changelog: v0.9.3...v0.9.4
Contributors
poryfly, codemayq, and 21 other contributors