LLaMA-Factory

LLaMA-Factory和Unsloth是社区中成熟的LoRA微调和SFT工具

LLaMA-Factory integrates training algorithms such as incremental pre-training, instruction-supervised fine-tuning (SFT), reward model training (Reward Model), PPO, DPO, KTO, and ORPO.

LLaMA-Factory is an open-source large-model fine-tuning platform, and its GitHub Star count has reached 60.1K.

LLaMA-Factory is used by major companies such as NVIDIA, Alibaba Cloud, and Amazon.

LLaMA-Factory was originally initiated by a student from Beihang University.

LLaMA-Factory offers both a Web UI client interface and a command line, both supporting zero-code operation.

The models supported by LLaMA-Factory cover the LLaMA series, MoE multi-expert system models, the multimodal LLaVA model, and Chinese models such as Qwen, Baichuan, and ChatGLM.

MiniCPM-V 4.7 is compatible with the SGLang, vLLM, llama.cpp, and Ollama inference frameworks, and supports the SWIFT and LLaMA-Factory fine-tuning frameworks.

Axolotl simplifies the reproduction cost of fine-tuning experiments through a configuration-driven approach, while LLaMA-Factory provides a one-stop solution aimed at domestic users.

Common data format specifications include the Alpaca format and the ShareGPT format, both of which are supported for automatic parsing by mainstream training frameworks such as LLaMA-Factory and Axolotl.

LLaMA-Factory is a training framework that integrates multiple PEFT methods and supports the complete pipeline from data preprocessing to export.

LLaMA-Factory is one of the most mainstream open-source large model fine-tuning frameworks, supporting supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) and other fine-tuning methods for dozens of mainstream open-source models including LLaMA, Qwen, and ChatGLM.

LLaMA-Factory provides a visual Web UI, enabling users to complete fine-tuning tasks without a deep understanding of the underlying algorithms.

Xingchen 4.0 29B is compatible with fine-tuning tools such as MindFormers and LLaMA-Factory, as well as inference frameworks such as SGLang and vLLM.

Unsloth's differentiated positioning is as an acceleration layer embedded within the existing Hugging Face ecosystem, where users need only replace a few lines of code to gain speed and memory benefits, and it is compatible with rather than competing against tools such as Axolotl and LLaMA-Factory.

对于生产环境部署建议选择经过社区充分验证的稳定版本，学习实验则可追新以接触最前沿的技术实现

LayerSmith recently added a set of image configurations targeting large model training and fine-tuning, covering LoRA/QLoRA, advanced PyTorch training, and LLaMA-Factory.