QLoRA
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method for large language models that combines model quantization with Low-Rank Adaptation (LoRA). Its core mechanism quantizes pretrained model weights into a low-precision format (e.g., 4-bit) to substantially reduce memory usage, while training only a small number of learnable low-rank adapter parameters. This approach achieves performance close to full fine-tuning while significantly reducing computational resource requirements, making large-scale language model fine-tuning feasible on limited hardware.
Core Facts
LoRA reduces fine-tuning costs by training only a small number of adapter layers (low-rank matrix bypass training).
QLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
Unsloth is an open-source tooling team focused on large model quantization and fine-tuning acceleration, providing GGUF quantized versions for mainstream open-source models.
Timeline (last 90 days)
Just one QLoRA continual pre-training (CPT) run is enough to install a coupled well-being disposition into a model.
Unsloth's LoRA and QLoRA training now supports splitting each layer of the model across multiple GPUs, by setting the UNSLOTH_TP_SIZE environment variable and launching with torchrun.
The precision handling when LoRA weights are merged back into the backbone, and the memory savings versus accuracy loss of post-quantization fine-tuning (such as QLoRA), both need to be measured in practice before deployment.
LoRA is often combined with quantization (such as QLoRA), which can adapt to downstream tasks while significantly reducing memory and compute requirements.
On a test set of 600 synthetic invoices, the QLoRA fine-tuned model achieved a full-invoice exact rate of 83.0%, an auto-approval rate of 86.8%, and 0 out of 461 validation-coverage field errors in approved invoices (upper bound 0.8%).
Developer raihan-js used Qwen2.5-VL-3B-Instruct with a QLoRA adapter, fine-tuning for 1 epoch on 3000 synthetic invoices, taking 4 hours 57 minutes on a single RTX 3060.
Efficient fine-tuning usually refers to parameter-efficient fine-tuning techniques such as LoRA and QLoRA, which train only a very small number of additional parameters, greatly reducing VRAM and compute requirements, enabling individual developers to fine-tune models on consumer-grade GPUs.
With QLoRA, GPU computation density is relatively low, making data pipeline shortcomings more likely to surface, so that the impact of num_workers, caching and preprocessing parallelism on end-to-end time is further amplified.
When a developer fine-tuned a 7B model on a home RTX 4070, a single full QLoRA training run took about 11 hours.
Fine-tuning gpt-oss-20b with QLoRA requires approximately 14 GB of VRAM, and Unsloth's official recommendation is at least 16 GB.
20 more timeline events
All Facts (20)
LoRA reduces fine-tuning costs by training only a small number of adapter layers (low-rank matrix bypass training).
90%VerifiedQLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
90%VerifiedUnsloth is an open-source tooling team focused on large model quantization and fine-tuning acceleration, providing GGUF quantized versions for mainstream open-source models.
85%VerifiedQLoRA was proposed by a research team at the University of Washington in 2023, quantizing base model weights to 4-bit (NF4 format)
85%VerifiedQLoRA quantizes the base model weights in 4-bit NF4 format, making it possible to fine-tune a 13-billion-parameter model on a single GPU.
80%VerifiedLoRA and QLoRA are the currently mainstream parameter-efficient fine-tuning methods, allowing fine-tuning to be completed on consumer-grade GPUs.
80%VerifiedLoRA reduces the number of trainable parameters by injecting low-rank decomposition matrices into the model's weight matrices, making it possible to fine-tune models with billions of parameters on consumer-grade GPUs.
75%VerifiedQLoRA quantizes the base model to 4-bit precision for storage, while applying LoRA adapters on top of the quantized model for training.
75%VerifiedQLoRA quantizes pre-trained model weights to 4-bit, making it possible to fine-tune a 65-billion-parameter large language model on a single consumer-grade GPU with 24GB of VRAM
75%VerifiedNF4 (4-bit NormalFloat) is an information-theoretically optimal data type proposed in the QLoRA paper (Dettmers et al., 2023).
75%VerifiedQLoRA introduces 4-bit quantization on top of LoRA, further compressing memory requirements to a fraction of the original.
65%VerifiedQLoRA was proposed in 2023 by Tim Dettmers and colleagues at the University of Washington.
65%VerifiedQLoRA combines 4-bit quantization, making it possible to fine-tune 70B parameter models on consumer-grade GPUs.
65%UnverifiedPEFT库支持LoRA和QLoRA等参数高效微调方法，可将需要更新的参数量从数十亿降低到数百万级别
90%UnverifiedQLoRA（Quantized Low-Rank Adaptation）是2023年由华盛顿大学提出的技术
90%UnverifiedUnsloth内置LoRA/QLoRA参数高效微调方法支持
85%UnverifiedGitHub上的开源项目unsloth-llama3-alpaca-lora提供了一套完整的4-bit QLoRA微调流水线，专门针对LLaMA 3 8B模型做了优化
85%Unverified大模型部署调优的核心技能包括使用Ollama、vLLM等框架部署开源模型，通过LoRA、QLoRA进行参数高效微调，以及利用RAG技术结合企业私有知识库
85%UnverifiedQLoRA技术可将7B参数模型的显存占用从约28GB降低到不足6GB
80%UnverifiedUnsloth在QLoRA基础上做了额外工程优化，使实际显存消耗比原始QLoRA实现还要低30-50%
75%