Fine-tuning methods: full fine-tuning, LoRA and QLoRA

Full fine-tuning

In full fine-tuning, you continue training the whole model on your data, so every parameter can change.

It gives the most freedom, and it costs the most. Training needs memory for the weights, the gradients and the optimizer state, which together can take several times the memory of the weights alone. For a 7 billion parameter model, that is far beyond a single consumer GPU, and every fine-tuned version is a full copy of the model to store and serve.

GPT-3 has 175 billion parameters. Updating all of them for one specific task isn't practical for most teams, and for most tasks it isn't necessary either.

Parameter-efficient fine-tuning (PEFT)

PEFT methods update only a small subset of parameters, or add a small number of new ones, and keep the rest of the model frozen. That reduces both compute and memory, which are the two main reasons to use them.

PEFT methods include:

• LoRA, low-rank adaptation, the most widely used.
• Adapters, small new layers inserted into the network.
• Prefix tuning, which learns vectors added to the input of each layer.
• Prompt tuning, which learns "soft prompt" vectors added to the input.

Hugging Face's peft library implements these, and it's what most open-source fine-tuning code uses.

How LoRA works

The weights of a neural network are stored in matrices. When you fine-tune, you're computing an update to those matrices.

The insight behind LoRA is that this update usually has low rank, which means it can be expressed with far fewer numbers than the full matrix. The rank of a matrix is the number of independent rows or columns it has, and a low-rank matrix can be written as the product of two much thinner matrices.

So instead of learning a full update ΔW for a weight matrix W of size d × d, LoRA learns two matrices: A of size r × d and B of size d × r, where r (the rank) is small, like 8 or 16. Their product B × A has the same shape as W, and during the forward pass the model uses W + B × A. W stays frozen, and only A and B are trained.

def lora_params(d_in: int, d_out: int, r: int) -> tuple[int, int]:
    full = d_in * d_out
    lora = r * d_in + d_out * r
    return full, lora


full, lora = lora_params(4096, 4096, r=8)
print(f"full update: {full:,} parameters")
print(f"LoRA r=8:    {lora:,} parameters ({lora / full:.2%})")

For one 4096 × 4096 matrix, a rank 8 adapter trains 65,536 numbers instead of 16,777,216, about 0.39%.

LoRA is usually applied to the attention layers, especially the query and value projections, and sometimes to all linear layers. After training, you save only A and B, an adapter of a few megabytes, and you can merge it into the base weights or load it on top of the base model when you serve.

The LoRA paper reported that on GPT-3 175B, LoRA reduced the number of trainable parameters by 10,000 times and GPU memory needs by 3 times compared with full fine-tuning, with quality on par with full fine-tuning on the tasks they tested.

LoRA settings

Setting	What it does	Common starting point
r (rank)	Size of the adapter, how much it can learn	8 or 16
lora_alpha	Scaling factor for the update	Often 2 × r, like 16 or 32
target_modules	Which weight matrices get adapters	Attention projections like q_proj, v_proj, or all linear layers
lora_dropout	Regularisation	0.05 to 0.1
Learning rate	How fast the adapter learns	Higher than full fine-tuning, often around 1e-4 to 2e-4

Start with defaults, train, evaluate on your test set, and change one setting at a time.

How QLoRA works

Quantization stores each weight with fewer bits, for example 4 bits instead of 16, which cuts the memory the model needs. See quantization.

QLoRA combines the two ideas:

1. Load the frozen base model in 4-bit precision, using a data type called NF4 (4-bit NormalFloat), designed for the distribution of neural network weights.
2. Add LoRA adapters, kept in higher precision (16-bit), and train only those.
3. During the forward pass, the 4-bit weights are converted back to 16-bit for the calculation.

The paper added two more tricks: double quantization, which also quantizes the scaling constants, and paged optimizers, which avoid memory spikes during training. With these, the authors fine-tuned a 65 billion parameter model on a single 48 GB GPU.

The quantization applies to the frozen base model. The adapters you train are the same LoRA adapters, so the LoRA settings don't change.

Code: LoRA and QLoRA with Hugging Face

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

model_name = "meta-llama/Llama-3.2-1B"

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_name)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=quant_config,
    device_map="auto",
)

lora_config = LoraConfig(
    r=8,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules=["q_proj", "v_proj"],
    bias="none",
    task_type="CAUSAL_LM",
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

Remove quantization_config and you have plain LoRA. Then you train with the Trainer from transformers, or the SFTTrainer from Hugging Face's TRL library, and save the adapter with model.save_pretrained("output/adapter").

Two things to keep in mind:

• Use the tokenizer the model was trained with. AutoTokenizer.from_pretrained(model_name) loads exactly that.
• Format your data the way the model expects it, using its chat template for chat models.

bitsandbytes 4-bit quantization is built for NVIDIA GPUs. On a laptop CPU or an Apple Silicon Mac, plain LoRA on a small model is the practical option, or use a free GPU on Google Colab.

Which method to choose

	Full fine-tuning	LoRA	QLoRA
Trainable parameters	All	Well under 1% to a few %	Same as LoRA
GPU memory	Highest	Much lower	Lowest
Output	Full model copy	Small adapter	Small adapter
Quality	Highest ceiling	Close to full on many tasks	Close to LoRA
Speed	Slowest	Fast	Slower than LoRA (dequantization)

For most teams starting out, LoRA or QLoRA on an open model is the right default. Full fine-tuning makes sense when you have the hardware and a task where you've measured that adapters fall short.

From the floor

In AI Engineer HQ, we first fine-tuned Llama with LoRA on the IMDb review dataset for sentiment classification: we prepared the data as JSONL with a prompt and a completion ("positive" or "negative"), set the rank to 8 and alpha to 32, trained for one epoch with a step limit because we were on a CPU, saved the model and ran sample predictions.

In the next session, I explained the code line by line and then changed it into QLoRA: the training code stays the same, the only change is a BitsAndBytesConfig that loads the base model in 4-bit NF4 and is passed as quantization_config when loading the model. The reason I gave for quantization was running a big model on a small device without a GPU, like a phone or an edge device.

In the legal LLM project lab, we used Llama 3.2 1B, because at around 1 billion parameters it is small enough to fine-tune on a laptop, with LoRA and a 4-bit configuration, and the same training pipeline.

Related

• Quantization
• Serving fine-tuned adapters
• Building a fine-tuning dataset
• How an LLM works

Sources

• LoRA: Low-Rank Adaptation of Large Language Models, Hu and others (2021)
• QLoRA: Efficient Finetuning of Quantized LLMs, Dettmers and others (2023)
• PEFT documentation, Hugging Face
• TRL documentation, Hugging Face