Skip to content
AImpact
IT EN
Training Intermediate Also known as: Low-Rank Adaptation

LoRA

/lor-ah/

A fine-tuning technique that trains only a small set of extra parameters instead of the whole model, cutting compute cost and the size of the resulting file.

ShareLinkedInX

In practice

It lets you customize a 70-billion-parameter model on a consumer GPU. You save adapters of a few MB that plug on top of the base model. It is the practical standard for adapting open-weight models to specific use cases.

Related terms

Fine-tuning Quantization

Seen in the wild

4 entries mentioning it
  1. OCI Generative AI adds Llama 3.1 70B and 405B, with dedicated AI clusters and LoRA fine-tuning
    Medium
  2. bitsandbytes 0.43: official QLoRA + FSDP support and Windows
    Medium
  3. S-LoRA and Punica: thousands of LoRA adapters served from a single base model
    High
  4. GPT4All: a ChatGPT-style assistant that runs offline on a laptop CPU
    High
← All terms