Skip to content
AImpact
IT EN
Training Beginner Also known as: Affinamento · Adattamento

Fine-tuning

An extra training step where a ready-made model is trained on a smaller, more specific dataset to improve its performance on a certain task or domain.

ShareLinkedInX

In practice

You do it when the base model does not match the style, jargon, or formats you need. It requires good labeled data and GPUs. Often you start with a lightweight variant like LoRA before doing a full fine-tune.

Related terms

LoRA Foundation model RLHF

Seen in the wild

10 entries mentioning it
  1. Thinking Machines Lab releases Inkling: a 975B-parameter multimodal MoE with Apache 2.0 open weights
    High
  2. bitsandbytes 0.43: official QLoRA + FSDP support and Windows
    Medium
  3. S-LoRA and Punica: thousands of LoRA adapters served from a single base model
    High
  4. Gorilla: a fine-tuned LLaMA for writing API calls
    Medium
  5. WizardLM: Evol-Instruct evolves instructions for fine-tuning
    Medium
  6. Vicuna-13B: the open chatbot GPT-4 rates above 90% of ChatGPT
    High
  7. DreamBooth: generate your subject in any style with 3-5 photos
    High
  8. Textual Inversion: inject a custom concept into diffusion models
    Medium
  9. InstructGPT: human-feedback fine-tuning that teaches GPT-3 to follow instructions
    High
  10. FLAN: instruction tuning that teaches models to follow directions
    High
← All terms