Engineering

RAG vs fine-tuning in Python: which fits your project?

You want a language model that knows your organisation. There are two main routes, and they solve different problems.

Engineering2 min readPython People editorial team

The difference in brief

RAG retrieves the relevant passages from your sources for each question and passes them to the model. The model doesn't change; the knowledge lives in your documents.

Fine-tuning further trains a model on your examples. The model learns a format, style or task. It does not give you reliable factual knowledge that you can easily update later.

Side by side

RAGFine-tuning
Strong atFacts from your own, changing sourcesA consistent format, style or task
New informationAdd a documentRetrain
CitationsYesNo
Per-user permissionsEasy to manageNot possible inside the model
What you needGood sources and strong retrievalLots of good examples and an evaluation set

Our rule of thumb

Almost always start with RAG and a good evaluation set. A strong model with the right context handles most knowledge questions. Consider fine-tuning only when you need a task done the same way every time, want to cut costs with a smaller model, or need a format that instructions alone can't enforce.

The two aren't mutually exclusive: a fine-tuned small model with RAG around it can be a good, affordable combination.

In Python: start by measuring

Whichever route you take, start with a yardstick. For RAG, the first question is whether the right passage is found at all. You can measure that in a few lines of Python, before a language model is even involved.

def hit_rate(test_questions, search, k=5):
    """Share of questions where the right passage is in the top k."""
    hits = 0
    for q in test_questions:
        results = search(q["question"], k=k)
        if q["passage_id"] in {r.id for r in results}:
            hits += 1
    return hits / len(test_questions)

For RAG, teams often use PostgreSQL with pgvector, embeddings from sentence-transformers or an API, and Ragas for evaluation. For fine-tuning, the usual route is Hugging Face Transformers with PEFT (for example LoRA), or your model provider's fine-tuning API.

Need a Python developer?

Call us or send your vacancy. Within 48 hours you hear which developers in our network fit and what it costs.