AI glossary

RAG

Summary

RAG (Retrieval-Augmented Generation) is the technique of giving a model, at answer time, the relevant pieces of your documents so it responds with your information.

Updated

What it is

RAG combines search and generation: first it retrieves the most relevant pieces of your documents (retrieval), and then the model generates the answer based on them (generation). That way it answers with your data, not only with what it learned in training.

Why it matters to an agency

You can give an agent your business knowledge (manuals, policies, catalogs) without paying for fine-tuning. You update the knowledge by changing the documents, not the model.

Example

A support chatbot that answers by citing the client's manual: it finds the right passage and uses it to reply accurately.

By the numbers

A typical RAG pipeline splits documents into chunks of 500 to 1000 tokens and retrieves the 3 to 5 most relevant ones per query. Unlike fine-tuning, it retrains nothing: updating knowledge means reindexing a document, a matter of minutes, while retraining a model takes hours and budget.

FAQ

Frequently asked questions

RAG or fine-tuning?+

RAG for knowledge that changes or is specific (company docs). Fine-tuning to teach a constant style or format. RAG is usually cheaper.