What is RAG? Retrieval-Augmented Generation Explained

What is RAG?

Retrieval-Augmented Generation (RAG) is an AI architecture that combines two capabilities: searching for relevant information in a knowledge base and generating natural responses using a language model (LLM).

Unlike a traditional chatbot that “makes up” answers based on training, a RAG system cites real sources from your documents.

Why does it matter?

LLMs like GPT-4 or Llama have a known problem: they hallucinate. They generate convincing but fabricated text. RAG solves this by forcing the model to base answers on real documents.

Citai’s RAG Pipeline

Stage 1: Hybrid Search

The system searches for the most relevant fragments using two simultaneous methods:

• Semantic search (pgvector): Understands meaning. “How do I cancel my plan?” finds “subscription cancellation process”
• Keyword search (BM25): Finds exact matches. “ERR_429” finds it instantly
• RRF Fusion: Combines both rankings taking the best of each method

Stage 2: Reranking

A cross-encoder model reorders ~20 candidates evaluating query+document together. Slower but much more precise.

Stage 3: MMR Diversity

Maximum Marginal Relevance eliminates redundant fragments. If 3 of the top 5 say the same thing, keeps 1 and brings others with complementary information.

Stage 4: Generation with citations

The LLM receives filtered fragments as context and generates a natural response citing exact document, page, and fragment.

Who is RAG for?

RAG is ideal for any company with internal documentation that wants:

• An assistant that answers questions about products or services
• Reduce support tickets with verifiable automatic answers
• A chatbot that doesn’t make things up

Citai implements enterprise-grade RAG with a 3-stage pipeline. Try it free →