CitaiCitai
Back to blogCreate free account
Practical GuideMarch 8, 2026·8 min read

How to Choose the Ideal Chunk Size for Your Knowledge Base

Citai Team

March 8, 2026 · 8 min read

Why chunk size matters

Chunk size is arguably the most impactful parameter in a RAG system. Too small and the fragment loses context. Too large and the embedding gets diluted with irrelevant information.

Fundamental rule: A chunk should contain one complete idea. Not half a sentence, not three different topics.

The two key parameters

chunk_size (size in characters)

Defines how many characters each fragment has. Default in most systems: 1024 characters (~200-250 words).

chunk_overlap (overlap)

Defines how many characters repeat between consecutive chunks. Prevents ideas from being cut at boundaries. Typical value: 200 characters (~15-20% of chunk).

512small chunks (precise answers)
1024standard size (balance)
2048large chunks (broad context)

Recipe by document type

FAQs and short documentation

  • chunk_size: 512 | chunk_overlap: 50
  • Precise matches for self-contained Q&A units.

Technical manuals

  • chunk_size: 1024 | chunk_overlap: 200
  • Enough context for technical sections with overlap preserving continuity.

Legal documents / contracts

  • chunk_size: 1500-2048 | chunk_overlap: 300
  • Legal clauses are long and self-referential. Cutting them loses legal meaning.

Articles / blog posts

  • chunk_size: 768 | chunk_overlap: 150
  • Journalistic paragraphs are natural units of ~150-200 words.

Call / chat transcripts

  • chunk_size: 512 | chunk_overlap: 100
  • Conversations have short turns. Small chunks capture each exchange.
Tip: If unsure, start with 1024/200 (the standard). Then adjust based on Playground results.

Diagnosing bad chunking

Symptom: Confidence always low

Cause: Chunks too large → embedding averages too many ideas. Fix: Reduce chunk_size to 512-768.

Symptom: Incomplete answers

Cause: Chunks too small → LLM lacks context. Fix: Increase chunk_size to 1024-1500.

Symptom: Redundant chunks in results

Cause: Excessive overlap. Fix: Reduce chunk_overlap to 10% of chunk_size.

Symptom: Ideas cut in half

Cause: Insufficient overlap. Fix: Increase chunk_overlap to 15-20% of chunk_size.

The fine-tuning method

  1. Baseline: Set 1024/200 and process documents
  2. Test: Run 10-15 real questions in the Playground
  3. Analyze: Do returned chunks contain expected information?
  4. Adjust: Too generic → reduce size. Incomplete → increase.
  5. Reprocess: After changing parameters, reprocess the full KB
  6. Re-test: Run same questions and compare metrics
3-5typical iterations
10-15recommended test questions
15-20%ideal overlap ratio

Adjust chunking from each knowledge base’s RAG config. The Playground shows real-time impact.

Try Citai for free

Create your intelligent knowledge base in minutes. No credit card required.

Create free account

Related articles

Technology

Anatomy of a Contextualized Chunk: Before and After

Step-by-step analysis of how an LLM transforms raw chunks into semantically rich fragments for search.

Read article → Practical Guide

How to Prepare Your Documents for AI: Complete Optimization Guide

Learn best practices for preparing PDFs, DOCX, Excel, TXT, and Markdown before uploading to Citai. Maximize answer quality with well-structured documents.

Read article → Technology

Embeddings and Semantic Search: How Context-Aware AI Works

Understand how embeddings transform text into vectors to find information by meaning, not exact words.

Read article →
Anatomy of a Contextualized Chunk: Befor... Hybrid Search: How to Balance Semantic a...