How to Prepare Your Documents for AI: Complete Optimization Guide

Why Document Preparation Matters

The quality of a RAG system’s answers depends directly on the quality of the documents that feed it. It’s not enough to just upload files — you need to prepare them so the AI processes them optimally.

Universal Principles (apply to all formats)

Before diving into each format, these principles always apply:

1. One Topic per Section

Each section or fragment should cover one topic only. Citai splits your documents into chunks of ~500-1000 tokens. If a section mixes topics, the resulting chunk will be ambiguous and the reranker will penalize it.

❌ Bad:

“Our plans range from $10 to $100/month. To cancel, email support. We accept Visa and Mastercard. Business hours are 9 AM to 6 PM.”

✅ Good: Separate into sections: “Plans and pricing”, “Cancellation”, “Payment methods”, “Business hours”.

2. Descriptive Hierarchical Headings

Use clear, hierarchical headings. They help the system understand structure and provide context for each chunk.

❌ Bad: “Section 3”, “Additional information”, “Other”

✅ Good: “Return policy for online purchases”, “Installation requirements for Windows 11”

3. Avoid Redundant Information

If the same content appears in 5 different documents, the system will retrieve it 5 times and waste context slots. Consolidate information in a canonical document.

4. Up-to-date Information

Outdated documents generate incorrect answers. If the price changed from $10 to $15, update the document. Citai can’t know which version is correct.

5. Clear and Direct Language

AI processes direct text better than convoluted text:

❌ “Users who wish to proceed with the cancellation of their membership shall…”

✅ “To cancel your membership, go to Settings > Plan > Cancel.”

PDF: The Most Common (and Most Problematic) Format

PDFs are the most uploaded format to Citai, but also the most unpredictable because not all PDFs are the same.

Types of PDFs and How Citai Processes Them

Best Practices for PDFs

Before uploading:

1. Verify text is selectable. Open the PDF and try selecting text with your mouse. If you can’t, it’s a scanned PDF and extraction will be poor.

2. Remove unnecessary pages. Decorative covers, blank pages, generic table of contents, and back covers don’t provide useful information but generate junk chunks.

3. Simplify complex tables. Tables with merged cells, nested tables, or tables spanning multiple pages extract poorly. If you have a critical table, convert it to list or plain text format.

4. Watch out for repeated headers and footers. “Company XYZ — Confidential Document — Page 3” will appear in every chunk and contaminate embeddings. If possible, regenerate the PDF without headers/footers.

5. Avoid PDFs over 100 pages. Split large documents into logical sections (e.g., a technical manual PDF → one PDF per chapter). This improves chunking accuracy.

What to Avoid in PDFs

• Scanned PDFs without OCR: Citai extracts what it can, but if the PDF is an image, there’s no text to extract. Use tools like Adobe Acrobat or OCRmyPDF before uploading.
• Multi-column layouts: Text may extract in wrong order (column mixing). Single column is better.
• Text in images: Infographics, diagrams with embedded text — the text won’t be extracted. Add a text description below.
• Interactive PDF forms: Fillable fields may not extract correctly.

DOCX (Word): The Most Reliable Format

Word files (.docx) are probably the best format for Citai because they maintain a clean and predictable structure.

Why DOCX is Ideal

• Headings (Heading 1, 2, 3) are preserved → better chunking
• Simple lists and tables extract correctly
• No layout ambiguity like in PDFs
• Easy to edit and keep updated

Best Practices for DOCX

1. Use native Word styles. Apply “Heading 1”, “Heading 2”, etc. instead of just making text bold and bigger. Semantic styles help the parser understand hierarchy.

2. Structure with clear hierarchy:

   • Heading 1: Main topic (e.g., “Return Policy”)
   • Heading 2: Subtopics (e.g., “Deadlines”, “Conditions”, “Process”)
   • Heading 3: Details (e.g., “Electronics returns”)

3. Simple tables. Tables with one header row + data rows work perfectly. Avoid merged cells.

4. Remove comments and track changes. Before uploading, accept all changes and delete comments — otherwise they may end up as part of the extracted text.

5. Remove decorative images. Logos, graphical separators, decorative icons — they don’t contribute to text extraction and can confuse the parser.

Markdown: The Favorite Format for Technical Teams

Markdown is excellent for Citai because it’s plain text with clear semantic structure.

Why Markdown Works So Well

• Heading hierarchy (#, ##, ###) is explicit
• No hidden formats or design layers
• Easy to version with Git
• Code blocks are preserved perfectly
• Markdown tables parse cleanly

Best Practices for Markdown

1. Hierarchical headings are mandatory. Every section needs a heading. Markdown without headings is a monolithic block that generates low-quality chunks.

# Installation Guide              ← H1: main topic
## Prerequisites                   ← H2: section
### Operating System               ← H3: subsection
- Windows 10 or higher
- macOS 12 or higher

## Step-by-Step Installation       ← H2: another section
### 1. Download the Installer      ← H3: step
...

2. One file per topic. It’s better to have 10 Markdown files of 500 lines than 1 file of 5000 lines. Smaller files = more coherent chunks.

3. Use lists. Bulleted or numbered lists process very well and generate compact chunks with dense information.

4. Simple tables. Markdown tables are ideal for structured data:

| Plan    | Price | Queries/month |
|---------|-------|---------------|
| Free    | $0    | 50            |
| Starter | $29   | 500           |
| Pro     | $99   | 5,000         |

5. Code blocks for technical content. If documenting APIs or commands, use blocks with specified language:

\`\`\`bash
curl -X POST https://api.citai.ai/chat \
  -H "Authorization: Bearer sk-..." \
  -d '{"message": "How to install?", "kb_id": "..."}'
\`\`\`

TXT: Simple but Effective

Plain text files (.txt) are the most basic format but can work surprisingly well if prepared correctly.

When to Use TXT

• Database or CRM exports
• Call or chat transcriptions
• Content extracted from external sources
• Legacy documentation without formatting

Best Practices for TXT

1. Simulate structure with conventions. Since TXT has no formatting, use clear conventions:

=== RETURN POLICY ===

--- Deadlines ---
* 30 days for physical products
* 14 days for digital products

--- Conditions ---
* Unused product
* Original packaging

2. Blank lines as separators. Use double blank lines between sections. Citai uses these signals to determine chunk boundaries.

3. One idea per paragraph. Each paragraph should be self-contained. Avoid 500-word paragraphs that mix multiple topics.

4. Clean content before uploading:

   • Remove duplicate lines
   • Strip unnecessary special characters
   • Normalize encoding to UTF-8
   • Remove excessive whitespace

5. Add context at the beginning of the file. The first lines establish context for the entire document:

Company: Acme Corp
Document: Internal procedures manual
Last update: January 2026
Department: Human Resources
---
[content here]

Excel (.xlsx / .xls): Spreadsheets as a Knowledge Base

Excel files are ideal for structured data: product catalogs, price lists, FAQ tables, inventories, etc. Citai extracts all sheets as tabular text.

How Citai Processes Excel Files

• Each sheet is extracted separately
• Row 1 headers are used as column names in the generated text
• Rows are converted to text in the format column: value
• All sheets in the file are processed

Best Practices for Excel

1. Descriptive headers in row 1 (required). Row 1 defines how each column is identified in the extracted text. “Name” is better than “Col_A”. “Price excl. tax (USD)” is better than “Price”.

2. One sheet per topic. If you have a file with 5 sheets on different topics, consider splitting it. Sheets are processed as independent documents within the same file.

3. No merged cells. Merged cells break the tabular structure and generate confusing text. Always use individual cells.

4. No empty rows in the middle of data. Empty rows can fragment extraction. If you need to separate sections, use a “Category” column instead of empty rows.

5. No unlabeled formula columns. If a column has formulas that calculate something, make sure the header explains what the value represents (e.g., “Price with 20% discount”, not just “Price2”).

6. Clean data, no decorative formatting. Colors, borders, and styles are not extracted. Focus only on cell content.

URL Ingestion: Documents from the Web

Citai also allows ingesting content directly from URLs (HTML, PDF, DOCX).

Best Practices for URL Ingestion

1. Stable URLs. Use URLs that don’t change (avoid URLs with temporary tokens or sessions).
2. Public content. URLs requiring login aren’t accessible to the crawler.
3. Prefer content pages. Landing-style pages with lots of design and little text generate low-quality chunks.
4. One URL per topic. It’s better to ingest 10 specific documentation pages than the site’s homepage.

Format Comparison Table

Common Mistakes That Destroy Quality

1. “I’ll upload everything and be done”

Uploading 200 files without reviewing is the recipe for mediocre answers. Review at least the most important documents.

2. Duplicate Documents

Same content in PDF and DOCX. The system treats them as distinct documents and retrieves duplicate information, wasting context slots.

3. Outdated Documents Alongside Current Ones

If you have “Policy 2024” and “Policy 2025”, the AI doesn’t know which is current. Remove the old version.

4. Scanned PDFs Without OCR

Appears to upload correctly but no text was extracted. Check the chunks in the admin panel to verify.

5. Tables as Primary Information Source

If your most important content is in complex tables, extract the information as text or lists before uploading.

6. Documents in Mixed Languages

A document that alternates between English and Spanish generates confused embeddings. Separate by language when possible.

Pre-Upload Checklist

Use this list before uploading each document to Citai:

• [ ] Is the text selectable? (for PDFs)
• [ ] Do headings use semantic hierarchy? (H1 > H2 > H3)
• [ ] Does each section cover one topic only?
• [ ] Did I remove unnecessary pages/sections?
• [ ] Is the content up to date?
• [ ] Are there no duplicates with other uploaded documents?
• [ ] Are tables simple? (no merged cells)
• [ ] Is the encoding UTF-8?
• [ ] Is the language consistent?
• [ ] Did I remove repetitive headers/footers? (PDFs)

After Uploading: Verification with the Playground

Once you upload your documents, use the RAG Playground to verify quality:

1. Ask questions that should be answered by your documents. If the answer is incorrect or low confidence, review the generated chunks.

2. Review chunks. Go to the knowledge base and review generated fragments. If you see chunks mixing topics or with cut-off content, the document needs restructuring.

3. Adjust parameters if needed. The Playground lets you modify chunk_size, chunk_overlap, and other parameters to optimize quality.

4. Test variations. Ask the same question in different ways to verify the system responds consistently.

Citai supports PDF, DOCX, Excel, Markdown, TXT, and URLs. With well-prepared documents, you’ll get accurate answers with verifiable sources. Start free →