Hybrid Search: How to Balance Semantic and Keyword Search in RAG

Two ways to search, one answer

Modern RAG systems use hybrid search: combining semantic search (vectors) with keyword search (BM25). Each has distinct strengths.

Semantic search (vectors)

• Understands meaning: “cancel subscription” ≈ “unsubscribe from plan”
• Works with synonyms, paraphrases, natural language
• Can fail with exact technical terms or codes

Keyword search (BM25)

• Finds exact matches: “ERR_429”, “SKU-7832”, “Article 15”
• Ideal for proper names, IDs, specific terminology
• Doesn’t understand synonyms or variations

The key parameter: semantic_weight

Controls the ratio between methods. From 0.0 (BM25 only) to 1.0 (semantic only).

Recipe by content type

Technical docs (APIs, SDKs)

• semantic_weight: 0.4-0.5 — many codes and function names need exact match.

Support / customer service

• semantic_weight: 0.6-0.7 — users ask in natural language with synonyms.

Legal documents

• semantic_weight: 0.5 — perfect balance between exact terms and concepts.

Educational content / guides

• semantic_weight: 0.7-0.8 — conceptual questions, rarely exact terms.

Product catalogs

• semantic_weight: 0.3-0.4 — SKUs, model names, specs need exact match.

Result fusion: how RRF works

After getting results from both methods, they’re combined using Reciprocal Rank Fusion (RRF):

1. Each method returns top-K candidates with ranking
2. RRF scores based on position: 1 / (rank + k)
3. Candidates in both rankings get a boost
4. semantic_weight adjusts each ranking’s influence

Other search parameters

top_k (initial candidates)

• Low (10-15): Faster, may miss relevant results
• High (25-40): More candidates, better recall
• Recommended: 20

similarity_threshold

• 0.1: Permissive | 0.3: Standard | 0.5: Strict

Playground diagnosis

1. Run a query and examine separate results (Vector, BM25, Merged)
2. If vector is good but BM25 is noise → raise semantic_weight
3. If BM25 finds correctly but vector brings irrelevant → lower semantic_weight

Adjust hybrid search from each knowledge base’s RAG configuration.