Smart Routing: Automatic Knowledge Base Selection with Embeddings

What is Smart Routing?

When you have multiple knowledge bases — for example, one for “Product”, another for “Technical Support”, and another for “Billing” — how does the chatbot know which one to search? Smart Routing solves this automatically, selecting the most relevant KBs for each question without user intervention.

How Does It Work?

The Smart Routing process follows these steps:

1. Query embedding — the user’s question is converted into a 384-dimensional vector
2. Centroid loading — precalculated centroids for each KB are retrieved (cached in Redis for 24 hours)
3. Cosine similarity — similarity is calculated between the query vector and each centroid
4. Top-3 selection — the 3 KBs with highest similarity are selected
5. Minimum threshold — only KBs with similarity >= 0.15 are included (avoids completely irrelevant KBs)
6. Search in selected KBs — the RAG pipeline searches only in the chosen KBs

Complete Flow

User question → Embedding (384d) → Centroid comparison → Top 3 KBs → RAG Pipeline (Hybrid Search → Rerank → MMR) → Response with sources

Practical Example

Imagine a tenant with 3 KBs:

• Product KB: Feature documentation, usage guides
• Support KB: Troubleshooting, known errors, procedures
• Billing KB: Plans, pricing, invoices, payments

If the user asks “how do I change my plan?”, the Billing centroid will have the highest similarity, followed possibly by Product. The Support KB will be excluded or have low relevance.

Centroid Cache in Redis

Recalculating centroids for every question would be expensive. That’s why Citai caches centroids in Redis:

• 24-hour TTL — centroids are recalculated once a day
• Automatic invalidation — when uploading a new document or reprocessing chunks, the cache is invalidated
• Efficient calculation — average of all embedding vectors in the KB

Smart Routing vs Manual Selection

Activation in Citai

• Internal chat: “Auto” button that activates Smart Routing. If you manually select a KB, automatic routing is deactivated
• Widget: When an API key has access to multiple KBs, Smart Routing is enabled by default
• Playground: Always manual selection (for debug and testing)

Conclusion

Smart Routing eliminates the friction of choosing knowledge bases manually. Using embedding centroids and cosine similarity, Citai routes each question to the most relevant KBs automatically — improving response accuracy and simplifying the user experience.

Let AI choose the best knowledge base for you. Try Smart Routing in Citai →