Changelog
Update history and new features in Citai.
Smart Diagnostics for Golden Set + Humanized UX
The Golden Set now self-diagnoses: each item with weak metrics shows actionable suggestions with buttons to open Playground, edit item or adjust RAG config. Run-level patterns flag systemic issues. Item editor redesigned with tag chips + visual source picker (no more JSON or UUIDs).
- +"Patterns detected" block above the run when ≥60% items share a problem
RAG Evaluation Framework — Per-KB Golden Set + Regression Tests
Automated test bench with per-KB Golden Set. 5 metrics (faithfulness, answer relevancy, context precision/recall, citation accuracy), baseline comparison, automatic regression detection >5% with admin email. Auto triggers on FAQ approval, Smart Suggestions apply, doc reprocessing, and weekly.
- +Full CRUD for datasets and items with tenant isolation
Faithfulness Detection + Action Loop — Anti-hallucination with closed-loop actions
An NLI model verifies that every LLM answer is supported by retrieved chunks. Each detected hallucination triggers actions: create curated FAQ, mark false positive, add to golden set. Detected patterns suggest systemic changes (problematic KB, LLM model worse than the rest, deterioration vs previous period).
- +Playground card with color badge and unsupported claims list
Advanced Playground — Knowledge Graph and Chunk Viewer
Two new visual diagnostic tools in the RAG Playground to understand exactly which chunks are being retrieved and how they relate to each other.
- +Click any node to see its individual scores, full text, and the stage where it was filtered out
Email Threading — Conversation Threads
Chain incoming emails and responses into conversation threads with cumulative context for increasingly precise responses.
- +Smart context: last 3 messages in full + older ones summarized, injected into LLM
Full Observability Stack
Complete observability stack with structured logging, metrics and distributed tracing.
- +Prometheus metrics (requests, latency, tokens, cache hits)
Security Sprint
Remediation of 8 vulnerabilities detected in security audit.
- +2 high severity vulnerabilities fixed
Email Analytics, Channel Comparison and ROI
Dedicated Email Assistant metrics, cross-channel performance comparison and ROI calculator.
- +Cross-channel performance comparison: Chat vs Widget vs Email
AI Email Assistant
Module for generating email responses based on Knowledge Bases, with smart pre-analysis and tool calling.
- +Smart pre-analysis: detects topics, KB coverage and suggests instructions
Visual RAG — Image Extraction
Automatic image extraction from PDF and DOCX documents to enrich RAG responses.
- +Image extraction from DOCX with python-docx
Tool Auto-Inject and Per-API-Key Filtering
Automatic tool data injection into LLM context and granular tool control per widget.
- +Per-API-key tool filtering (tool_ids per widget)
Enterprise: Reports, CSAT and White-Label
Scheduled email reports, advanced CSAT with NPS and configurable white-label per plan.
- +Advanced CSAT: NPS score, weekly trend, top negative comments, per widget
Tool Calling — External API Integration
The agent can invoke external APIs during conversations to get real-time data.
- +Pre-built templates: weather, news, currency, generic webhook
Document Sync and GDPR Retention
Automatic document synchronization, configurable retention policies and webhook retry with backoff.
- +Configurable GDPR retention policies per tenant
Live Chat and Invoicing
Real-time human support with handoff from the AI assistant and PDF invoice generation.
- +Live Chat panel with waiting queue, active and resolved sessions
CI/CD and Automated Testing
Complete testing infrastructure with continuous integration to ensure quality on every deploy.
- +GitHub Actions CI pipeline with PostgreSQL and Redis services
GDPR Privacy & FAQ Image Upload
Data compliance, multimedia upload with crop editor and smart reseed.
- +GDPR Art. 17 account deletion — soft delete with password confirmation
Audit Log & Traceability
Critical action logging with configurable retention and date-based purge.
- +AuditLogView with filters by action, user, resource and date range
Observability, OAuth & 2FA Authentication
Professional security with error monitoring, Google login and two-factor authentication.
- +Security headers: HSTS, X-Frame-Options, CSP, X-Content-Type-Options
Contextual Retrieval & Advanced Playground
LLM-enriched chunks with full document context and advanced RAG diagnostic tools.
- +Advanced Playground: 3 modes (standard, A/B compare, batch)
Blog, Smart Language & Public Pages
SEO, technical content and automatic language detection to reach more users.
- +Automatic language detection (ES/EN/PT/DE/FR/IT) with system prompt injection
Semantic Cache & Smart Routing
Intelligent optimization: cached responses and automatic routing to the right KB.
- +KB Health Score: 0-100 rating with 5 weighted metrics
Advanced Analytics & RAG Playground
Full visibility into your assistant's performance with real-time metrics.
- +Interactive charts: time series, resolution doughnut, tag bar charts
Embeddable Widget & API Keys
Your AI assistant directly on your website. Copy a snippet and you're live.
- +Real-time SSE streaming via fetch + ReadableStream
Smart FAQ & Confidence Scoring
Instant answers from FAQs and confidence scoring for every response.
- +Combined confidence scoring: 50% reranking + 50% vector similarity
Multi-Tenancy & Admin Panel
Real data isolation between organizations with a complete admin panel.
- +2 roles with granular permissions: admin and member
Authentication & Transactional Email
Secure login system with JWT, email verification and invitations.
- +Automatic refresh token rotation (one-time use)
Conversational Interface & i18n
Enhanced chat with agent personality, dark mode and 6 languages.
- +Conversation starters: customizable quick-start buttons
Agentic RAG & Code Execution
The assistant searches for information autonomously and executes code in a sandbox.
- +Sandboxed code execution — generates and runs Python on the fly
Chat with Streaming & Sources
Real-time chat interface with responses that cite their sources.
- +Responses with cited sources — each paragraph references its document
Hybrid Search & Knowledge Bases
The search engine that combines semantics and keywords to find the best answer.
- +RRF fusion (Reciprocal Rank Fusion) to combine both rankings
Project Foundation
Base architecture, tech stack and first design decisions.
- +Multilingual embedding model (paraphrase-multilingual-MiniLM)