Open-Source RAG Platform chatVia.ai: Full-Stack Retrieval Pipeline and Self-Hosted Deployment

Title

Open-Source RAG Platform chatVia.ai: Full-Stack Retrieval Pipeline and Self-Hosted Deployment

Meta Description

chatVia.ai, an open-source RAG platform, puts vector retrieval, BM25, and RRF fusion all inside Postgres, with self-hosting, unlimited agents, and automatic updates. This article breaks down its ingestion, chunking, hybrid retrieval, reranking, and evaluation pipeline, and explains the boundary between the free core and the paid Pro tier.

Article Content

A RAG platform called chatVia.ai, built by a single developer over the course of a year, has officially been open-sourced. It tries to answer a straightforward question: why should every team pay a monthly fee for "an AI agent that can read their own documents"? The answer is to stuff the entire retrieval stack into Postgres, keep data on your own servers, and provide a one-line install command with automatic updates. According to the original Reddit post, the platform supports unlimited agents and unlimited replies, with no need to run a separate vector database.

From Ingestion to Generation: A Tunable RAG Pipeline

chatVia.ai's RAG pipeline is organized by stage, and each stage exposes tunable parameters rather than being a black-box wrapper.

Ingestion and Chunking: Content Awareness Is the Starting Point

Files, websites, plain text, Q&A pairs, and YouTube transcripts are all converted into documents. Web crawling respects path rules, and when re-crawling, only changed parts are re-embedded, avoiding a full recomputation. Chunking strategy branches by content type: Markdown is split by headings, prose is split by sentences, and everything else uses fixed windows with overlap. Chunk size and overlap ratio can be adjusted per agent, which makes a big difference for retrieval effectiveness across different document structures (such as API docs versus blog posts).

Contextual Enrichment and Embedding: Making the "Second Option" Retrievable

Before embedding, each chunk gets a short LLM-generated context describing its position in the original text. This way, even if a chunk only says "the second option," it can still match the correct semantics when retrieved on its own. Embedding models can be chosen from OpenAI, Mistral, Gemini, Cohere, Jina, Voyage, Qwen3, or OpenRouter, stored as halfvec with an HNSW index, which the author claims cuts index size in half with no loss in recall.

halfvec is a half-precision (16-bit floating point) vector type provided by the pgvector extension. Compared with the default float32 vectors, it uses only 2 bytes per dimension, roughly halving storage and memory overhead. HNSW (Hierarchical Navigable Small World) is an approximate nearest neighbor index based on a multi-layer graph. At query time it greedily descends from the sparse top layer down to the dense bottom layer, finding neighbors in logarithmic complexity, at the cost of slower index construction and additional memory usage. HNSW's core tunable parameters include M, which controls the number of graph connections, and ef_construction, the build-time search width; there is also ef_search at query time, which trades off recall against latency. Half-precision vectors can indeed significantly compress index size, but the impact on recall depends on the model and data distribution and usually requires empirical verification rather than a universal guarantee.

Hybrid Retrieval and Fusion: Vector Plus BM25

Each query runs two retrievers in parallel: dense vector similarity based on pgvector, and sparse BM25 keyword search inside Postgres. Both semantic matches and exact word matches enter the candidate set, and Reciprocal Rank Fusion is then used to merge the two ranked lists. The advantage of RRF is that it requires no manual tuning of score weights—chunks that rank high in both lists naturally float to the top.

BM25 is a classic sparse retrieval algorithm in information retrieval. It scores documents based on term frequency (TF) and inverse document frequency (IDF), applies length normalization to long documents, and favors documents containing the query keywords. Unlike dense vector retrieval, BM25 relies on exact term matching, so it is especially sensitive to rare words such as proper nouns, model numbers, and code identifiers—precisely the cases that vector models tend to blur. RRF (Reciprocal Rank Fusion) is a training-free, tuning-free rank fusion method: for each document, take the reciprocal of its rank in each retrieval list and sum them, so higher ranks contribute more. Its advantage is that it uses only ranks rather than raw scores, so there is no need to worry about the incompatibility of scales between vector similarity and BM25 scores; it is simple to implement and robust.

Reranking and Filtering: Blocking Noise Triggered by "Hello"

Fused candidates are re-scored against the actual question by a cross-encoder reranker (Cohere, Jina, Voyage, etc.), and only the top passages are kept. A similarity gate discards weak matches, preventing a single greeting from dragging in irrelevant chunks. Query rewriting handles follow-ups like "what about the second one?"—the query is first rewritten using conversation context before retrieval, and when a query is too vague, the agent can proactively ask for clarification.

Cross-encoders are fundamentally different in structure from the common dual-tower embedding models. Dual-tower models encode the query and document separately into vectors and then compute similarity; document vectors can be precomputed offline, making them suitable for large-scale recall. Cross-encoders, by contrast, concatenate the query and document and feed them together into a Transformer, letting the attention mechanism fully interact between the two, so scoring is more precise—but document representations cannot be precomputed, and inference must be done one by one over a small number of candidates. This is exactly why it is placed at the end of the retrieval pipeline, reranking only the top passages after fusion: trading acceptable latency for ranking quality. A common practice is to first recall dozens to hundreds of candidates with a cheap retriever, then use a cross-encoder to precisely rerank the top few for the generative model.

Citations and Guardrails: Verifiable and Controllable

Every answer links back to its source chunks, so users can check the original text instead of blindly trusting the model. Guardrails intercept prompt injection, off-topic questions, toxic content, and personal data before content reaches the model. Retrieval evaluation lets you write test questions once and have an LLM-as-a-judge score each answer, so when adjusting chunking, embeddings, or rerankers, you can first know whether retrieval is getting better or worse before deciding whether to ship.

LLM-as-a-judge is the mainstream paradigm for evaluating generative systems today: a capable model scores system outputs against given criteria (such as relevance, faithfulness, and completeness), replacing expensive and slow human annotation. In RAG scenarios, it can batch-evaluate "whether the answer is supported by the retrieved source text," enabling quick comparison of retrieval quality after changing chunk size, embedding model, or reranker. Its limitations should also be noted: judge models may exhibit position bias, length bias, and biases related to their own training distribution, and the quality of the rubric design directly determines the reliability of conclusions. In practice, therefore, it is often paired with a small amount of human annotation for calibration, and key metrics are sampled multiple times to reduce variance.

Generation Models and Peripheral Capabilities

The generation side is not tied to a single vendor; open-weight models such as Qwen3, or models on OpenRouter, Mistral, Groq, and Scaleway, can be switched at any time. Beyond the pipeline, the platform provides a chat widget, hosted chat pages, a ⌘K command bar, and integrations with Slack, WhatsApp, Telegram, Teams, Instagram, and Messenger. Tools and MCP support approvals and logging, and teams, roles, audit logs, two-factor authentication, and passkeys are also within the open-source scope.

Open-Source Boundary and Commercialization: What the Pro Key Unlocks

The author adopts a common open-source business model: core RAG capabilities are free, and a Pro key unlocks the REST API, live human takeover, in-chat forms and lead capture, voice conversations, image/video/audio generation, deeper analytics, custom domains, and more workspaces. The author says plainly that a project maintained by one person needs to support itself. For self-hosting teams, this means they can first validate retrieval quality at zero cost and then pay for peripheral features as needed.

Why "the Retrieval Stack Moving into Postgres" Is Worth Noticing

Putting vector retrieval, BM25, and RRF fusion all inside Postgres reduces the number of operational components. For small and medium teams, one fewer dedicated vector database means one fewer set of backups, monitoring, and permission systems. The halfvec plus HNSW trade-off (halved index, unchanged recall) has practical significance in storage-cost-sensitive scenarios. However, it also ties the performance ceiling to Postgres tuning, and behavior under large-scale concurrency needs real load testing to verify. The author's invitation at the end of the post is direct: if you've tried it, tell me what's broken.

Who It's For

Teams with their own documents, sensitivity about data leaving their domain, and no desire to pay a subscription fee per agent are the most direct target users of this project. RAG developers who have already stumbled over chunking, recall, and reranking pitfalls can also find reference points in its tunable parameters and retrieval evaluation design.

One-line Summary

chatVia.ai is an open-source RAG platform that stuffs the full retrieval stack into Postgres, with a free core and peripheral features unlocked by a Pro key.

Paragraph Summary

chatVia.ai is a RAG platform built over a year by a single developer and open-sourced. Its core idea is to put vector retrieval, BM25 keyword search, and RRF fusion all inside Postgres, eliminating the need for a separate vector database and keeping data on your own servers. Its pipeline covers content-aware chunking, LLM contextual enrichment, multi-vendor embeddings, hybrid retrieval, cross-encoder reranking, query rewriting, guardrails, and citation traceback, and it offers LLM-as-a-judge retrieval evaluation so you can judge before shipping whether a change improves retrieval. The platform supports multiple generation models and comes with peripheral capabilities such as a chat widget, multi-platform messaging integrations, and team permissions. In terms of business model, the core RAG capabilities are free and open source, while a Pro key unlocks features such as the REST API, human takeover, and voice and multimodal generation.