AI Agents Aren't Hallucinating — They're Using the Wrong "Version" of Reality

When "Correct" Numbers Add Up to the Wrong Answer

When AI agents produce incorrect output, we tend to blame "hallucination." But a point raised by a researcher on Reddit deserves a second look: the agent retrieved a completely legitimate, verifiably real source, extracted correct numbers from it, and even cited it properly — yet still arrived at the wrong answer. The problem isn't that the model fabricated content; it's that this piece of information belongs to a different point in time.

This distinction seems subtle, but it points to a severely underestimated failure mode in Agentic AI: the fact itself is true, but it doesn't hold for the current question. Better embedding models, larger context windows, or stacking another reasoning agent on top won't automatically fix this. What the system needs to know is: which facts actually belong to the same question along the time dimension.

How Temporal Mismatches Turn Analysis Into "Garbage"

The author illustrates the severity of the problem with a concrete financial scenario. Suppose an agent is analyzing a company's revenue data across multiple quarters. It retrieves several revenue figures — some from restated financials, some from different reporting periods, and some pointing to different entities.

Each number is correct in its original context, and the citation links can be clicked and verified. But when the system assembles these "individually correct" numbers together, the final analytical conclusion is meaningless. This type of error is extremely hard to spot: traditional fact-checking processes can barely detect it, because every step traces back to a legitimate source.

Common engineering intuition fails here. Vector retrieval excels at finding "semantically relevant" content, but it can't determine "whether this piece of evidence satisfies the temporal constraints of the current question." Expanding the context window just stuffs more conflicting temporal versions into the prompt, potentially amplifying the confusion.

Vector retrieval (dense retrieval) is the mainstream paradigm in current RAG systems: encode queries and document chunks into high-dimensional vectors, then recall the closest chunks by cosine similarity or similar measures. Its strength is semantic generalization — it can match even when the question's wording differs from the original text. But similarity essentially only captures "semantic proximity" and carries no temporal constraints: a company's Q2 2024 and Q2 2025 revenue statements are nearly indistinguishable in vector space, and pre- and post-restatement financial versions are also highly similar. Pure vector recall therefore returns multiple temporal versions together, with no mechanism to determine which one satisfies the current question's temporal premises. This is why the author in OETRA retains dense search as a recall method but additionally introduces explicit obligations and a temporal allocation layer.

The OETRA Approach: Define What Evidence You Need First, Then Go Find It

The author built a research system called ChronoRAG-G for this purpose, with OETRA (Obligation-Explicit Temporal Retrieval and Allocation) as the underlying method. Its core shift is: instead of letting the retrieval module "find relevant material," it first precisely derives which specific facts are needed to answer the question.

For example, if a question requires Company A's revenue in Q2 2024, Company A's revenue in Q2 2025, and Company B's corresponding figures, this actually constitutes four independent evidence obligations. The system's process is:

• Create an explicit obligation for each required fact
• Retrieve evidence separately for each obligation, rather than a single fuzzy search
• Preserve source identity, provenance, reporting period, and alternative values in a graph-structured context corpus
• Check temporal and semantic compatibility before allocating evidence
• Ground the final answer on allocated facts, with bounded correction steps

One detail that's easy to misread: the graph structure's role here is to organize evidence and temporal relationships. Retrieval itself still uses dense search, including targeted expansion when evidence is missing. The author's goal isn't to make the model "sound more confident," but to make the evidence behind an answer individually auditable.

Evidence obligation borrows the concept of "obligation" from formal reasoning: write "one specific fact necessary to answer this question" as a checkable constraint, e.g., "entity=Company A, metric=revenue, reporting period=Q2 2024, version=latest restatement." An obligation is a slot to be satisfied, not a natural language query. The benefit: the retrieval target shifts from "relevant" to "verifiably satisfied," and each slot can be independently judged as hit or miss, temporally compatible or not, and whether it needs replacing with an alternative value. The "slot coverage" metrics in the evaluation are based on this structure — strict coverage requires slots to be filled by exactly matching sources, while micro coverage allows relaxation under semantic equivalence. Together they distinguish "did we find it" from "did we find the right one."

What the Evaluation Data Shows

The author evaluated on 480 updated earnings call transcripts using 1,005 temporal QA questions, with the following results:

• 80.70% audited answer accuracy
• 95.92% strict traceable slot coverage
• 98.66% micro required slot coverage
• 97.31% hybrid evidence resolution rate

One finding is particularly noteworthy: 167 answers judged incorrect had complete evidence traceability chains. This means that even if the system successfully resolved every required evidence chunk, that's still not sufficient to guarantee the final answer is correct. Retrieval quality and answer construction quality must be evaluated separately — a blind spot in many current agent evaluation systems.

The author also built an interactive demo that allows viewing frozen evaluation runs, temporal alignment, evidence allocation, and failure trajectories, rather than just accepting these numbers.

Earnings call transcripts are a high-difficulty corpus for temporal QA: companies disclose revenue, guidance, and other figures during quarterly calls, which may later be revised due to restatements, metric adjustments, or subsequent disclosures — and the same metric across different quarters is highly similar in wording. The author's choice of 480 "updated" transcripts means the corpus contains both original and revised versions, naturally creating temporal conflict scenarios. The 1,005 temporal QA questions require answers to be anchored to a specific reporting period and specific version. This design enables the evaluation to directly expose "correct number but wrong version" failures, rather than merely testing whether the model can retrieve relevant passages.

Which Scenarios Actually Need This

Financial research is the most direct application scenario for this method, since the evaluation itself was conducted in this domain. But the author's vision is broader: enterprise agents dealing with constantly changing compliance policies, revised contracts, historical business records, or documents updated every few weeks will encounter the same class of problem.

Merely knowing that a fact is true isn't enough. The agent needs to know when it was true, which source supports it, and whether it's valid for the question currently being answered. The author explicitly states these are potential application directions, not claims of production deployment — current evaluation is also limited to financial temporal QA.

Open Questions for Agent Builders

The question posed at the end of the article is aimed at everyone handling dynamic business data: How do you handle temporal conflicts and revision information? Do you verify which version of a fact the agent actually used, or do you mainly rely on retrieval scores, citations, and final answer evaluation?

The author's view can be summed up in one sentence: An agent knowing when a fact is true is just as important as knowing whether it is true. As more and more agents connect to real-time business data, engineering the "temporal version" dimension may be more worth investing in than continuing to stack reasoning capabilities.

Full methodology and evaluation are available in the research paper (DOI: 10.5281/zenodo.22116070).

One-line Summary

Agent errors often stem from retrieving true facts that belong to the wrong temporal version; OETRA fixes temporal mismatches with explicit evidence obligations.

Paragraph Summary

The article points out that agent output errors don't necessarily stem from hallucination — they may result from retrieving a real information version that belongs to another point in time: the numbers are legitimate, citations verifiable, yet the information doesn't hold for the current question. Using cross-quarter revenue analysis, the author shows how such temporal mismatches render analytical conclusions meaningless, and how traditional fact-checking struggles to detect them. To address this, the OETRA method is proposed: first create explicit evidence obligations for each required fact, then retrieve each one, verify temporal and semantic compatibility, and preserve source identity and reporting period in a graph structure. Evaluation on 480 earnings transcripts and 1,005 temporal QA questions shows high slot coverage, but 167 incorrect answers had complete evidence chains — indicating that retrieval resolution and answer construction must be evaluated separately.