What a real citation is in a RAG system (and why most chatbots fake it)

TL;DR

A real citation connects each claim in an answer to the exact passage that supports it: document, page and span. Build it by keeping source metadata on every chunk, making the model cite chunk IDs, validating those IDs after generation, and refusing when nothing supports the answer.

Quick answers

What is a citation in a RAG system?: A pointer from a specific claim in the answer to the specific passage it came from: document, page and ideally the exact span of text.
Why do chatbots show wrong sources?: Because many just list the retrieved documents under the answer, whether the model used them or not. The sources are decoration, not evidence.
How do you stop an AI from inventing answers?: Validate its citations after generation and refuse when the retrieved passages don't support the question. Saying "that isn't in your documents" is a feature.

The idea behind the name

Byte Zentrix is built on one idea: every answer traced back to the exact byte it came from. A claim without a source is only an opinion, however confident it sounds. That's the standard AI answers should meet, and it's where this studio's name comes from.

Fake citations vs real citations

	Decorative sources	Real citations
What's shown	Every retrieved document	Only passages used for a claim
Granularity	Whole file	Document, page, span
Checked?	No	IDs validated after generation
When nothing matches	Answers anyway	Refuses and says why

Step 1: never lose where a chunk came from

Citations are decided at ingestion time, not generation time. If a chunk doesn't carry its document, page and character offsets, you can't point back to it later. Store them with every chunk.

python

from dataclasses import dataclass

@dataclass
class Chunk:
    id: str          # "msa-2024#p12#c3"
    doc: str         # "Master Service Agreement 2024.pdf"
    page: int        # 12
    start: int       # char offset on the page
    end: int
    text: str

Step 2: make the model cite IDs, not titles

Give each retrieved passage a short ID in the prompt and require the model to put the ID after every claim. IDs are easy to validate; free-text titles are easy to hallucinate.

text

Answer using ONLY the sources below.
After every sentence, cite the source id in brackets, e.g. [S2].
If the sources do not answer the question, reply exactly:
"That isn't covered in the documents."

[S1] (MSA 2024, p.12) Refunds are available within 45 days for Premium SLA...
[S2] (Refund Policy v3, p.4) Refunds are prorated to unused service days...

Step 3: validate after generation

Parse the IDs out of the answer. Any ID that wasn't in the retrieved set is a hallucinated citation: drop the sentence or regenerate. For higher-stakes domains, add a cheap check that the cited passage actually contains the key terms of the claim.

python

import re

def validate(answer: str, allowed: set[str]) -> bool:
    cited = set(re.findall(r"\[(S\d+)\]", answer))
    return bool(cited) and cited <= allowed

Step 4: design the refusal

If the best retrieval score is below a threshold, or validation fails twice, don't answer. Tell the user what you searched and suggest where the answer might live. Users forgive "I don't know". They don't forgive a confident wrong answer about a contract.

Step 5: show the source, not just the name

Render each citation as a footnote marker next to the claim it supports.
On hover or tap, show the exact passage with the matched span highlighted.
Link to the original document at the right page.

Answer, then show the chain. If there's no chain, don't answer.