fix: suppress citation SSE events for unresolved (hallucinated) citation IDs - #2287
Open
octo-patch wants to merge 1 commit into
Open
Conversation
When an LLM generates a citation ID that does not match any retrieved
chunk, SSEFormatter.yield_citation_event was explicitly setting
payload=None and emitting the event, producing a confusing stream
message of the form {"is_new":true,"payload":null}.
The previous condition
not is_new or "payload" not in citation_data
mistakenly treated a missing "payload" key the same as a repeated
citation. Only repeated citations (is_new=False) should have their
payload nulled out; a new citation with no resolvable source should
be silently suppressed instead.
After this fix:
- Repeated citations: payload set to null (existing behaviour)
- New citations with a found source: payload included (existing behaviour)
- New citations with no matching source: event suppressed, no null-
payload surprise for clients (fixes SciPhi-AI#2224)
Co-Authored-By: Octopus <liyuan851277048@icloud.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2224
Problem
When streaming RAG responses, the LLM sometimes generates citation IDs that do not correspond to any retrieved document chunk (i.e., hallucinated citations). In
SSEFormatter.yield_citation_event, the previous conditiontreated a missing
"payload"key (new citation, source not found) the same as a repeated citation (where nulling the payload is intentional). As a result, the server emitted streaming events like:{"id":"c910e2e","object":"citation","is_new":true,"span":{"start":411,"end":420},"payload":null}Clients receiving
is_new=truewithpayload=nullcannot distinguish between a genuinely new citation (where they expect full source data) and a hallucinated one.Solution
Split the condition into two explicit branches:
is_new=false): setpayload=nullas before — the client already has the payload from the first occurrence.is_new=true, no"payload"key): silently suppress the event. The citation ID was not in the retrieved chunks, so there is nothing meaningful to send. The citation also does not appear in the finalfinal_answerevent (that path already filters oncitation_payloads), keeping streaming and final-answer behaviour consistent.is_new=true,"payload"key present): unchanged — emits with full payload.Testing
payload=null.search_results_collector) no longer produce a streaming event, consistent with the final-answer behaviour.