Fix AI Guard message truncation to keep newest messages - #3
Open
Copilot wants to merge 1 commit into
Open
Conversation
…dest
The truncate_messages method in Evaluation used Array#first(max_length),
which kept the oldest messages and dropped the newest ones when the
message count exceeded max_messages_length. This contradicted the
documented behavior ('Older messages are omitted once the message limit
is reached') and caused the messages recorded on the ai_guard span's
metastruct tag to show stale context instead of the most recent
conversation turns.
This only affected the messages surfaced in the span for observability;
the full, untruncated message list was still sent to the AI Guard
evaluation API, so evaluation decisions were unaffected.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes
truncate_messagesinlib/datadog/ai_guard/evaluation.rbso that when the number of messages exceedsmax_messages_length, the newest messages are kept and the oldest are dropped, instead of the other way around.Motivation:
Users reported that AI Guard keeps the oldest messages and drops the newest ones once the message limit is hit. The bug was
serialized_messages.first(max_length), which keeps the oldestnmessages. This contradicts the documented behavior inconfiguration.rb: "Older messages are omitted once the message limit is reached." Swapped to.last(max_length)to match the documented/intended behavior.Note: this only affects the
messagesfield recorded on theai_guardspan's metastruct tag (used for observability in the trace UI). The full, untruncated message list is still sent to the AI Guard evaluation API, so evaluation decisions were never affected by this bug.Change log entry
Yes. Fixed AI Guard to retain the most recent messages (instead of the oldest) when truncating messages recorded in the trace once
max_messages_lengthis exceeded.Additional Notes:
Generated with AI assistance; the change and tests were reviewed before submission.
How to test the change?
Updated the existing spec in
spec/datadog/ai_guard/evaluation_spec.rb("truncates metastruct messages to max_messages_length") which previously asserted the buggy behavior (keeping the oldest messages); it now asserts the newest messages are retained. Ran:125 examples, 0 failures.