Skip to content

fix: raise FilterError for incomparable document values in range filters - #13618

Open
Harshil26202 wants to merge 1 commit into
deepset-ai:mainfrom
Harshil26202:fix/filter-ordering-incomparable-document-value
Open

Harshil26202 wants to merge 1 commit into
deepset-ai:mainfrom
Harshil26202:fix/filter-ordering-incomparable-document-value

Conversation

@Harshil26202

Copy link
Copy Markdown

Related Issues

  • No existing issue. Found while checking filter edge cases against InMemoryDocumentStore.

Proposed Changes:

>, >=, < and <= filters raised a raw TypeError when a document stored a list or dict in the filtered field. For example, {"field": "meta.page", "operator": ">", "value": 1} fails on a store containing Document(meta={"page": [1, 2]}) with TypeError: '>' not supported between instances of 'list' and 'int'. One such document is enough to break the filter for the whole store, and callers catching FilterError don't catch it.

These operators already raise FilterError for other incomparable values (a non-date string, or a list as the filter value). This change routes the final comparison through a small _compare helper that turns the TypeError into a FilterError with the two types in the message.

How did you test it?

  • New test_document_matches_filter_raises_filter_error_for_incomparable_document_value (4 operators x list and dict document values). All 8 cases fail on main with TypeError and pass with this change
  • hatch run test:unit test/utils/test_filters.py: 166 passed
  • hatch run test:types: no issues in 568 source files
  • hatch run fmt: clean

Notes for the reviewer

I put the test in test/utils/test_filters.py rather than the FilterDocumentsTest mixin, because the mixin also runs against external document stores that filter natively and may legitimately handle this case differently.

This PR was fully generated with an AI assistant (Claude Code). I have reviewed the changes and run the relevant tests.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

A list or dict stored in the filtered field made >, >=, < and <= filters
raise a raw TypeError from the comparison. Convert it to FilterError, which
is what these operators already raise for other incomparable values.
@Harshil26202
Harshil26202 requested a review from a team as a code owner November 7, 2026 18:32
@Harshil26202
Harshil26202 requested review from bogdankostic and removed request for a team November 7, 2026 18:32
@vercel

vercel Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

@Harshil26202 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Nov 7, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@chrikrah chrikrah left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Harshil26202 nothing blocks a merge. Reverting filters.py fails 8 tests:

# head 59bb7192, base 0ad74747, python 3.12, haystack imported from the exported head tree
$ PYTHONPATH=. python -m pytest test/utils/test_filters.py -q -p no:cacheprovider
# head
166 passed in 0.19s
# head with haystack/utils/filters.py reverted to base
8 failed, 158 passed in 0.26s

# Document(meta={"page": 3}), Document(meta={"page": [1, 2]}), filter meta.page > 1
# head: InMemoryDocumentStore.filter_documents and MetadataRouter.run both raise FilterError
# base: both raise TypeError: '>' not supported between instances of 'list' and 'int'

# filter side: {"field": "meta.page", "operator": ">", "value": {"a": 1}} on Document(meta={"page": 1})
# head: FilterError: Can't compare a value of type 'int' with a value of type 'dict' ...
# base: TypeError

non-blocking: one list-valued doc still fails the whole query, where None doesn't match. I'd keep raising, like bad date strings.

@bogdankostic bogdankostic self-assigned this Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants