Skip to content

Ignore unused categorical IDs during feature extraction - #1137

Open
AryanRG wants to merge 1 commit into
blue-yonder:mainfrom
AryanRG:fix/observed-categorical-ids
Open

AryanRG wants to merge 1 commit into
blue-yonder:mainfrom
AryanRG:fix/observed-categorical-ids

Conversation

@AryanRG

@AryanRG AryanRG commented Oct 23, 2026

Copy link
Copy Markdown

With pandas 2, an ID column containing unused categories makes wide and dictionary inputs produce feature rows for IDs absent from the data. This can happen after filtering a categorical column.

Use observed=True when grouping these inputs so only IDs with observations are processed. Add extraction regressions covering wide, long and dictionary inputs, with and without sorting, while preserving the categorical index metadata.

Validation: feature-extraction tests excluding Dask, Black and isort pass. Dask and the full dependency matrix were not run.

Prepared with AI assistance.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant