Skip to content

Preserve DataFrame row alignment for array targets in FeatureSelector - #1136

Open
akshitkhokhani wants to merge 1 commit into
blue-yonder:mainfrom
akshitkhokhani:codex/fix-feature-selector-target-index
Open

akshitkhokhani wants to merge 1 commit into
blue-yonder:mainfrom
akshitkhokhani:codex/fix-feature-selector-target-index

Conversation

@akshitkhokhani

Copy link
Copy Markdown

FeatureSelector.fit gives ndarray targets a new default index even when the feature DataFrame has a different index. This breaks sklearn pipeline cross-validation with an index assertion. If the DataFrame instead has permuted integer index labels, selection can silently associate targets with the wrong rows and discard a predictive feature.

Construct the target Series with the feature DataFrame's index so ndarray targets follow row order. Explicit Series targets keep their existing label alignment. The fit documentation clarifies both behaviors.

Adds regressions for nondefault and permuted indexes, reordered Series targets, and an actual sklearn pipeline cross-validation workflow. Related to the pipeline workflow discussed in #960; that issue does not establish this specific ndarray cause.

Validation: all 10 selector unit tests pass, including existing ndarray-input and multiclass cases. Both new tests fail before the fix. Black 22.12.0, isort 5.12.0 and git diff --check pass. No full-suite claim.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant