Skip to content

Keep tf_adjustment_weight 0.0 through the settings and creator round trip - #3345

Open
赵瑞哲 (zhaoruizhe81) wants to merge 1 commit into
moj-analytical-services:masterfrom
zhaoruizhe81:fix/issue-3344-tf-weight-zero
Open

赵瑞哲 (zhaoruizhe81) wants to merge 1 commit into
moj-analytical-services:masterfrom
zhaoruizhe81:fix/issue-3344-tf-weight-zero

Conversation

@zhaoruizhe81

Copy link
Copy Markdown

Type of PR

  • BUG
  • FEAT
  • MAINT
  • DOC

Is your Pull Request linked to an existing Issue or Pull Request?

Fixes #3344

Give a brief description for the solution you have provided

What

ComparisonLevel.as_dict() now always writes tf_adjustment_weight when the level has TF adjustments, instead of skipping exactly-zero weights. Adds three regression tests: an as_dict() round trip, the comparison creator path, and an end-to-end predict() check.

Why

The constructor default for tf_adjustment_weight is 1.0, so a level configured with the documented "no adjustment" weight of 0.0 silently came back as 1.0 after any settings dict / model JSON / comparison creator round trip, and save_model_to_json() wrote 1.0 with no warning. This contradicts the docs ("A weight of zero is equivalent to turning TF adjustments off") and made the zero-weight branch in _tf_adjustment_sql() unreachable. It has been a problem on first load since 4.0.7, when create_comparison_dict() started calling as_dict().

Only levels with weight exactly 0.0 change: they now get an explicit "tf_adjustment_weight": 0.0 in the output dict. A weight of 1.0 was already written before this change, so existing saved models are unaffected.

How I verified

  • Repro script from the issue, run on master 68ced2a4 (the base of this branch): before the change it exits 3 (RESULT: BUG PRESENT — weight saved as 1.0, mw_tf_adj_first_name non-zero); with the change it exits 0 (RESULT: OK — weight 0.0, adjustment 0.0).
  • The three new tests in tests/test_tf_adjustment_weight_zero.py all fail on unmodified master and pass with this change (checked by stashing and re-applying the one-line fix).
  • Full DuckDB test suite: 382 passed, 16 skipped, 0 failed (tests for other SQL dialects are deselected locally by conftest.py).
  • ruff check and ruff format --check pass on both changed files.

AI assistance

Per the contributing guidance: this PR was prepared with AI assistance — the patch, the tests and the local verification runs above were produced with the help of an AI coding agent, and I reviewed and submitted the result.

PR Checklist

  • Added documentation for changes
  • Added feature to example notebooks or tutorial (if appropriate)
  • Added tests (if appropriate)
  • Updated CHANGELOG.md (if appropriate)
  • Made changes based off the latest version of Splink
  • Run the linter
  • Run the spellchecker (if appropriate)

…trip

ComparisonLevel.as_dict() skipped tf_adjustment_weight when it was
exactly 0, but the constructor default is 1.0, so any level configured
with the documented "no adjustment" weight 0.0 silently came back as
1.0 after a settings dict / model JSON / comparison creator round trip,
and save_model_to_json() wrote 1.0.

Always write the weight when the level has TF adjustments (a weight of
1.0 was already written, so existing saved models are unaffected) and
add regression tests for the as_dict round trip, the comparison creator
path, and end-to-end predict.

Fixes moj-analytical-services#3344

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tf_adjustment_weight 0.0 is silently reset to 1.0 (as_dict() omits a zero weight)

1 participant