Skip to content

Avoid integer overflow in energy features - #1133

Open
antonsoo wants to merge 1 commit into
blue-yonder:mainfrom
antonsoo:fix/integer-energy-features
Open

antonsoo wants to merge 1 commit into
blue-yonder:mainfrom
antonsoo:fix/integer-energy-features

Conversation

@antonsoo

Copy link
Copy Markdown

Integer samples retain their dtype through extract_features, so energy calculations can overflow before returning a floating result. For example:

import numpy as np
import pandas as pd
from tsfresh import extract_features

frame = pd.DataFrame({
    "id": [0] * 4,
    "sample": np.array([20000, 30000, -20000, -30000], dtype=np.int16),
})
features = extract_features(
    frame, column_id="id", n_jobs=0, disable_progressbar=True,
    default_fc_parameters={"abs_energy": None, "root_mean_square": None},
)
print(features.iloc[0].to_dict())
# Before: {'sample__abs_energy': -9728.0, 'sample__root_mean_square': nan}
# After:  {'sample__abs_energy': 2600000000.0,
#          'sample__root_mean_square': 25495.097567963923}

Promote integer and Boolean inputs to float64 before squaring or taking the dot product in abs_energy, root_mean_square and energy_ratio_by_chunks. Chunk ratios have the same problem: the numerator uses a floating exponent but the denominator currently squares in the integer dtype. Floating input dtypes and unrelated calculators are preserved.

Regression checks cover signed and unsigned integers from 8 to 64 bits, Boolean arrays/Series, uneven chunk partitions, and native extraction. The full suite passes (321 passed, 14 skipped). Changed-file hooks pass; all-file hooks reproduce one existing README whitespace violation on unchanged main.

Additional checks load speech and music into 8/16/24/32-bit PCM WAVs and extract 192 windows through serial and multiprocessing paths. Independent Python-integer sums of squares identify 926 wrong feature values before the change and none after it. All 384 floating-input windows and six unrelated features across 576 rows remain byte-identical. This checks arithmetic and extraction behavior, not classification quality. Integer inputs require a float64 work array; full-width integer energy remains a floating approximation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant