Honor parameter distribution in Skopt and Goptuna suggestion services - #2689
Honor parameter distribution in Skopt and Goptuna suggestion services#2689saivedant169 wants to merge 1 commit into
Conversation
The Skopt and Goptuna suggestion services built their search spaces from the parameter type only and ignored the declared distribution: Skopt hardcoded a log-uniform prior for every double parameter, and Goptuna always built a uniform distribution, so a parameter's distribution was silently discarded. Map the distribution to the backend equivalent (uniform and log-uniform) and raise a clear error for distributions the backend cannot sample (normal and log-normal, and log-uniform for Goptuna int parameters), consistent with the Optuna suggestion service. Signed-off-by: Saivedant Hava <saivedant169@gmail.com>
|
Hi @andreyvelich, whenever you get a window, mind taking a look at this one? Same distribution-handling line as #2666, just extended to the skopt and goptuna services, which were silently ignoring the declared distribution. No rush. |
|
Thanks @saivedant169! |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: andreyvelich The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
Hi @andreyvelich, this one has been approved and is green apart from the |
|
@andreyvelich this one's still sitting on the flaky e2e, logs are from July. Mind kicking off a rerun? |
|
/retest |
What this PR does / why we need it
The Skopt and Goptuna suggestion services built their search spaces from
parameter.typeonly and never readparameter.distribution, so a declared distribution was silently discarded:skopt/base_service.py) hardcoded a"log-uniform"prior for every double parameter and left integer parameters uniform, so adistribution: uniformdouble (the default) was sampled log-uniformly.goptuna/converter.go) always builtUniformDistribution/IntUniformDistribution, so adistribution: logUniformparameter was sampled uniformly.Both now honor the declared distribution. Because both libraries support uniform and log-uniform directly (
skopt.space.Real/Integertake aprior, and goptuna hasLogUniformDistribution), the distribution is mapped through rather than rejected. Fornormal/logNormal, which neither library samples natively (andlogUniformfor goptuna int parameters, which goptuna has no distribution for), the service now raises a clear error naming the parameter and the unsupported distribution, consistent with the Optuna service in #2666.Note this changes the Skopt default for double parameters from log-uniform to uniform, which is the intended correction.
Which issue(s) this PR fixes
Fixes #2688
Testing
converter_test.go(log-uniform maps toLogUniformDistribution;normalon a double andlogUniformon an int return an error). All pass withgo test ./pkg/suggestion/v1beta1/goptuna/.test_skopt_service.pycovering the distribution to prior mapping and the fail-loud path for unsupported distributions. These run undermake pytest-skopt.create_optimizer: a uniform double now yieldsprior="uniform", a log-uniform double yieldsprior="log-uniform", and anormaldouble raisesValueError.