Skip to content

Make power cone projection scale-invariant (fixes #491) - #493

Merged
bodono merged 9 commits into
masterfrom
fix-pow-cone-scale-invariant
Oct 16, 2026
Merged

bodono merged 9 commits into
masterfrom
fix-pow-cone-scale-invariant

Conversation

@bodono

@bodono bodono commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Fixes #491.

Problem

proj_power_cone used absolute tolerances (POW_CONE_TOL = 1e-9) for the membership checks and the Newton stopping rule |f| < tol. With normalization and Anderson acceleration, the iterates that reach the projection in the issue's problem are tiny (~1e-6 to 1e-9). Newton stopped at r = 2.69e-9, f = -5.2e-10, which passes the absolute test but is ~24% from the root. That is exactly the reported x = 0.1238 vs. optimal 0.1. The residuals don't catch this because they assume the projection is exact, so SCS returned SCS_SOLVED, and it does so at every eps, not only 1e-8.

Fix (src/cones.c, proj_power_cone)

  • Scale invariance: project v / ||v||_inf and rescale the result. Projection onto a cone is positively homogeneous, so the tolerances become relative to |v|.
  • Cancellation-free x(r): when xh < 0, x(r) uses a form without cancellation, and the 1e-12 floor is gone. The floor overrode the true value when a component was tiny and negative.
  • Bracketed Newton: phi(r) = x^a y^(1-a) is concave, so the sign of f = phi - r brackets the root. When a Newton step leaves [lo, hi] (from the left of a concave f it overshoots), an Illinois regula falsi step is used, then bisection. Bisection alone needed ~50 iterations when the root is within a few ulps of rh.
  • Near the apex: right of the root, Newton on f(r)/r is tried (and used if it stays in the bracket). That form removes the trivial root at 0 and converges much faster than Newton on f, which is only linear near the double root. Restricting it to the near-apex regime saved a few projection evaluations on some inputs but made whole solves worse (see the solver sweep below), so it is taken whenever f < 0.
  • Starting point: by default r = rh/2, with two better guesses where they apply.
    • Exactly one of xh, yh negative is the most common case for ADMM iterates. There, start at the root of the small-r approximation x(r) ~ a * rh * r / |xh|: r0 = (a * rh / |xh|)^(a / (1 - a)) * yh, or the mirror image. The true root is often tiny, and Newton from rh/2 took many shrinking steps to reach it.
    • For points just outside the cone (phi(rh) >= rh/2), such as converging iterates, start at r = rh. This only applies when |dx/dr| and |dy/dr| at rh are at most POW_CONE_START_SLOPE; otherwise the first step is too small to represent and looks converged.
    • Correctness doesn't depend on the guesses, since the bracket safeguards every step.
  • Stopping rule: stop on the step, not on |f|. The step must be small relative to r and, scaled by sens = max(1, |dx/dr|, |dy/dr|), small relative to |v|. Near the apex x can be ~1000x more sensitive to r than r is. The relative part also keeps tiny components accurate relative to themselves when the root is near 0 (near the apex), which is the case that matters for the solves below. When the root is within a few ulps of rh and a component is tiny, no stopping rule in this r parametrization can do that; the error there is still small relative to |v| (the 2.3e-9 row below). The loop also stops when the bracket is that narrow ((hi - lo) * sens <= TOL) or only a few ulps wide.
  • Constants: POW_CONE_TOL = 1e-12, POW_CONE_START_SLOPE = 1e3, POW_CONE_EPS = DBL_EPSILON (1e-6, 10, FLT_EPSILON under SFLOAT). POW_CONE_MAX_ITERS = 50.

Testing

  • test_power_cone_small_scale uses the issue's exact data. It fails on master and passes here.

  • test_power_cone_proj_tiny_component projects points just outside the cone that have one tiny positive component and p = 0.001 / 0.999, and checks them against 60-digit reference values. It fails on an intermediate commit of this PR that returned them unprojected.

  • run_tests_direct and run_tests_indirect pass. The SFLOAT library builds with only a warning that is already on master.

  • Issue reproducer: x = 0.09999999974, dist(s, K) = 1e-17 (was x = 0.1238, dist = 2.9e-3).

  • Projection accuracy vs. a 60-digit decimal reference (KKT-verified on most sets; on the extreme-p / adversarial sets some true roots are ~1e-3000, so the reference is exact only to ~1e-59, far below double precision). Max error relative to ||v||_inf, mean Newton evaluations, and iteration-cap hits, 3000 points each:

    point set master this PR
    typical (` v ` in 1e-3..1e3)
    just outside the cone 3.1e-5, 3.3 its, 77 at cap 1.2e-12, 1.7 its
    tiny positive component, just outside, p near 0/1 2.9e-2, 13.4 its, 732 at cap 2.3e-9, 4.5 its
    near the issue's dual solution (near-polar) 8.0e-8, 14.0 its 9.3e-13, 25.3 its
    extreme p (0.001, 0.01, 0.99, 0.999) 5.3e-5, 7.5 its, 363 at cap 2.6e-12, 5.4 its
    adversarial (badly scaled / near apex) 6.2e-1, 10.4 its, 479 at cap 1.9e-12, 10.7 its

    This PR never hits the 50-iteration cap in these sets.

  • CVXPY benchmark: 52 models built with real 3-d power cones (approx=False), solved by scs-python built against master and against this PR on macOS arm64 / Accelerate, at eps = 1e-5 and 1e-8. The models are p-norm regression (p = 1.1, 1.5, 3), geometric-mean maximisation (uniform and skewed weights), power utility (p = 0.05, 0.3, 0.95), a power-penalty portfolio, CVXPY's pcp_1/2/3, and the issue's problem.

    • Accuracy: no objective is worse than master (vs. a tight Clarabel solve), and no model goes from solved to not solved. Two go the other way (skewed geometric means). The power-cone violation of the returned s is at most 0.06x the primal tolerance; on master it is up to 0.19x, and 2.4e4x on the issue's problem.
    • Speed (measured on 720f475, which differs from the final code only in when the f(r)/r step is taken): total solve time is dominated by iteration counts, which shift between builds in both directions because the ADMM trajectory is sensitive to last-bit differences in the projection. Per iteration, cone time is +8% vs. master (geometric mean over the 104 solves), and solve time is +8%. That ranges from 0.73-0.99x on p-norm (p = 1.1), portfolio and p = 0.95 utility models, to 1.17-1.45x on geometric-mean, pcp and p = 0.05/0.3 utility models. Before the last commit (speed-ups from profiling recorded projection inputs) cone time per iteration was +24%.
    • CVXPY's own SCS / power-cone tests (test_conic_solvers.py::TestSCS, test_pow_cone_nd.py, test_power_atom.py, test_power_tools.py, test_perspective.py, test_atoms.py) pass identically on both builds.
  • Solver sweep, 390 problems: the 78 below with u jittered by 0-4%, eps = 1e-8. Not solved / total iterations: master (78-problem version) 61 / 6.2M; f(r)/r step only when Newton on f would more than halve r (720f475) 18 / 2.77M; only when both components are negative 26 / 3.48M; whenever f < 0 (final) 16 / 2.50M.

  • Earlier solver sweep (measured on fd25f7a), min -z s.t. (u, 1, z) in K_p with u in 1e-12..1e-6 and p in {0.001, 0.01, 0.1, 0.9, 0.99, 0.999}, eps = 1e-8 (78 problems): master leaves 61 not solved (6.2M total iterations); this PR leaves 3 (476k). These problems are badly conditioned, so individual outcomes are sensitive to rounding-level changes in the projection. For p = 0.9 and small u, z* is below eps_abs, so errors in z there reflect the solver tolerance, not the projection.

  • Reviewed five times by a second model; the last review approved the final code with no substantive findings. Earlier findings that are addressed: a converged step being discarded for bisection, cap hits when the root is near 0, a start guard that let tiny components through for small p, and the stopping-rule nits. One suggestion, stopping on the sensitivity-scaled step alone, made tiny-u solves worse, so it is combined with the relative test instead.

🤖 Generated with Claude Code

bodono and others added 9 commits October 14, 2026 18:16
proj_power_cone used absolute tolerances (POW_CONE_TOL = 1e-9) for the
membership checks and the Newton stopping rule. When the homogeneous,
normalized iterates are small, Newton stopped with r far from the root
and SCS reported SCS_SOLVED with the slack well outside the cone
(issue #491: x = 0.1238 vs optimal 0.1).

- Project v scaled to unit inf-norm and rescale the result, so all
  tolerances are relative to |v|.
- Stop Newton on step size rather than |f|: near the polar boundary f is
  very flat in r, so a small |f| does not imply an accurate r.
- Tighten POW_CONE_TOL to 1e-12 (now relative).
- Add test_power_cone_small_scale regression test from the issue.

Fixes #491

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI showed the regression test's dual cone check failing on Linux: for
points just outside the polar cone, f(r) has a near-double root and the
Newton iteration either crawled to the iteration cap or stopped on a
spurious small step clamped to r = rh.

- Evaluate x(r) in a cancellation-free form and drop the 1e-12 floor,
  which overrode the true value when a component was tiny and negative.
- f(r) = phi(r) - r with phi concave, so the sign of f brackets the root.
  Keep [lo, hi] and fall back to bisection when a step leaves it.
- On the right of the root also try the Newton step for f(r) / r, which
  converges fast near the double root.
- Raise POW_CONE_MAX_ITERS to 50 (only hit in hard cases; mean ~7.5 on
  well-scaled inputs).
- Use c = -100 in the regression test (still fails on master) so |y|
  is moderate for verify_solution_correct's absolute dual cone check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit changed the regression test's objective from -1000
to -100 to get past verify_solution_correct's absolute 1e-5 dual cone
check. That masked a real inaccuracy rather than fixing it, so restore
the issue's -1000.

The inaccuracy: the Newton loop stopped once |r_new - r| <= 1e-12 in
normalized units, but for inputs near the apex (e.g. -y for the issue's
dual solution, |y| ~ 7500) the root r is ~1e-15 while x(r) is ~1000x
more sensitive to r than r itself. Against a 60-digit reference that
error reached 1.25e-9 relative (~9e-6 absolute here), right at the
test's threshold. Stopping relative to r brings it to ~1.6e-15.

Also use a 1e-6 tolerance under SFLOAT so it is reachable in float.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A converged Newton step lands on r, which is an end of the bracket,
  so the strict bracket test rejected it and replaced it with a
  bisection midpoint, wasting many iterations. Test convergence on the
  raw step (closed bracket) before the bisection fallback.
- The step test relative to r cannot be met when the root is at or near
  0 (near the apex, or p close to 0 or 1), so those calls ran to the
  iteration cap. Also stop once the bracket width, scaled by the
  sensitivity of x and y to r, is below POW_CONE_TOL.
- Tone down the comment on the f(r) / r step: it only helps once r is
  close to the root.

Against a 60-digit reference this keeps max error <= 1e-12 relative on
typical, near-apex and extreme-p inputs, with no cap hits, and mean
iterations on typical inputs drop from 8.1 to 5.9 (master: 4.9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Converging iterates are typically just outside the cone, with the root
near rh. Starting at rh / 2, Newton on the concave f overshoots past the
root from the left and the bracket safeguard falls back to bisection.
Start at rh instead when x(rh) = xh and y(rh) = yh are positive and
phi(rh) >= rh / 2; the guard matters, since with a tiny xh the slope at
rh is huge and the first step is spuriously small.

Against the 60-digit reference: points just outside the primal boundary
drop from 17.1 to 1.5 mean iterations (max error 8.7e-9 -> 1.1e-9);
typical, near-apex, extreme-p and adversarial sets are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
63c0fa4 started the Newton solve at r = rh when phi(rh) >= rh / 2. For
small p that does not exclude a tiny positive component (xh^p ~ 1), and
there dx/dr = -a * rh / xh is so large that the first step rounds to 0,
which the step test took for convergence: e.g. p = 0.001 and
v = (1e-12, 1, 1.848) was returned unprojected (error 0.43).

- Start at rh only when |dx/dr| and |dy/dr| at rh are at most
  POW_CONE_START_SLOPE (1e3; 10 under SFLOAT).
- Converged step: also require |step| * max(1, |dx/dr|, |dy/dr|) <=
  POW_CONE_TOL, in addition to the relative-to-r test. (The sensitivity
  test alone, without the relative one, let tiny components come out
  inaccurate relative to themselves and made solves on such problems
  worse.)
- Bracket stop: drop hi - lo <= POW_CONE_TOL * hi, which could fire with
  large sensitivity; stop when (hi - lo) * sens <= POW_CONE_TOL or when
  the bracket is a few ulps wide (new POW_CONE_EPS).
- When a Newton step leaves the bracket, use an Illinois regula falsi
  step before bisecting: bisection alone needed ~50 iterations when the
  root is within ulps of rh (a point just outside the cone with a tiny
  component), and hit the iteration cap.
- Add test_power_cone_proj_tiny_component, checking the projection of
  such points against 60-digit reference values; it fails on 63c0fa4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nits: take POW_CONE_EPS from <float.h> instead of hard-coding it,
note why POW_CONE_START_SLOPE is smaller under SFLOAT, and say what
f_lo / f_hi are where they are declared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiling on projection inputs recorded from CVXPY power-cone models
showed nearly every call runs the Newton loop, and that most of the extra
cost over master was extra Newton evaluations from poor starting points:

- With exactly one negative component (the common case for ADMM
  iterates), start at the root of the small-r approximation
  x(r) ~ a * rh * r / |xh|, i.e. r0 = (a * rh / |xh|)^(a / (1 - a)) * yh
  (or the mirror image), instead of rh / 2. The root is often tiny and
  Newton from rh / 2 took many shrinking steps to reach it.
- Start at r = rh only when the point is close to the cone
  (phi(rh) >= rh / 2), as well as having moderate slopes there.
- Take the Newton step on f(r) / r only when Newton on f would more than
  halve r (its slow regime near the apex); elsewhere it overshoots left of
  the root and alternates with the f step.
- Compute phi(0) once for the membership check and the bracket.

Accuracy against a 60-digit reference is unchanged or better on every
point set, with no iteration-cap hits. On the CVXPY benchmark, cone time
per iteration relative to master goes from +24% to +8% (geometric mean),
and is below master on p-norm, portfolio and p = 0.95 utility models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
720f475 restricted the Newton step on f(r) / r to when Newton on f
would more than halve r. Review showed that trigger is false in the
near-double-root regime the step exists for (the f-Newton ratio there is
above 1/2), so near-polar inputs, issue #491's class, lost it. A trigger
on both components being negative fixed the projection-level cost, but
whole solves were what mattered: on 390 jittered tiny-u solves
(min -z s.t. (u, 1, z) in K_p), unsolved / total iterations were

  r_new < r / 2 (720f475):        18 / 2.77M
  xh < 0 && yh < 0:                26 / 3.48M
  unrestricted (this commit):      16 / 2.50M

and the unrestricted step was also best on the reviewer's own sweep. The
new starting points from 720f475 are kept. Accuracy against the
60-digit reference is unchanged on every point set; mean evaluations go
from 30.0 to 25.3 near the issue's dual solution and 11.9 to 10.7 on the
adversarial set, and 4.7 to 4.8 on typical points.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bodono
bodono merged commit e9e57c5 into master Oct 16, 2026
94 of 95 checks passed
@bodono
bodono deleted the fix-pow-cone-scale-invariant branch October 16, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant