Add non-linear refinement (Levenberg–Marquardt) for homography and affine, as OpenCV does after RANSAC

[kalwalt]

Summary

Both the current minimal-sample fit and the refit proposed in #185 minimise the algebraic DLT residual — ‖Lh‖ subject to ‖h‖ = 1. That is not the same as minimising reprojection error, and the gap is exactly where OpenCV's sub-pixel accuracy comes from.

After RANSAC, cv::findHomography runs 10 iterations of Levenberg–Marquardt over the inliers:

Mat H8(9, 1, CV_64F, H.ptr<double>());
LMSolver::create(makePtr<HomographyRefineCallback>(src, dst), 10)->run(H8);

HomographyRefineCallback minimises the forward transfer error (asymmetric — src → dst only), 2N residuals over 9 parameters, with an analytic Jacobian:

double ww = h[6]*Mx + h[7]*My + h[8];
ww = fabs(ww) > DBL_EPSILON ? 1./ww : 0;
double xi = (h[0]*Mx + h[1]*My + h[2])*ww;
double yi = (h[3]*Mx + h[4]*My + h[5])*ww;
errptr[i*2]   = xi - m[i].x;
errptr[i*2+1] = yi - m[i].y;
 
Jptr[0] = Mx*ww; Jptr[1] = My*ww; Jptr[2] = ww;
Jptr[6] = -Mx*ww*xi; Jptr[7] = -My*ww*xi; Jptr[8] = -ww*xi;
Jptr[12] = Mx*ww; Jptr[13] = My*ww; Jptr[14] = ww;
Jptr[15] = -Mx*ww*yi; Jptr[16] = -My*ww*yi; Jptr[17] = -ww*yi;

cv::estimateAffine2D does the same via Affine2DRefineCallback with refineIters (default 10), and — notably — no extra runKernel: for a linear model, LM from the RANSAC seed converges to the least-squares solution anyway.

jsfeatNext has no LM solver. linalg exposes lu_solve, cholesky_solve, svd_solve, svd_decompose, eigenVV — enough to build one, but nothing ready-made.

Why open this separately from #185

#185's refit is ~20 lines and closes the doc/code mismatch. This is a new numerical component with its own API, tests and portability implications. Bundling them would stall a cheap fix behind an expensive one.

Ordering: #185 (refit) → f64 accumulators → this. Each is measurable on its own. If the adapter's sub-pixel assertion in webarkit/webarkit is meant to hold at full strength rather than at the 1px tolerance currently used in webarkit/webarkit#8, this is the issue that gets it there.

Proposed change

• A linalg.lm_solve (or a small lm_solver_t) taking a callback that fills residuals and Jacobian, with damping, a max-iteration cap and a convergence epsilon — the minimum needed to mirror cv::LMSolver
• homography2d.refine(from, to, model, count, iters) implementing the forward-transfer residual and Jacobian above
• affine2d.refine(...) likewise
• Wired in at whatever layer motion_estimator.ransac: doc comment promises a refit over all inliers; jsfeatNext has no layer that performs one #185 settles on (inside ransac() under Option A, inside find_homography() under Option B)

WASM / no_std portability

Per the boundary discipline we've been holding to: LM needs to solve a damped normal system each iteration. If cholesky_solve on JᵀJ + λI is sufficient, this stays closed-form-ish and portable. If it turns out to need SVD for rank-deficient cases, that conflicts with the no_std constraint we set for anything targeting the Rust/WASM side, and the refinement may have to live only in the TS layer. Worth deciding before implementing rather than after.

Acceptance criteria

• LM solver with unit tests on a known-answer problem (e.g. fit a homography to exact data perturbed by a known amount, assert convergence to ground truth)
• Homography refinement reproduces cv::findHomography's output on a fixed correspondence set to a documented tolerance
• Analytic Jacobian verified against a finite-difference Jacobian in a test
• ww ≈ 0 guarded as OpenCV does (fabs(ww) > DBL_EPSILON ? 1./ww : 0)
• Benchmark: 10 LM iterations over N inliers per frame is a real cost in a tracking loop — measure it, and make iteration count configurable (OpenCV exposes refineIters for exactly this reason)
• Decision recorded on the no_std question above

Out of scope

• Symmetric transfer error or Sampson distance. OpenCV uses forward-only; matching it is the goal here, not improving on it.
• RHO / USAC estimators.

Related

• motion_estimator.ransac: doc comment promises a refit over all inliers; jsfeatNext has no layer that performs one #185 (linear refit — prerequisite)
• f64 accumulators issue (prerequisite; LM starting from an f32-noisy seed wastes iterations)
• webarkit/webarkit#8

[kalwalt]

Fixed via jsfeatNext#193 (merged into dev): linalg.lm_solve (Levenberg-Marquardt over a damped normal system via cholesky_solve — additive damping decided for the no_std/WASM question, no SVD dependency), homography2d.refine()/affine2d.refine(), and motion_estimator.find_homography's optional refine_iters parameter (default 0, no behavior change unless a caller opts in). Fixed a real mask/model inconsistency bug found by Qodo review (mask wasn't recomputed after LM refine) before merge.