Robust Local Optimization Done Right

James Pritts  ·  Kevin Köser

Kiel University, Kiel, Germany

ACCV 2026

Abstract

RANSAC scoring and local optimization (LO) impose different robustness requirements, motivating the separation of hypothesis selection from refinement. We systematically isolate the effects of robust-loss shape, scale miscalibration, and optimization strategy on the accuracy of essential matrix, fundamental matrix, and homography estimation. A profile-marginal score marginalizes the nuisance inlier scale and selects an inlier partition, from which we estimate the scale that sets the LO loss width. The estimated scale reduces sensitivity to miscalibration. Methods with broad basins of attraction can recover from strongly perturbed initial estimates yet degrade accurate score-selected hypotheses, showing that basin size alone is insufficient to assess RANSAC LO. We also derive an optimizer matched to the profile-marginal score that guarantees monotonic improvement of the scoring objective. Our experiments show that matching the scoring and refinement objectives is not necessary to achieve the best geometric accuracy, which challenges a common prescription. As a proof of concept, we use our findings to compose a RANSAC that achieves state-of-the-art essential matrix estimation accuracy on the PhotoTourism dataset.

BibTeX

@inproceedings{Pritts2026LocalOptimization,
  title     = {Robust Local Optimization Done Right},
  author    = {Pritts, James and K{\"o}ser, Kevin},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  year      = {2026}
}