Scale-invariant feature operator

In the fields of computer vision and image analysis, the scale-invariant feature operator (or SFOP) is an algorithm to detect local features in images. The algorithm was published by Förstner et al. in 2009.^[1]

The scale-invariant feature operator (SFOP) is based on two theoretical concepts:

• spiral model^[2]
• feature operator^[3]

Desired properties of keypoint detectors:

• Invariance and repeatability for object recognition
• Accuracy to support camera calibration
• Interpretability: Especially corners and circles, should be part of the detected keypoints (see figure).
• As few control parameters as possible with clear semantics
• Complementarity to known detectors

scale-invariant corner/circle detector.

Maximize the weight ${\displaystyle w}${\displaystyle w}= 1/variance of a point ${\displaystyle p}${\displaystyle p}

${\displaystyle w(\mathbf {p} ,\alpha ,\tau ,\sigma )=\left(N(\sigma )-2\right){\frac {\lambda _{min}(M(\mathbf {p} ,\alpha ,\tau ,\sigma ))}{\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )}}}${\displaystyle w(\mathbf {p} ,\alpha ,\tau ,\sigma )=\left(N(\sigma )-2\right){\frac {\lambda _{min}(M(\mathbf {p} ,\alpha ,\tau ,\sigma ))}{\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )}}} 

comprising:

1. the image model^[2]

• Distance d of an edge from a reference point p in a spiral feature

${\displaystyle {\begin{aligned}\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )&=\sum _{n=1}^{N(\sigma )}[(\mathbf {q} _{n}-\mathbf {p} )^{T}\mathbf {R} _{\alpha }\mathbf {\nabla } _{T}g(\mathbf {q} _{n})]^{2}G_{\sigma }(\mathbf {q} _{n}-\mathbf {p} )\\&=N(\sigma )\mathbf {tr} \left\{R_{\alpha }\mathbf {\nabla } _{\tau }\mathbf {\nabla } _{\tau }^{T}R_{\alpha }^{T}*\mathbf {p} \mathbf {p} ^{T}G_{\sigma }(\mathbf {p} )\right\}\end{aligned}}}${\displaystyle {\begin{aligned}\Omega (\mathbf {p} ,\alpha ,\tau ,\sigma )&=\sum _{n=1}^{N(\sigma )}[(\mathbf {q} _{n}-\mathbf {p} )^{T}\mathbf {R} _{\alpha }\mathbf {\nabla } _{T}g(\mathbf {q} _{n})]^{2}G_{\sigma }(\mathbf {q} _{n}-\mathbf {p} )\\&=N(\sigma )\mathbf {tr} \left\{R_{\alpha }\mathbf {\nabla } _{\tau }\mathbf {\nabla } _{\tau }^{T}R_{\alpha }^{T}*\mathbf {p} \mathbf {p} ^{T}G_{\sigma }(\mathbf {p} )\right\}\end{aligned}}}

2. the smaller eigenvalue of the structure tensor ${\displaystyle \underbrace {M(\mathbf {p} ,\alpha ,\tau ,\sigma )} _{\text{structure tensor}}=\underbrace {G_{\sigma }(\mathbf {p} )} _{\text{weighted summation}}*\underbrace {(R_{\sigma }\nabla _{\tau }\nabla _{\tau }^{T}R_{\sigma }^{T})} _{\text{squared rotated gradients}}}${\displaystyle \underbrace {M(\mathbf {p} ,\alpha ,\tau ,\sigma )} _{\text{structure tensor}}=\underbrace {G_{\sigma }(\mathbf {p} )} _{\text{weighted summation}}*\underbrace {(R_{\sigma }\nabla _{\tau }\nabla _{\tau }^{T}R_{\sigma }^{T})} _{\text{squared rotated gradients}}}

Reduce the 5-dimensional search space by

• linking the differentiation scale ${\displaystyle \tau }${\displaystyle \tau } to the integration scale

${\displaystyle \tau =\sigma /3}${\displaystyle \tau =\sigma /3}

• solving for the optimal ${\displaystyle {\hat {\alpha }}}${\displaystyle {\hat {\alpha }}} using the model

${\displaystyle \Omega (\alpha )=a-b\cos(2\alpha -2\alpha _{0})}${\displaystyle \Omega (\alpha )=a-b\cos(2\alpha -2\alpha _{0})}

• and determining the parameters from three angles, e. g.

${\displaystyle \Omega (0^{\circ }),\Omega (60^{\circ }),\Omega (120^{\circ })\quad \rightarrow \quad a,b,\alpha _{0}\quad \rightarrow \quad {\hat {\alpha }}}${\displaystyle \Omega (0^{\circ }),\Omega (60^{\circ }),\Omega (120^{\circ })\quad \rightarrow \quad a,b,\alpha _{0}\quad \rightarrow \quad {\hat {\alpha }}}

• pre-selection possible:

${\displaystyle \alpha =0^{\circ },円\rightarrow ,円{\mbox{junctions}},\quad \alpha =90^{\circ },円\rightarrow ,円{\mbox{circular features}}}${\displaystyle \alpha =0^{\circ },円\rightarrow ,円{\mbox{junctions}},\quad \alpha =90^{\circ },円\rightarrow ,円{\mbox{circular features}}}

• non-maxima suppression over scale, space and angle

• thresholding the isotropy ${\displaystyle \lambda _{2(M)}}${\displaystyle \lambda _{2(M)}}:
  eigenvalues characterize the shape of the keypoint, smallest eigenvalue has to be larger than threshold ${\displaystyle T_{\lambda }}${\displaystyle T_{\lambda }}
  derived from noise variance ${\displaystyle V(n)}${\displaystyle V(n)} and significance level ${\displaystyle S}${\displaystyle S}:

${\displaystyle T_{\lambda }(V(n),\tau ,\sigma ,S)={\frac {N(\sigma )}{16\pi \tau ^{4}}}V(n)\chi _{2,S}^{2}}${\displaystyle T_{\lambda }(V(n),\tau ,\sigma ,S)={\frac {N(\sigma )}{16\pi \tau ^{4}}}V(n)\chi _{2,S}^{2}}

• Results of different detectors on a Siemens star
• Sfop: junctions red, circular features cyan
  Sfop: junctions red, circular features cyan
• Edge-based Regions
  Edge-based Regions
• Intensity-based Regions
  Intensity-based Regions
• MSER
  MSER
• Harris affine
  Harris affine
• Hessian affine
  Hessian affine
• Lowe
  Lowe

• Corner detection
• Feature detection (computer vision)

• Project website at University of Bonn