How reliable are manual landmarks?
To test the validity of the assumption
that manual landmarks are a suitable ground truth,
we re-manually-landmarked the left eye pupil and nose tip on 300
BioID faces.
(The re-marked points are points 0 and 14 in this
figure,
reproduced from Appendix A of the stasm Master's thesis.)
The position of the eye pupil can usually be estimated by hand reliably.
Estimating the position of the tip of the nose is much more subjective.
The mean accuracy of all manual landmarks across a face should be
somewhere between these two extremes.
The figure to the right compares these two landmarks to the manually landmarked
BioID FGNET
points.
(The FGNET points are the ones used as a reference in the stasm project.)
The distances are divided by the inter-eye distance
to make them commensurate with me17s.
The mean inter-eye distance is 61 pixels for these images.
Thus a me17 of 1/61 = 0.016 corresponds to one pixel.
The stasm stacked model is also shown, for calibration.
(This line is the "BioID test set" line on the graph
on the Stasm homepage.
It is almost the same as the "Trimmed Model" line in Figure 6.6 in
the stasm Master's thesis, but is for a slightly later version of stasm).
The figure indicates that mean me17s less than (very roughly) 0.02
should be taken with a pinch of salt.
A "perfect" automatically landmarked face measured against
a manually landmarked face could have an apparent me17 of 0.02
just because of manual landmarking variance.
Using just two landmarks on just the BioID set limits the scope of this experiment.
We would be interested in any results you may have that extend the results here.
May 2008
Back to Stasm homepage