Still Picture Modeling and Recognition

This page presents results from our object modeling and recognition work. To spare you the pain of waiting for massive downloads before viewing this page, all the images are simply linked, rather than being included directly.

Some of the images below are 3D renderings. These images render in a lighter color patches that would otherwise be occluded by the object.

This page presents one input image for each model, to give you an idea of the what the real-world object looks like. Also, the file size of the images is limited to enhance your browsing experience. If you would like the full set of input photos at full file size, you may find them on the data page.

Shoe

Modeling:

Recognition:

Test image 1
... detected patches
... correlation based matches before refinement -- shown at detected scale.
... correlation based matches before refinement -- shown with full-sized support region.
... correlation based matches after refinement
... recognized patches
... recovered pose

Here is a table of statistics from the recognition process for all the models against test image 1:

	Hypothesized matches	Recognized patches	Residual	Reprojection error	Camera ratio
Bear	6965	1	inf	inf	0
Vase	1805	0	inf	inf	0
Apple	999	0	inf	inf	0
Salt	890	0	inf	inf	0
Shoe	892	50	1.862	1.73776	0.95851
Spiderman	1380	0	inf	inf	0
Rubble	1563	0	inf	inf	0

Hypothesized matches: Matches between patches in the image and patches in the model based on normalized correlation.
Recognized patches: Patches that correctly match to the model, as determined by the matching constraints (generally robust estimation followed by neighborhood constraint).
Residual: Numerical error in pose estimation. Essentially the predicted reprojection error.
Reprojection error: Average distance in pixels between patch center measured in image and patch projected from model thru recovered pose.
Camera ratio: The ratio of the lengths of the two vectors that make up the projection matrix for the recovered pose. After applying the Euclidean upgrade associated with the cameras in the model, this ratio should be very close to 1 if the pose is consistent with those cameras (and the test image comes from a Euclidean Universe via a camera with square pixels :).

A residual or reprojection error of "inf" means that there were not enough matches to estimate the pose. Currently we require at least two good matches, although theoretically one match is enough if the model is Euclidean.

Un-recogntion: The following are poses for a handful of the models that did not match test image 1. We made these by skipping the neighborhood constraint step. Note that the number of hypothesized matches is larger because our process uses the recovered pose to search for more matches. All these result are, of course, bogus.

Hypothesized
matches Recognized
patches Residual Reprojection
error
Apple 2154 20 (image) 70.7798 87.0664 (image)
Salt 2120 37 (image) 206.021 305.88 (image)
Vase 3340 29 (image) 189.827 292.587 (image)

	Hypothesized matches	Recognized patches	Residual	Reprojection error
Apple	2154	20 (image)	70.7798	87.0664 (image)
Salt	2120	37 (image)	206.021	305.88 (image)
Vase	3340	29 (image)	189.827	292.587 (image)

We can test for the presence of an object using thresholds on the number of recognized patches, on the residual, and on the "squareness" of the pixels in the recovered pose. Below is a test of a set models against a set of images. The table gives the number of recognized patches and, where possible, the aspect ratio of the pixels in the recovered pose. In paranthesis it shows the type of interest point detector used to construct the model or to process the image: (L) is the Laplacian detector and (H) is the Harris detector.

Bear & Vase (H) Apple & Salt (H) Shoe (L) Spiderman (L)
Bear (H) 91 -- 0.976335 1 1 318 -- 0.679124
Vase (H) 45 -- 0.996062 0 0 1
Apple (H) 0 13 -- 0.997009 0 0
Salt (L) 0 54 -- 0.949849 0 0
Shoe (L) 0 0 50 -- 0.95851 0
Spiderman (L) 4 -- 0.330492 0 0 28 -- 0.99138
Rubble (L) 2 -- 0.994382 0 0 18 -- 0.982189

	Bear & Vase (H)	Apple & Salt (H)	Shoe (L)	Spiderman (L)
Bear (H)	91 -- 0.976335	1	1	318 -- 0.679124
Vase (H)	45 -- 0.996062	0	0	1
Apple (H)	0	13 -- 0.997009	0	0
Salt (L)	0	54 -- 0.949849	0	0
Shoe (L)	0	0	50 -- 0.95851	0
Spiderman (L)	4 -- 0.330492	0	0	28 -- 0.99138
Rubble (L)	2 -- 0.994382	0	0	18 -- 0.982189

Teddy Bear

The following teddy bear results were not all produced at the same time, so some images may be of a trivially different model than the others.

Modeling:

Recognition:

Test image 1 in significantly different pose from imput images
... recognized patches
... recovered pose
Test image 2 -- Bear and vase
... recognized patches
... recovered pose

Apple (Fuji)

Modeling:

Recognition:

Spiderman

Modeling:

Recognition:

Rubble (Spiderman base)

Modeling:

Recognition:

Salt Can

Modeling:

Recognition:

Vase

Modeling:

Recognition: