3D Recognition Software

Here are links to the packages you will need to build the recognition software:

3D software -- This package and the "fl" library are provided under the UIUC/NCSA Open Source Licence.
fl library -- Support library.
LAPACK -- Needed if you don't already have LAPACK routines built on your system. Get lapack.tgz, which should appear first in the list of software.
BLAS -- Needed if you don't already have BLAS routines build on your system. BLAS supports LAPACK. If you use the Fortran reference implementation, you will not get the best performance. Some alternatives are:
- High-Performance BLAS by Kazushige Goto -- This is my current preferred BLAS for Pentium IV.
- ASCI Red Pentium II BLAS 1.2F -- Actually, I use version 1.3e because earlier versions have a bug. You have to register and then follow the Pentium III link in the page you receive after registering.
- MKL -- Intel's numerical library, which provides all of LAPACK and BLAS, but isn't free.
FFMPEG -- Needed only if you use the video programs.
libtiff and libjpeg -- These are provided with RedHat distributions, and almost certainly with other distributions as well. libtiff is only needed if you want to use TIFF for input. libjpeg is required because some programs in the 3D software output in JPEG.
X and GL -- Also typically included in Linux distributions.

The sofware is developed on a RedHat 9 system using gcc-3.3. Use at least gcc-3.2 to build the software, because it makes extensive use of STL, and gcc's implementation of STL has been changing recently, certainly between 2.x and 3.x.

First ensure that all the support software (everything besides 3d and fl) is built. Then configure the fl library. You do this by editing the file "Config" in the first directory level of the installation. It contains paths to each of the supporting libraries (such as LAPACK) and options for including them in the compilation and link processes. First run "make depend", and then finally build the fl library by running "make".

Finally, configure and build the 3D software. The configurations for the 3D software are in the same style as for the fl library, but they are included directly in the Makefile. Edit the makefile to point to the libraries, run "make depend", and then build whatever 3D programs you wish to use.

The programs within the 3D suite are independent, and you don't have to build them all to use one of them. However, there is a kind of pipeline of processing for achieving various ends, which requires the application of several of the programs in sequence. For example, to form a 3D model from a set of input pictures, you use the following sequence:

findpoint -- Locate interest points in the pictures. Output is a file that contains interest points for all the selected pictures.
adaptpoint -- Determine the affine-invariant patches around each interest point (if possible). Input is an interest point file from the previous stage. Output is a file of extended interest point information.
findmatch -- Find matches between pairs of images. Input is the extended interest point information, and a list of image pairs. Output is a separate match file for each image pair.
learnmodel -- Constructs a 3D model. Input is the extended interest point file and a set of match files. Output is a model file, which contains 3D information as well as images of each patch.

After forming a 3D model, you can view it using the program "render".

Below are brief summaries of the usage for each program in the suite. You may also follow these two links to view detailed information on the software internals:

findpoint

Applies one or more interest operators to a set of images. This is the first stage of processing in order to find affine-invariant patches in the images. The next stage is affine-adaptation, which the program "adaptpoint" handles. The output of this program is a file containing interest points for all of the images.

All the programs in the suite use the same convention for parameter passing, which is as follows:

You specify a parameter on the command line by writing parameter_name=value. That is, write the parameter name, followed by an equal sign, followed by the value, with no spaces.
The parameter names are case-insensitive.
The parameter names may be abbreviated as long as they are not ambiguous. For example, you can abbreviate "method" as "me" and "maxpoints" as "ma", but you cannot abbreviate either one of them as "m".
You can put all the parameters in a simple text file where each line consists of one parameter=value pair. Blank lines are ingnored, and you may use the "#" character to start a comment at any point on any line. To specify the parameter file on the command line, use the special purpose parameter "include". For example, if the parameter file is named "parms", the command line might read:
```
findpoint include=parms
```
An entry on the command line (or in a parameter file) which does not contain an "=" is considered a file name. Each program interprets file names in its own way.

Here are the parameters specific to this program:

Parameter Default Explanation
{image file names} Specify which image file to process. The paths to these images will be stored in the resulting point file, and will be relative to the current directory.
outFileName The resulting interest point file.
resumeFileName An interest point file from a previous, and presumably interrupted, run. The file can be incomplete as a result of the interruption. The program will use any good results from the file, and then complete the processing on the unfinished work.
method hl A comma separated list of codes specifying which interest point detectors to run. The detectors are:
hl -- HarrisLaplacian
h -- Harris
l -- Laplacian
s -- Hessian
d -- Difference-of-Gaussian

maxpoints 5000 The largest number of points that any given detector may contribute per image.
harrisFactor 0.03 Controls the threshold on Harris response required for a point to be accepted. The value specifies the number of standard deviations above zero to set the threshold. The statistic is calculated over all the Harris maxima in the image.
laplaceFactor 2 Controls the threshold on Laplacian response required for a point to be accepted. The value specifies the number of standard deviations away from the average response. The average and standard deviation are calculated over all the Laplacian maxima in the image.
nms 1 The radius of the neighborhood around a point within which it must be the largest value in order to be considered a local maximum.
firstScale 1 The smallest scale at which to run each of the detectors. You would normally only change this value if you know that all features in the images are at larger scales, and don't won't the list of interest points cluttered by irrelevant small scale features.
laplaceFirstScale firstScale Allows one to choose a different start scale for the Laplacian detector than for the other detectors.
lastScale 25 The largest scale at which to run each of the detectors. You would normally change this value if you know that some features in the images are at very large scales.
stepSize sqrt(2) Determines the spacing between scale levels. Specifies the amount by which to multiply the current scale level in order to find the next scale level.
extraSteps 20 Determines the granularity when scanning for characteristic scale of a given interest point. Specifies the number of subdivisions between each point detector scale level.
alphaColor 0 Specifies a color to paint over the background of an image, where foreground is indicated by the alpha channel. This can be used to remove distracting texture from an image. If this value is zero, the image is not modified.

Parameter	Default	Explanation
{image file names}		Specify which image file to process. The paths to these images will be stored in the resulting point file, and will be relative to the current directory.
outFileName		The resulting interest point file.
resumeFileName		An interest point file from a previous, and presumably interrupted, run. The file can be incomplete as a result of the interruption. The program will use any good results from the file, and then complete the processing on the unfinished work.
method	hl	A comma separated list of codes specifying which interest point detectors to run. The detectors are: hl -- HarrisLaplacian h -- Harris l -- Laplacian s -- Hessian d -- Difference-of-Gaussian
maxpoints	5000	The largest number of points that any given detector may contribute per image.
harrisFactor	0.03	Controls the threshold on Harris response required for a point to be accepted. The value specifies the number of standard deviations above zero to set the threshold. The statistic is calculated over all the Harris maxima in the image.
laplaceFactor	2	Controls the threshold on Laplacian response required for a point to be accepted. The value specifies the number of standard deviations away from the average response. The average and standard deviation are calculated over all the Laplacian maxima in the image.
nms	1	The radius of the neighborhood around a point within which it must be the largest value in order to be considered a local maximum.
firstScale	1	The smallest scale at which to run each of the detectors. You would normally only change this value if you know that all features in the images are at larger scales, and don't won't the list of interest points cluttered by irrelevant small scale features.
laplaceFirstScale	firstScale	Allows one to choose a different start scale for the Laplacian detector than for the other detectors.
lastScale	25	The largest scale at which to run each of the detectors. You would normally change this value if you know that some features in the images are at very large scales.
stepSize	sqrt(2)	Determines the spacing between scale levels. Specifies the amount by which to multiply the current scale level in order to find the next scale level.
extraSteps	20	Determines the granularity when scanning for characteristic scale of a given interest point. Specifies the number of subdivisions between each point detector scale level.
alphaColor	0	Specifies a color to paint over the background of an image, where foreground is indicated by the alpha channel. This can be used to remove distracting texture from an image. If this value is zero, the image is not modified.

adaptpoint

Performs the affine-adaptation process on the interest points detected by findpoint. Affine-adaptation transforms each interest point into a normalized form which is (more or less) invariant under local affine transformations of the texture pattern in the image. The output is another file containing the adapted interest points. The adapted points form the basis for most of the other processing in the still image portion of the suite.

Parameter	Default	Explanation
pointFileName		The raw interest point file output by findpoint.
outFileName		The resulting affine-invariant interest point file.
resumeFileName		An affine-invariant interest point file from a previous, and presumably interrupted, run. The file can be incomplete as a result of the interruption. The program will use any good results from the file, and then complete the processing on the unfinished work.
nms	-1	"Non-max suppression". During affine-adaptation, multiple interest points will converge to roughly the same location. All but one of those points must be eliminated. This parameter gives the radius around a given point within which to check for duplicate points. There are several options for non-max suppression: nms > 0 -- Use a fixed number of pixels and Euclidean distance. nms < 0 -- Use -nms * point.scale pixels and Mahalanobis distance. nms = 0 -- Don't do non-max suppression.
lockPosition	0	If set to 1, indicates that during the adaptation process the point's position should not change. This allows you to control which patch parameters are adapted.
lockScale	0	If set to 1, indicates the during the adaptation proces the point's scale should not change. If you set both lockScale and lockPosition, then only shape changes.
iterate	1	There are three parameters (with total dimension of 7) that describe each point. They are: location (2D), scale (1D), and shape (4D). Each parameter has an influence on the estimates of the other two. Therefore, the adaptation process iteratively estimates each parameter in turn using the current values of the other two parameters. By prohibiting the iteration, you can get a quick raw estimate of the parameters. The main objective would be to get an estimate of shape.
scaleCeiling	25	Limits the growth of the scale parameter during iteration. Sometimes affine-adaptation will (correctly) find increasingly larger scales for a given point. The cost for processing increases dramatically with scale. If you know that you only want to consider points under a certain size, then you can make the program run faster by setting this limit.
alphaColor	0	Specifies a color to paint over the background of an image, where foreground is indicated by the alpha channel. This can be used to remove distracting texture from an image. If this value is zero, the image is not modified.
showWindow	0	Turns on graphical monitoring of the adaptation process.
fast	0	Enables certain optimizations in the adaptation process which favor speed over repeatability.
fixAngle	0	If set to 1, then the program skips all processing except the final angle estimation.

showpoint

Allows you to view the results of interest point processing. You specify an interest point file and an image file, and it will render markings in the image for the patches surrounding each interest point. This program is mainly a diagnostic utility.

Parameter	Default	Explanation
{image file names}		Specify the image files for which you want to display interest points.
pointFileName		An interest point file output by either findpoint or adaptpoint.
supportRadial	1	Amount by which to multiply the characteristic scale of a given point in order to size the resulting mark in the image.
show	1	Indicates that the image should be diplayed in a window.
write	0	Indicates that the image should be written to disk.
monochrome	0	Indicates that the image should be rendered in grayscale. The interest points themselves will still be in color.
rectangle	0	Indicates that the interest points should be marked as parallelograms rather than ellipses.
center	0	Indicates that there should be a mark at the exact location of the interest point, in addition to a mark that indicates the shape and scale of the associated patch.
zoom	1	An amount by which to scale up the entire image.
lineWidth	1	How thick to stroke the line marking the boundary of a patch.
firstScale	-inf	The lower limit on the range of scales for which to display interest points. All other interest points are ignored.
lastScale	inf	The upper limit on the range of scale for which to display interest points. Used in conjunction with firstScale to focus on a specific subset of interest points.
alphaColor	0	Specifies a color to paint over the background of an image, where foreground is indicated by the alpha channel. If this value is zero, the image is not modified.

3D Recognition Software

findpoint

adaptpoint

showpoint

findmatch

learnmodel

render

recognize

video

play

compare