3D Photography Dataset

Yasutaka Furukawa and Jean Ponce

Beckman Institute and Department of Computer Science
University of Illinois at Urbana-Champaign

The following are multiview stereo data sets captured in our lab: a set of images, camera parameters and extracted apparent contours of a single rigid object. Each data set consists of 24 images. Image resolutions range from 1400x1300 pixels^2 to 2000x1800 pixels^2 depending on the data set. For calibration, we have used "Camera Calibration Toolbox for Matlab" by Jean-Yves Bouguet to estimate both the intrinsic and the extrinsic camera parameters. All the images have been corrected to remove radial and tangential distortions. For contour extraction, first, Photoshop has been used to segment the foreground from each image(pixel-level). Second, segmentation results have been used to initialize the apparent contour(s) of an object. Last, a b-spline snake has been applied to extract apparent contours in a sub-pixel level.

Images are provided in the JPEG format. Camera parameters are provided in the same format as that of "Camera Calibration Toolbox for Matlab". Apparent contours are provided in our own format, but we hope are fairly easy to interpret. Please see the other sections at the bottom of this website for more details. Unfortunately, we do not have ground truth for all the data sets, but we believe that we can develop some photometric ways to evaluate the reconstruction results. This page is still under construction, and we keep on updating the contents as well as adding more data sets as we capture.

Note: we also provide visual hull data sets.

Data sets

Toy Dinosaur
24 images
Approx 2000x1500 pixels^2
images
camera_params
contours

Toy Mummy
24 images
Approx 1600x1100 pixels^2
images
camera_params
contours

Action Figure (Morpheus)
24 images
Approx 1400x1200 pixels^2
images
camera_params
contours

Action Figure (Predator)
24 images
Approx 1800x1800 pixels^2
images
camera_params
contours

Action Figure (Warrior)
24 images
Approx 1500x1500 pixels^2
images
camera_params
contours

Action Figure (Soldier)
24 images
Approx 1300x1400 pixels^2
images
camera_params
contours

Human Skull Cast
Homo Heidelbergensis - Pose 1
24 images
Approx 2000x1800 pixels^2
images
camera_params
contours

Courtesy of
Jodi Blumenfeld (UIUC) and
Steve Leigh (UIUC)

Human Skull Cast
Homo Heidelbergensis - Pose 2
24 images
Approx 2000x1800 pixels^2
images
camera_params
contours

Courtesy of
Jodi Blumenfeld (UIUC) and
Steve Leigh (UIUC)

Human Skull Cast - Pose 1
24 images
Approx 1500x1500 pixels^2
images
camera_params
contours

Courtesy of
Jodi Blumenfeld (UIUC) and
Steve Leigh (UIUC)

Human Skull Cast - Pose 2
24 images
Approx 1700x1600 pixels^2
images
camera_params
contours

Courtesy of
Jodi Blumenfeld (UIUC) and
Steve Leigh (UIUC)



Experimental Setups

We captured the above data sets in our lab by using 3 fixed cameras (Canon EOS 1D Mark II) and a motorized turn table (please see a picture below). We have followed the following 3 steps to acquire data.

1. Put 3 cameras in different heights (or 1 at the top and 2 almost at the same height).
2. Put an object on the motorized turn table, and take pictures while rotating the table. The table rotates approximately 45 degrees every time. We rotate it 8 times and each camera takes 8 images. Then at total, we obtain 8 x 3 = 24 images.
3. Replace an object with a calibration board (checker board pattern), and take pictures of the board while rotating the table 8 times in exactly the same way.

Note that although we have tried to make the incoming light diffuse and uniform as much as possible, lighting conditions with respect to an object are different in each image.




File Formats

As is described in the last section, each data set has been acquired by 3 cameras, and each camera has taken 8 pictures. Image files from 00.jpg through 07.jpg have been taken by the first camera, and hence, these 8 pictures share the same intrinsic camera parameters. Similarly, image files from 08.jpg through 15.jpg have been taken by the second camera, and Image files from 16.jpg through 23.jpg have been taken by the third camera. There is a camera parameter file for each camera: camera0.m (resp. camera1.m and camera2.m) for the first (resp. the second and the third) camera. Each file stores the intrinsic camera parameters of the corresponding camera and extrinsic camera parameters of the associated 8 images. For example, camera0.m contains intrinsic camera parameters for the first camera and extrinsic camera parameters of images 00.jpg through 07.jpg. The format of the camera parameter file is the same as that of "Camera Calibration Toolbox for Matlab". Please see their webpage for more detailed information. For convienience, we have computed the projection matrix from camera parameters and added the matrix to the file. Also note that marginal backgrounds have been clipped away to reduce the sizes of input images while keeping an object fully visible in each image. The principal point has been modified accordingly.

An apparent contour is represented as a chain of points in our data sets (a piece-wise linear structures) and provided in a simple format. Each file starts with a single line header and an integer representing the number of apparent contours in the corresponding image, which are followed by the data sections of apparent contours. Note that a single image can have multiple apparent contours. A data section of an apparent contour starts with an integer representing the number of points in the component, followed by their actually 2D image coordinates. Points are listed in a counter clock-wise order for apparent contours containing foreground image region inside. Similarly, points are listed in a clock-wise order for apparent contours containing background image region inside (holes). Since all the objects in our data set are single connected components, each image has a single apparent contour of the first kind, which is given as the first apparent contour in all the files, but can have multiple holes. In some images, especially those of very complicated objects, there exists too many holes in a single image, and we could not extract all of them. However, we believe that this is not a critical issue, because a good visual hull can be still constructed, and our reconstruction algorithm did not have problems in using such visual hulls. If you build a visual hull and are not satisfied with the outcome, you can extract the silhouettes by yourself. The other thing you may want to try is to build a visual hull by using the apparent contours we provided, then project the visual hull back onto each image, and extract boundaries of its projection. Missing holes can be detected from the boundaries, furthermore, you may be able to extract more accurate silhouettes starting from the boundaries. In those cases, we really appreciate it if you can provide us more accurate data files.


Calibration Accuracy

We have performed the following 3 tests to check the accuracy of the camera parameters.

- Firstly, we need to make sure that the behavior of the turn table is repetitive, because pictures of an object and pictures of the calibration grids have not been taken at the same times. Note that we don't care if the rotation angle is exactly 45 degrees or not, but we want the rotation angle to be the same every time. We confirmed that the rotation is repetitive as follows.
- Put a paper with some textures on the turn table, and take a picture from a fixed camera.
- Rotate the table (approximately) -45 degrees.
- Rotate the table (approximately) 45 degrees.
- Take a picture.
- Rotate the table (approximately) -45 x 2 = 90 degrees.
- Rotate the table (approximately) 45 x 2 = 90 degrees.
- Take a picture.
- and etc.
Those images look identical.

- Secondly, we take pairs of (radially and tangentially undistorted) images, and draw a bunch of epipolar lines in them. We can check an accuracy of camera parameters by using frontier points, or salient image features where the epipolar lines go through. We put pairs of images with epipolar lines here for one of our objects. Please compare 2 images, which file names are ??_??_0.jpg and ??_??__1.jpg. We believe that epipolar lines go through the same image features or are off by a pixel or two at most.

- Lastly, we can use the reconstruction of our algorithm for the check, that is, look at the alpha-blended surface textures backprojected from different images. Backprojected textures are consistent with each other only when the camera parameters and the geometry of an object are correct. We observed that backprojected textures are consistent even for surface structures that are a few pixels long. The following 2 pairs of images are examples of such alpha-blended textures. For each pair, left picture shows the consistent alpha-blended textures, while the right one is inconsistent.

Consistent
alpha-blended textures
Inconsistent
alpha-blended textures
Consistent
alpha-blended textures
Inconsistent
alpha-blended textures



Our Results

We include below sample results of our 3D photography algorithms both on our own datasets and datasets from another sources. These are described in articles that are (or will shortly be) under submission. The confidentiality policy of the corresponding conferences prevents us from posting these articles until they are accepted for publication. In the mean time, please contact us directly for more details.

results/dinosaur-0-blue.jpg
dinosaur-0-blue.jpg
212.30 KB
results/dinosaur-0-tex.jpg
dinosaur-0-tex.jpg
251.92 KB
results/dinosaur-blue1.jpg
dinosaur-blue1.jpg
257.47 KB
results/dinosaur-close-blue.jpg
dinosaur-close-blue.jpg
211.67 KB
results/dinosaur-close-tex.jpg
dinosaur-close-tex.jpg
279.10 KB
results/mummy-0blue0.jpg
mummy-0blue0.jpg
241.87 KB
results/mummy-0-tex.jpg
mummy-0-tex.jpg
219.28 KB
results/mummy-1blue1.jpg
mummy-1blue1.jpg
255.47 KB
results/mummy-1-tex.jpg
mummy-1-tex.jpg
223.94 KB
results/mummy-2.jpg
mummy-2.jpg
414.27 KB
results/mummy-closeup-0blue0.jpg
mummy-closeup-0blue0.jpg
308.69 KB
results/mummy-closeup-0tex0.jpg
mummy-closeup-0tex0.jpg
375.75 KB
results/mummy-closeup-1blue1.jpg
mummy-closeup-1blue1.jpg
298.09 KB
results/mummy-closeup-1tex1.jpg
mummy-closeup-1tex1.jpg
352.28 KB
results/mummy-closeup-2blue2.jpg
mummy-closeup-2blue2.jpg
384.74 KB
results/mummy-closeup-2tex2.jpg
mummy-closeup-2tex2.jpg
397.50 KB
results/nskulla-140.jpg
nskulla-140.jpg
196.31 KB
results/nskulla-b.jpg
nskulla-b.jpg
256.31 KB
results/nskulla-t.jpg
nskulla-t.jpg
388.22 KB
results/nskullb0.jpg
nskullb0.jpg
234.95 KB
results/nskullb1.jpg
nskullb1.jpg
312.50 KB
results/nskullb-blue0.jpg
nskullb-blue0.jpg
235.59 KB
results/nskullb-blue2.jpg
nskullb-blue2.jpg
332.23 KB
results/predator-blue0.jpg
predator-blue0.jpg
205.41 KB
results/predator-blue1.jpg
predator-blue1.jpg
178.79 KB
results/predator-close0-b.jpg
predator-close0-b.jpg
177.09 KB
results/predator-close0-t.jpg
predator-close0-t.jpg
341.65 KB
results/predator-tex0.jpg
predator-tex0.jpg
202.34 KB
results/predator-tex1.jpg
predator-tex1.jpg
191.84 KB
results/skull.jpg
skull.jpg
244.53 KB
results/skull_t.jpg
skull_t.jpg
4.12 KB
results/skull-0-blue.jpg
skull-0-blue.jpg
290.72 KB
results/skull-0-tex.jpg
skull-0-tex.jpg
345.61 KB
results/skull-1-blue.jpg
skull-1-blue.jpg
311.49 KB
results/skull-1-tex.jpg
skull-1-tex.jpg
369.09 KB
results/skull-closeup-blue.jpg
skull-closeup-blue.jpg
266.62 KB
results/skull-closeup-tex.jpg
skull-closeup-tex.jpg
382.57 KB
results/skull-wb.jpg
skull-wb.jpg
376.38 KB
results/steve-0-blue.jpg
steve-0-blue.jpg
160.70 KB
results/steve-1-blue.jpg
steve-1-blue.jpg
191.61 KB
results/steve-closeup-blue.jpg
steve-closeup-blue.jpg
273.51 KB
results/steve-closeup-tex.jpg
steve-closeup-tex.jpg
379.86 KB
results/warrior0.jpg
warrior0.jpg
171.51 KB
results/warrior1.jpg
warrior1.jpg
148.06 KB

Acknowledgement

We thank Jodi Blumenfeld and Steve Leigh, Department of Anthropology at University of Illinois at Urbana-Champaign for providing us the skulls. We also thank Steve Sullivan and ILM for providing a dataset for this project. This work was supported in part by the Beckman Institute and the National Science Foundation under ITR grant IIS-0312438.

Send any comments to Y. Furukawa (yfurukaw -at- uiuc.edu) Last updated 2 May 2006 - je-pu-pu 無料カウンタ