Vladimir A. Knyaz, Sergey Yu. Zheltov
State Research Institute for Aviation System (GosNIIAS)
Victorenko str., 7, Moscow, 125319, RUSSIA
tel: 7-095-157-9205, fax: 7-095-788-6894
E-Mail: [email protected]
Abstract
Virtual reality approach creates background for creating virtual museums, virtual sites of the past and virtual worlds of the past. To fulfill such a virtual environment with virtual historical items one has a need for an accurate and productive mean for generating adequate 3D models of historical objects. The paper presents photogrammetry system, which realized vision-based technique for photorealistic 3D reconstruction of historical items.
As usual historical items are objects of complex form, which can not be reconstructed basing on single stereo pair. So the proposed technique of 3D reconstruction is based on object image acquisition in stripe structural light while object rotating around vertical axis. A set of cameras is used for whole surface reconstruction. The 3D coordinate calculating is performed in suggestions that the plane of projected structural light and the rotation axis of turntable are known. The system developed includes three CCD cameras, Pentium personal computer equipped with frame grabbers, structural light projector and PC-controlled turntable. Metric characteristics of reconstructed 3D models are obtained by special system calibration technique providing estimation of all important system geometry parameters.
After system calibration the 3D-reconstruction process performed automatically including image acquisition in structural light at every angle position of turntable. For accurate texture generation a set of object images in shadow-free light is acquired. The photorealistic texture is generated as a set of orthophotos on appropriate 3D model fragment. The developed methods and algorithms are realized in original Windows 9X software.
Introduction
Virtual reality applications and Internet communication make it possible remote exploring of virtual 3D model instead of real objects. An appearance of virtual museums provides the possibility for remote tourism, sites seeing and also for remote study and exploration of historical artifacts. Also they demand for precise and automated system for generating 3D models of historical items like the system described in (Chikatsu, 1997).
The requirements to virtual models for purposes of study and exploration are higher that for virtual tourism 3D models because explorer can has a need for measuring some geometric characteristics of object and for some additional visual information. So such a 3D model has to provide metric geometry characteristics and photorealistic texture for adequate interpretation. The aim of presented work is to develop an automated mean for metric textured 3D reconstruction for such complex spatial objects as historical artifacts.
The method of 3D reconstruction is based on object image acquisition in stripe structural light while object rotating around vertical axis. Because of impossibility of viewing all surface points from single camera while object rotating, three cameras are used for image acquisition. This number is the minimum sensor quantity needed for whole surface reconstruction. The 3D coordinate calculating is performed in suggestions that the plane of projected structural light and the rotation axis of turntable are known. After system calibration this technique provides automated obtaining of all object 3D coordinates in single coordinate system and acquiring appropriate set of images for photorealistic texturing.
System Outline
Chosen 3D reconstruction approach determines the following system configuration (Fig. 1a.):
- 3 high resolution CCD video cameras
- Structured light projector
- PC-controlled turntable
- Shadow-free light source
- Pentium III/128MB PC
- 2 frame grabbers
- Original software for Windows 9X
 |
|
| a | b |
Figure 1. The scheme and exterior view of automated 3D reconstruction system
Three Panasonic WV-CP610 CCD video cameras output composite video signal of high resolution (about 450000 pixels) which is converted into three 768×576-pixel BMP images used for image processing and 3D measurements. Cameras are located so that left camera is used for reconstruction and texturing of right part of the object, right camera is used for reconstruction and texturing of left part of the object and the upper camera is used for reconstruction and texturing of the upper part of the object. Two frame grabbers provide simultaneous real-time image acquisition from three cameras. Also frame grabber supports real time image processing for stripe line recognition and its sub-pixel image coordinate measurement. PC-controlled turntable provides rotation of the object for the given angle. The minimum angle step is 0.2°, the accuracy of positioning is 1¢. Structured light projector produces object lighting in form of contrast line, the turntable rotation axis being approximately in the plane of light. To acquire shadow-free images for texture generation three lamps are located around the object. The switching between shadow-free and structured light is performed by program. The exterior view of 3D-reconstruction system is shown in the Fig. 1b.
The developed software provides the complete technology of 3D-model reconstruction. It supports:
- image acquiring using 2 frame grabbers configuration
- light control
- image processing of special kind for process automation
- system calibration and orientation
- 3D model reconstruction
- 3D model texturing
- 3D model visualization
System Calibration
The system uses non-metric CCD camera equipped with 12-mm lens as image acquisition device. To provide metric characteristics of reconstructed objects system is preliminary calibrated by special procedure based on test field. The plane containing 49 reference points (Fig. 2.) is used as test field for calibration procedure which includes the following steps:
- Cameras interior orientation
- Cameras exterior orientation
- Stripe structural light projector calibration
- Turntable axis of rotation estimation
The basic method for unknowns determination during calibration process is least mean squares (LMS) estimation. For automating most time consuming procedures as image coordinate measurements and choosing initial values for unknowns original coded targets and automate initial approximation determination algorithm are used (Knyaz, 1998). 
Figure 2. Test field for calibration
The coded targets used as reference points provide invariance to rotation and scaling, robust recognition of a target number with correction of possible mistakes, exact measurement of a target center, target detection in the image with non-uniform contrast, short processing time. Elliptic approximation is used for high accuracy sub-pixel center of target determination.
- Interior orientation
- Exterior orientation
- Light plane determination
- Turntable rotation axis determination Another suggestion used for 3D reconstruction is the following: contrast line found in the image is the intersection of light plane with object which rotates with given step around given axis. The estimation of rotation axis is based on image acquisition of plane test field in n several positions while rotating the turntable. Then spatial coordinates of reference points are calculating basing on known exterior orientation of cameras for each angle position ji, i = 1,…,n of test field. The vector of unknown parameters for least mean squares estimation includes rotation axis orientation (Xa, Ya, Za , aa, wa, ka) and angles of rotation ji, i = 1,…,n.
Original calibration procedure (Knyaz, 1999) for low precision test field is used. It uses the next form of additional parameters describing CCD camera model (Beyer,1992) in collinearity conditions:
and allows to determine the parameters of interior orientation (principal point xp, yp, scales in x and y directions mx, my, affinity factor a, the radial symmetric K1,K2,K3 distortion and decentering P1,P2 distortion) using reference distances between several reference points measured with high precision and condition of plane test field.
The exterior orientation parameters ((Xi, Yi, Zi) – location and (ai,wi,ki) angle position of the left, right and upper (i=1,2,3) camera) are determined in external coordinate system connected with turntable. The same plane test field (Fig. 2.) is used. To resolve the problem of unreliable exterior orientation based on plane test field three stereo pair of test field situated at the three various locations within workspace are acquired. At LMS estimating stage all reference points are treated as united spatial test field consisting of three planes.
For determination of spatial coordinates for object points viewed only by one camera the following suggestion is used: the stripe line on the object produced by structured light projector is the line of intersection of the some plane with reconstructed object. For accurate determination of S plane orientation in exterior coordinate system a set of stereo images of test field is acquired, test field being located at several positions in workspace. Then stripe is extracted in the images and its 3D coordinates are calculated. The set of spatial lines determines the plane of structured light in exterior coordinate system. The plane parameters (a,b,c,d) are estimated by least mean square estimation.
3D Reconstruction Technique
The samples of 3D models (American Indian vase and skull) produced by described system are shown in Fig. 3. The following technique is used for 3D reconstruction. To determine spatial coordinate of object point presenting in the image this point has to be identified as satisfied to some geometrical condition. For point identification structural light projector is used. Stripe projecting allows simple recognizing points of intersection light plane and the object. Then all lighted points viewed at least by one camera are included in 3D model. 
Figure 3. Results of 3D reconstruction for complex objects
For spatial coordinate determination two methods are used. For lighted object point presenting at least in one image the condition of intersection light plane S and object is used for 3D coordinate calculating. If lighted object point is viewed by two cameras then correspondence problem is solved and 3D coordinate of the point is calculated from collinearity condition.
For 3D-model reconstruction the procedures of image acquisition in structured light and image processing are performed. These procedures are executed in cycle while given angle position of turntable is reached. On each step stripe is detected with sub-pixel accuracy by weight operator and image coordinates of stripe are recalculated to undistorted values basing on interior orientation results. The results of stripe detecting for each table angle are stored in a file.
Spatial coordinates are calculated according two described methods resulting in “cloud” of 3D coordinates. To resolve the problem of reconstructing the surface for object of complex spatial topology the method of projection on spatial figure like sphere or cylinder is used (Knyaz, 2000). Thus technique allows obtaining single valued projection, so the topology of object can be reconstructed by Delaunay triangulation procedure for projected points. Then the links between projected points are used for generation spatial surface of the object.
Texture Generation
Because of complicated topology of most of historical artifacts (e.g. sculpture or skull) a set of images in shadow-free light is needed for generating photorealistic texture. The number of images is depends on object topology and is chosen parameter.
To generate metric texture for reconstructed 3D model several images are acquired from all three cameras at given positions of turntable. Then 3D model is divided into corresponding fragments basing on turntable position. For each fragment texture is generated as orthophoto.
 |
|
| a | b |
Figure 4. The image and textured 3D model of American Indian vase
Fig. 4. shows the object (American Indian vase) image (a) and the result of textured 3D model reconstruction (b). Seven images were used for texture: three images from left camera, two images from right camera and two images from upper camera (for bottom texturing).
Fig. 4. demonstrates good quality of texture mapping but also shows a need for shadow-free light improvement to eliminate brightness difference near fragments’ borders.
Discussions and Conclusions
Automated PC-based digital photogrammetric system for 3D reconstruction such complex spatial objects as historical artifacts is developed. It provides obtaining the whole object 3D model by one step using three CCD cameras and PC-controlled turntable. Automated correspondence problem solution is provided by image acquisition in stripe structural light.
High precision of 3D reconstruction and texture mapping for non-metric CCD cameras used for image acquisition are provided by preliminary calibration. Calibration includes cameras interior and exterior orientation and estimation of orientation turntable rotation axis and structured light plane. Calibration process has high degree of automation due to coded target applying.
After calibration stage process of 3D reconstruction and texture mapping is fully automated and can be performed by “one-button-click”. The time required for one 3D model generation in start-stop mode is about 4 minutes, the most time consuming part of process (about 75%) being table rotation. The accuracy of spatial coordinate determination is at the level of 0.15mm, the point density is 100 000 points per 3D model.
Photorealistic texture is generated basing on a set of images acquired in shadow-free lighting. The accuracy in texture continuity at the places of different images bordering is about 1/3 pixel. High quality of texture mapping provides additional wide capacities for expert work.
The results of system application for real object 3D reconstruction demonstrate that the developed system provides high accuracy of produced 3D models and reasonable performance. The quality of produced 3D models are adequate for wide variety of virtual reality applications. The higher productivity can be achieved in real-time mode, which requires applying of advanced frame grabber.
References
- Chikatsu H., 1997, Anaj T., Hatano K. Real-time Ortho Imaging and Surface Modeling for Archeological Artifacts. Optical 3D Measurement Techniques IV, Edited by A.Gruen, O.Kuhbler, Zurich, 29 September- 2 October, 1997, pp. 19-26
- Knyaz V.A., 1998, A. V. Sibiryakov. The Development of New Coded Targets for Automated point Identification and Non-contact 3D Surface Measurements, International Archives of Photogrammetry and Remote Sensing, Vol. XXXII, part 5, Hakodate, Japan, 1998, pp. 80-85.
- Knyaz V.A., 1999, S.Yu. Zheltov, D.G. Stepanyants. Method for accurate camera orientation for automobile photogrammetric system. Proceedings of International Workshop on Mobile Mapping Technology, Bangkok, Thailand, April 21-23,1999, pp.4-3-1 – 4-3-6
- Beyer H.A., 1992. Accurate Calibration of CCD-Cameras, Proceedings of the IEEE Computer Society Conferencion Computer Vision & Pattern Recognition 1992 (CVPR92), Champaign, Illinois, June 1992, pp. 96-101
- Knyaz V. A., 2000, S. Yu. Zheltov. Approach to Accurate Photorealistic Model Generation for Complex 3D Objects. International Archives of Photogrammetry and Remote Sensing, Vol. XXXIII, part B5/1, Amsterdam, The Netherlands, 2000, pp. 428-433
