Fast Orthophoto Production using the Digital Sensor System

A. W. L. Ip
[email protected],

N. El-Sheimy
[email protected]

Mobile Multi-Sensor Research Group
Department of Geomatics Engineering, The University of Calgary,
2500 University Drive NW, Calgary, Alberta, Canada T2N 1N4

M. M. R. Mostafa
Applanix Corporation,
Richmond Hill,
Ontario, Canada L4B 3B3
[email protected]

1. Introduction
Digital camera technology started to attract photogrammetric researchers and system developers in the mid 1990s. Several efforts have been exerted to deploy the CCD-based digital camera technology into the airborne mapping environment. Some of these efforts focused on using the digital camera in a stand-alone mode (c.f., King et al, 1994; Mills et al, 1996) and confirmed the use of such cameras in the airborne environment.

More efforts focused on using the digital camera as a component of an integrated system (c.f., Mostafa et al, 1997; Cramer et al, 1997; Toth and Grejner-Brzezinska, 1998) where the System integration concept was initially proposed by Schwarz et al (1993) and successfully implemented for the land-based mobile mapping by El-Sheimy (1996). Other research efforts to develop airborne integrated systems were immediately directed towards the mapping industry and no sufficient publications are available albeit the validity of such research and its associated resulting mapping products (c.f., Congalton et al, 1998). Independently, the research outcome highlighted a number of conclusions:

1. The integrated airborne system approach (typically: digital camera/GPS/INS) far outperforms the stand-alone approach because of many advantages (c.f., Mostafa et al, 1997; Cramer et al, 1997; Toth and Grejner-Brzezinska, 1998)
2. A number of parameters have to be taken into account for the integrated system approach to work successfully: such as the calibration of system components, and the calibration of the integrated system as a whole. New software tools were recommended to be developed for such a purpose.
3. The existing photogrammetric algorithms and techniques are good enough to process the data delivered by these systems. However, seamless data flow between different commercial softwares was much needed back in the late 1990s and has been made possible in the last few years (c.f., Madani and Mostafa, 2001).

In this paper, The Digital Sensor System (DSS) is introduced as a dedicated product that was designed for the airborne mapping industry based on the aforementioned scientific research findings and years of experience of using such systems.

Manufactured by Applanix Corporation, the DSS is a fully operational, fully integrated all digital multi-sensor system developed for digital mapping data acquisition and processing. The DSS consists of a 4K x 4K digital camera, POS AV direct georeferencing system and a flight management system (FMS).

Unlike consumer type small format digital cameras, the DSS system has come through careful mechanical design, ruggedization, calibration, and testing. The DSS captures high resolution digital imagery together with their associated direct georeferencing data for various GIS applications such as stereo visualization, feature extraction and classification, orthophoto generation and digital elevation model (DEM) extraction.

Table 1: Specification of The DSS

The DSS has several features that make it an ideal tool for many mapping and GIS applications:

• It is light-weight and easily deployable in small aircraft and pilot only operations
• It has the flexibility to inexpensively collect colour and CIR from one platform
• It takes full advantage of direct georeferencing through the built-in POS AV system and processing tools
• It is fully warranted and calibrated as a complete mapping system: camera, direct georeferencing system and FMS
• It has a low-cost infrastructure: no special computer hardware is required, all post-processing can be done on a lap-top
• It works seamlessly with many of the existing digital mapping packages such as Image Station by Z/I, ERDAS Imagine by Leica, and Socet Set by BAE

In the following sections, an overview of The DSS design, calibration and performance is presented and the system performance in DEM extraction and orthophoto generation is discussed in some details using a real mapping mission flown over southern Ontario, Canada in September, 2003.

2. The DSS Sysyem Design
Although the DSS has been designed to accommodate off-the-shelf components, its design parameters included the aerial survey requirements. Therefore, different components of the DSS have been re-designed and custom machined, such as:

• The digital back is rigidly attached to the camera
• Camera lenses are optimized for focus at infinity using a special locking mechanism
• The camera digital back is modified for both color and CIR imaging
• The camera is mounted in a proprietary exoskeleton designed to keep the lens, the camera body and the IMU rigidly attached to each other
• The CCD chip is calibrated for minimum flaws
• The system has been ruggedized to survive the shock and vibrations exposed to in the airborne mapping environment
• The camera is radiometrically and geometrically calibrated, the IMU is calibrated, and the entire system is well calibrated prior delivery to the user.

The DSS comprises off-the-shelf components. The main advantage of this approach is that there is a clear upgrade path when new CCD and camera technology is introduced into the commercial market place. The DSS specifications are shown in Table 1.

Figure 1: The DSS
3. The DSS System Calibration
Before the DSS can be used for aerial mapping, it must be calibrated geometrically to relate the coordinate frames of the imager, the IMU, and the GPS antenna. Since no agency has been established for the digital camera calibration, this is done in house right after the DSS manufacturing at Applanix through a terrestrial calibration process. Using a calibration cage that is imaged from several angles, the interior orientation parameters (focal length, lens distortions, principle point positions), and the IMU/camera boresight angles are precisely computed.

Following the terrestrial calibration, an airborne calibration is performed for the first time the system is installed onboard an aircraft. A boresight test area is flown and the POSCAL TM software is used to refine the calibration parameters. This completes the DSS calibration process and the results of the airborne calibration are used for successive mapping missions. For details, see Mostafa, 2003.

4. The DSS Applications
One of the growing demands of the GIS and remote sensing industry is for high-resolution, geometrically accurate orthorectified colour and CIR imagery, that is available upon request. While higher resolution satellite imagery is now available (0.6 m), it often takes weeks to collect the area of interest, and even then the final geometric accuracy is only a few meters, at best. (Ref?) Hence aerial imagery is really the only way of meeting the high resolution, accuracy and quick delivery times being demanded.

In order to generate the orthorectified colour and CIR imagery, consisting of either single orthophotos or orthophoto mosaics, the exterior orientation (EO) of each image is required, as well as a Digital Elevation Model (DEM) These needs are fulfilled by the DSS with its high resolution imagery and direct georeferencing system, without the need for GCP and aerial triangulation Using the EO from the POS AV, and stereopair imagery from the DSS, a DEM can be automatically or semi-automatically extracted using high density elevation point determination from photogrammetry softcopies. Since the extracted DEM contains all features in the project area, it is up-to- date and there is no need to worry about terrain changes from existing DEM due to redevelopments in the area of interest. The stereo images can also be used for visualization and feature extraction.

The all-digital workflow and built-in direct georeferencing system of the DSS also provides several new possibilities for “rapid response” applications. For example, with an existing DEM it could be possible to generate lower accuracy orthophotos on the fly during the airborne data acquisition process in near-real time (i.e., before landing). Examples of rapid response applications are forest fire mapping, oil spills, hurricanes, terrorist attacks etc. Even more interesting however, is that full-accuracy orthophoto mosaics can be generated within hours or days (depending upon project size) after landing using differential GPS and the DEM extraction.

Hence with its quick processing times, ground resolutions in the range of 0.05 to 1 m, and geometric accuracy at the resolution level, the DSS is ideal for meeting the current market demands for orthorectified colour and CIR imagery.

Given these capabilities and its low cost, the DSS opens the door to many applications that often involve small localized areas, corridors or irregular spot shots:

• updating and maintaining cadastral GIS databases,
• classifying and mapping pervious and impervious surface areas,
• identifying wetland areas,
• updating land use maps,
• estimating crop yields and health,
• preparing timber stand inventories,
• planning for new construction sites,
• verifying areas for licensing and permitting
• pipeline management
• utilities infrastructure management
• oblique photography for land management
• disaster management

5. The DSS performance analysis
This section is dedicated to highlight the geometric accuracy of the DSS using a test flight over Southern Ontario, Canada. The objective of this flight is testing and validating the terrestrial calibration of the DSS and in-flight approaches and analysing its capability for DEM extraction and orthophoto generation. A subsection of this flight test was used for the analysis presented here. This section was flown over Ajax, Ontario, and consists of a total of 72 images taken from a flight altitude of about 1200 m Above Ground Level (AGL). This test area contains a total of 50 Ground Control Points (GCPs) surveyed using DGPS pseudokinematic approach. All ground control points have been used as check points, except for one point that was used for calibration and quality control purposes. The Southern Ontario flight trajectory is shown in Figure 2, while a summary of the test flight parameters is presented in Table 3. Note that only the Ajax part of this flight data is used for the analysis presented here.

Table 3: Summary of The Ajax Flight parameters

Figure 2: Southern Ontario Flight Trajectory
The POS AV data was first processed using POSPac TM , and the exterior orientation parameters were computed using the terrestrially calibrated boresight angles. Then, the imagery, the POS-derived exterior orientation parameters, and the GCPs were introduced to ZI ImageStation (ISAT) for tie point collection. In this part, the image block was sub-divided into three areas, namely

1. Commercial area (C)
2. Residential are (R)
3. Forested area (F)

Dividing the test allowed evaluating the performance of the automatic tie point generation process for different terrain type towards the in-flight calibration procedure in POSCal TM .

Additionally, it allows analysing the effect of different terrains on map production using the DSS. Using only 1 GCP for the in-flight calibration, tie point accuracy and POS-derived exterior orientation parameters are very important, especially in the forested area, tie point quality can be degraded significantly due to the lack of unique features and the resulting calibration parameters can therefore be less accurate. A summary of the configuration of each area is listed in Table 4.

Table 4: Terrain Information

For each terrain type, the ISAT-automatically generated tie points which were imported to POSCal TM along with POS-derived exterior orientation. The boresight and camera parameters were re-calibrated using only 1 GCP (around the centre of each area) to refine the parameters that were calibrated using the terrestrial calibration. The use of one GCP in the airborne calibration is needed to absorb any datum shifts. After the in-flight calibration, the data was imported again to ISAT where the EO Analysis function (Madani and Mostafa, 2001) was used to double check the validity of the POSCal TM calibrated parameters and to analyse the performance of the DSS. In the EO analysis tool of ISAT, the ground coordinates of checkpoints are computed for each stereopair using the EO parameters of the two images and the image coordinates of the checkpoints. Then the computed checkpoint coordinates are then compared with the land-surveyed coordinates. The differences between the DSS-computed and the land-surveyed coordinates of the checkpoints are then summarized by ISAT together with their statistics. The results of the EO Analysis on the three terrain areas are briefly presented in Table 5.

Table 5: Statistics of Checkpoint Residuals Extracted from ISAT EO Analysis

Examining the results presented in Table 5, the following observations can be made:

• The RMS of checkpoint residuals is always sub-metre in all the terrain areas. The statistical results presented here are more or less equivalent to the independent test results of the DSS in Japan and in Florida presented by Mostafa (2003).
• In the Forest area, the check point residuals in the vertical component are twice as much as those obtained in the commercial and residential areas. This can be blamed to the less accurate tie point image coordinates that were measured automatically as tree tops. In that case, however, the resulting tie point accuracy is usually less than that of the other cases, and, thus, resulting in less accurate calibration parameters. Therefore, forested areas are not suitable for the airborne calibration of the DSS.
• The RMS of the remaining y-parallax is about 5, 4, 6 microns, while the maximum is about 7, 6, and 12 microns in the Commercial, Forest, and Residential areas, respectively. Note that the pixel size of the CCD chip of the DSS is 9 microns. This implies that the RMS of the remaining y-parallax is always within 1 pixel (i.e. well within the GSD) and that the maximum remaining y-parallax is approximately 1 pixel which happened only in 10% of the Residential areas. Note that the remaining y-parallax is a measure of the relative accuracy of the final solution.

6. The DSS fast Orthophoto production
To demonstrate the DSS capability in producing orthophoto mosaics in timely fashion, the in-flight calibrated DSS data was imported directly to ERDAS IMAGINE OrthoBASE to perform DEM self-extraction. Using the image stereopairs, DEM was extracted with a 5 meter posting. This procedure was performed in fully automatic mode without any external reference (existing DEM) or manual interaction (pre-defined elevation points). Unlike normal mapping procedure, the DEM was brought to create orthomosaic for the test flight area without modification (review/edit), to demonstrate fast orthophoto production with minimum operator interaction. The extracted DEM and the corresponding orthomosaic from the test flight area are shown in Figure 3. Note that the orthomosaic illustrates the locations of GCPs only; it does not reflect the actual number of GCP available. A summary of the self-extracted DEM and the orthomosaic is presented in Table 6. Note that the EO parameters used for orthophoto generation was derived from flight calibration using only 1 GCP, to absorb any datum shifts.

Table 6: Summary of Self-Extracted DEM and Orthomosaic DEM Extraction

Figure 3: Self-Extracted DEM (left), Orthomosaic (right)
Table 6: Summary of Self-Extracted DEM and Orthomosaic DEM Extraction

To analyse the accuracy of the produced orthophoto, a number of the checkpoints were measured on the resulting orthomosaic and compared to the land-surveyed coordinates. The statistics of the checkpoint residuals with respect to individual terrain and overall test area are presented in Table 7.

Table 7: Orthomosaic Checkpoint Residuals (m)

Examining the results in Table 7, it is obvious that the checkpoint accuracy is better than 0.5 meter in all terrain areas. Note that more than 10 checkpoints have been used in each terrain area and they vary from parking lot lines, sidewalk curbs to driveway corners, etc. Table 8 represents the orthophoto horizontal accuracy requirement from The United States Geological Survey (USGS).

Table 8: USGS Orthophoto Horizontal Accuracy Requirement

Using the aforementioned horizontal accuracy requirements listed in Table 8 and the DSS-derived orthophoto accuracy presented in Table 7, it is clear that the orthomosaic generated from this DSS test flight data presented here can be resampled up to a map scale 1: 600. This has proven that fast orthophoto production using DSS data fits various mapping standards, with minimum operator interaction during the process.

7. Conclusions
The Digital Sensor System (DSS) has been used to collect airborne digital images and their associated direct georeferencing data over Southern Ontario, Canada to evaluate the performance of the DSS for Fast Orthophoto production. A subset of the images collected in the Southern Ontario Flight (flown over Ajax) has been used to analyse the calibration and quality control procedure developed by Applanix for the DSS. ISAT has been used to automatically collect tie points in the imagery and then POSCal has been used to calibrate the individual system components and the system as a whole. The airborne calibrated parameters (using only one control point) have been introduced to ISAT once more to perform an independent check of the checkpoint analysis using the EO Analysis capability of ISAT software. The results of such analysis is presented in some detail and confirmed the validity of the calibration and quality control procedure and tools developed for the DSS.

The DSS digital imagery along with their associated POS-derived exterior orientation parameters were then used to extract a Digital Elevation Model of the Ajax area. The semi automatically extracted DEM has been used to generate an orthophoto mosaic of the same area. The DEM extraction and Orthophoto mosaic generation have been produced using ERDAS Imagine, with minimal operator interaction. The resulting Orthophoto mosaic was evaluated using checkpoint residual analysis and the results of such analysis are presented in some detail.

Of final note: the orthophoto presented in this paper was produced in a matter of days, including data acquisition., In addition, the orthophoto generated by the DSS self-extracted DEM contains the most up-to-date terrain information, which is especially useful for GIS database update in which re/developing areas are of interest. Finally, the real-time navigation solution provided by the POS AV system could be used to produce a near real-time orthophoto using an existing DEM, such as from the USGS. This is very useful in rapid response applications such as fire and hazard monitoring or protection.

8. Acknowedgements
Financial Support of the first author has been partially provided by The University of Calgary, The National Sciences and Engineering Research Council of Canada (NSERC) and by Applanix Corporation. Z/I Imaging Corporation has provided the ISAT software which has been used in the automated tie point generation and EO analysis used in this research. Leica Geosystems provided the ERDAS Imagine software which has been used for the DEM generation and orthophoto production used in the presented analysis. Aircraft and Crew were provided by The Airborne Sensing Corporation, Toronto, Canada. Thanks to Joe Hutton, Greg Lipa, and Ernest Yap of Applanix Corporation for their cooperation during data acquisition, processing and analysis.

9. References

• Braun, J., 2003. Aspects on True-Orthophoto Production. Proceedings, 49th Photogrammetric Week, Stuttgart, Germany, September 1-5, 2003.
• Congalton, R.G., M. Balogh, C. Bell, K. Green, J.A.Milliken, and R. Ottman, (1998). Mapping and Monitoring Agricultural Crops and other LandCover in the Lower Colorado River Basin. PE&RS,64 (11):1107-1113.
• Cramer, M., D. Stallmann, and N. Halla, 1997. High Precision Georeferencing Using GPS/INS and Image Matching, Proceedings of the International Symposium on Kinematic Systems in Geodesy, Geomatics and Navigation, Banff, Canada, pp. 453-462.
• El-Sheimy N., 1996. A Mobile Multi-Sensor System For GIS Applications In Urban Centers. The International Society for Photogrammetry and Remote Sensing (ISPRS) 1996, Commission II, Working Group 1, Vol. XXXI, Part B2, pp. 95- 100, Vienna, Austria, July 9-19, 1996.
• King, D., Walsh, P., and Ciuffreda, F. (1994) “Airborne Digital Frame Camera for Elevation Determination” PE&RS 60(11): 1321-1326.
• Mills, J.P., I. Newton, and R.W. Graham, 1996: Aerial Photography for Survey Purposes with a High Resolution, Small Format, Digital Camera” Photogrammetric Record 15 (88): 575-587, October, 1996.
• Mostafa, M.M.R., 2003. Design and Performance of the DSS. Proceedings, 49th Photogrammetric Week, Stuttgart, Germany, September 1-5, 2003.
• Mostafa, M.M.R., K.P. Schwarz, and P. Gong, 1997. A Fully Digital System for Airborne Mapping, Proceedings of the International Symposium on Kinematic Systems in Geodesy, Geomatics and Navigation, Banff, Canada, pp 463-471.
• Reid, D.B., E., Lithopoulos, J., Hutton, 1998. Position and Orientation System for Direct Georeferencing (POS/DG), ION 54th Annual Meeting, Denver, Colorado, June 1-3, pp. 445-449.
• Schwarz, K.P., M.A., Chapman, M.E., Cannon, and P., Gong, 1993. An Integrated INS/GPS Approach to The Georeferencing of Remotely Sensed Data, PE&RS, 59(11): 1167-1674.
• Toth, C. and D.A. Grejner-Brzezinska, 1998. Performance Analysis of The Airborne Integrated Mapping System (AIMS TM ), International Archives of Photogrammetry and Remote Sensing, 32 (2):320-326.