Ortho mapping with mosaic datasets

Ortho mapping refers to the processes of generating ortho products, such as an image mosaic, a digital terrain model (DTM), and a digital surface model (DSM), from a set of remote-sensing images.

When many images are required to fully cover your study area, you need to mosaic these images together into an orthomosaic dataset. These satellite and aerial images come with geometric errors, as well as misalignments between adjacent image scenes due to systematic and nonsystematic factors. To improve geometric accuracy and reduce the misalignment, bundle block adjustment, also known as image triangulation, needs to be performed on these images.

The digital elevation model (DEM) used in the image orthorectification plays a very important role in the geometric accuracy of the final image mosaic, especially for areas with diverse terrains. In cases where high geometric accuracy of the output image mosaic is needed, you should derive high-resolution DSMs and DTMs from stereo pairs.

ArcGIS Desktop supports bundle block adjustment of satellite and aerial imagery. The software can generate ortho mapping products such as orthomosaics, DTMs, and DSMs from the bundle block adjustments. Below is a diagram of the high-level workflows for generating ortho mapping products.

There are six tasks in the ortho mapping workflow:

• Author a mosaic dataset.
• Perform bundle block adjustment.
• Generate a point cloud from a mosaic dataset.
• Generate a DTM and DSM.
• Balance color and reduce seams.
• Generate ortho mapping products.

Author a mosaic dataset

A mosaic dataset is the data model in ArcGIS that is used to manage and process a collection of images such as satellite images, aerial images, scanned aerial photos, and UAS and UAV images. The ortho mapping workflow starts from authoring a mosaic dataset from the images of your study area.

1. Create a mosaic dataset using the Create Mosaic Dataset tool.
2. Add the data using the Add Rasters to Mosaic Dataset tool, choosing the appropriate raster type for the data. For more information on raster types, see What is a raster type.

Below is a footprint view of a mosaic dataset created from UAV images.

Once a mosaic dataset is authored, the next step is to perform bundle block adjustment.

Perform bundle block adjustment

Bundle block adjustment is the process of refining image orientation and geometric parameters by solving a mathematic model established from a set of control points. The computed geometric parameters will then be used to correct image orientation and with the elevation model to orthorectify the images. The control points used in this process include three types:

• A tie point is the same point location appearing in multiple overlapped images. These multiple points do not have known ground coordinates, but each has its own image coordinates. A tie point may be a two-ray tie point where it has two corresponding points in two overlapped images, or it can be multiple rays, depending on how many overlapped images are used.
• A ground control point (GCP) is a point that has known ground coordinates.
• A check point is similar to a GCP but used in examining the adjustment accuracy.

The bundle block adjustment establishes collinearity equations from these control points and computes transformations using the least-squares algorithm by minimizing and distributing the errors among those control points. As a result, image geometric errors will be corrected or minimized, and mismatches will be resolved.

The ArcGIS Desktop bundle block adjustment solution includes a set of geoprocessing tools in the Ortho Mapping toolset:

• For automatic tie point generation
• For automatic computation of ground control points from a reference image or image chips
• For DTM and DSM generation
• For rigorous block adjustment for satellite images, aerial images, and drone images, as well as polynomial adjustment for remote-sensing images that cannot be described by frame cameras or Rational Polynomial Coefficients (RPCs)

The image matching computation leverages multiple cores of a single machine or multiple machines deployed through ArcGIS Server and can handle large image collection. In addition, a Block Adjustment window can be used in editing control points and analyzing adjustment results.

Below is the workflow for bundle block adjustment of a mosaic dataset in ArcGIS Desktop.

After a mosaic dataset has been created, the next three steps prepare the control points that are required in the bundle block adjustment process. After the various control points have been created, they should be analyzed in terms of coverage. Then the adjustments can be computed, and the solution can be applied to your mosaic dataset.
1. Use the Compute Tie Points tool to automatically generate matching points from the overlapping images within the mosaic dataset.

   In addition, the Block Adjustment window can be used to manually edit tie points.

2. Add GCPs for your mosaic dataset. There are several ways to add your GCPs provided by ArcGIS:
   • If you have a reference image, you can use the Compute Control Points tool to compute the GCPs, and then use the Append Control Points tool to add them to the control point table. The reference image can be any image that has accurate coordinate system information, such as an imagery layer from ArcGIS Data and Maps, or a georeferenced image chip.
   • If your GCPs are stored in a text file or table, you can import it using the Block Adjustment window.
   • If your GCP coordinates are not stored in a text file or table, you can manually enter coordinates from the Block Adjustment window.
3. Create check points using the same method as you would create GCPs. Once these check points have been created, use the Block Adjustment window to change the type from GCP to CHK.

   Check points are similar to GCPs but are used to help examine the adjustment accuracy.

4. Create the control point coverage and overlap tables by running the Analyze Control Points tool.

   Accurate and balanced control points are very important in the bundle block adjustment process. To visualize and assess the control point coverage, use the two tables within the Block Adjustment window.

5. When the control point table is ready, compute adjustments using the Compute Block Adjustments tool to obtain a solution table and a solution point table.

   RPC	Used for satellite images that have an RPC model.
   Frame	Used for aerial images.
   Polyorder	A nonrigorous adjustment algorithm that computes the transformation in map space without adjusting the camera orientation. It can be used for images that do not have an RPC or frame camera model. The primary goal is to reduce misalignment between overlapping images without emphasizing ground control accuracy.

   A solution table stores the computed transformation for each image. A solution point table contains all points that result from the bundle block computation, such as the points in 3D ground coordinates. For more information about the output tables, see Schemas of block adjustment tables.

   For UAS and UAV images, where there is less information on the camera orientation, the Compute Camera Model tool is designed to automatically compute the camera orientation parameters. First run the tool using a reduced image resolution level. Then compute the camera model again at full resolution with all block adjustment steps consolidated into a single geoprocessing tool.

   

6. Use the Block Adjustment window to analyze the block adjustment computation and further improve the accuracy.
   1. Analyze the root mean square (RMS) error for each image.
   2. Check the residuals of control points, such as sorting to find which images have large RMS.
   3. Find which points have residuals.
   4. Find where the blunder points exist.
   5. Edit, delete, or add more tie points.
7. Once you have refined your control points, recompute your adjustments using the Compute Block Adjustments tool.
8. When the adjustment accuracy is acceptable, run the Apply Block Adjustment tool to apply the adjustment to the mosaic dataset and images. The resultant mosaic dataset will have correct and improved georeferencing.

The geometric accuracy of the mosaic dataset can be further improved when high-resolution elevation data is create for the adjustment solution. The high-resolution elevation data can be generated from the adjusted mosaic dataset, such as shown in the Generate DTM and DSM section below.

Generate a point cloud from a mosaic dataset

A stereo image pair contains two images of the same area but taken at different angles. 3D points can be computed from a stereo image pair by triangulating these matching points. The mosaic dataset that is used to generate a point cloud must meet two requirements: the mosaic dataset has been geometrically corrected, and the mosaic dataset should have stereo pairs established using the Build Stereo Model tool.

If the mosaic dataset has not been geometrically corrected, you need to perform the bundle block adjustment. To create a good stereo pair, the two images should have a large area of overlap, and a moderate intersection angle. If there is not enough overlap or the intersection angle is either too small or too large, the pair of images is not a good stereo pair and should be eliminated when building the stereo model.

When the mosaic dataset is ready, run the Generate Point Cloud tool to create the output the point cloud. The results are saved as a folder of LAS files. The LAS files can then be used as an input to generate a DSM or DTM or to create a LAS dataset.

ArcGIS Desktop provides three algorithms to generate a point cloud:

• Enhanced Terrain Matching (ETM)—Computes quickly by extracting fewer feature points, which is useful for quick DTM generation
• Semi-Global Matching (SGM)—Generates relatively dense points with high detail, which is good for urban areas that have buildings
• Enhanced SGM (ESGM)—Produces clear building edges by increasing computational capacity

Below are two products of generating point clouds. The image on the left is an adjusted mosaic dataset. The image on the right is a LAS dataset model, which was created from the LAS points generated from the mosaic dataset.

Generate a DTM and DSM

The Generate Point Cloud tool outputs 3D points in LAS file format from a mosaic dataset. This can be used to create a surface: a DTM is created by interpolating the points on the ground, or a DSM can be created by interpolating all points.

The solution point table, which is the output of the block adjustment computation, can also be used to interpolate a DTM. Since solution points are not as dense as the LAS point cloud obtained from the Generate Point Cloud tool, the triangulation or natural neighbor methods should be used to avoid producing pixels with missing information.

Below is a DTM output on the left and a hillshaded DSM on the right.

Balance color and reduce seams

After the mosaic dataset’s geometric correction is finished, there are two additional techniques that can be used to improve its visual appearance: color balance the images, and compute the seamlines in the mosaic dataset.

Color balancing is often needed for satellite images because the cross-strip images are often taken on different days. Before you can perform color balancing, the statistics and histograms first need to be computed for each image in the mosaic dataset using the Build Pyramids and Statistics tool. Once the statistics and histograms have been built, run the Color Balance Mosaic Dataset tool to balance the image colors. In general, the Dodging method with the Second Order color surface produces a good result.

Seamlines are polygons that are used for defining mosaicking boundaries and resolving the image overlaps. The Build Seamlines tool provides some common algorithms to automatically compute seamlines:

• Radiometry—Derives seamlines based on the pixel values using the best path algorithm on a cost surface
• Voronoi—Partitions image footprints based on an area Voronoi diagram
• Disparity—Determines the seamlines by leveraging the disparity images of the stereo pairs and is useful for large-scale mosaicking where the mosaic area has buildings

To reduce the computation complexity and select the most appropriate images, the Compute Mosaic Candidates tool should be run prior to seamline generation and color balancing operations.

Below is an example of seamline generation built with the Disparity method on UAV images.

Generate ortho mapping products

The last step is to generate ortho products. ArcGIS Desktop provides tools to generate various forms of orthomosaic products based on your requirement.

• If the requirement is to share it as an image service, you can publish the mosaic dataset as an image service.
• If the requirement is to share as an ArcGIS Online tile service, you can generate a tile cache and use this workflow publish.
• If the requirement is to generate a mosaic in a file format, use the Copy Raster tool to generate the output as a .tif or .crf file.
• If the requirement is to generate image tiles based on a user-defined tile size, use the Split Raster tool to create a series of tiles from the orthomosaic dataset.
• If the requirement is to generate orthorectified image scenes, use the Export Mosaic Dataset Items tool to export each image within the mosaic dataset.

Below is the final ortho mapping result, which is an orthomosaic stored as a TIFF image.