Accuracy Assessment for Image Classification

Available with Spatial Analyst license.

An overview of the accuracy assessment workflow

Accuracy assessment is an important part of any classification project. It compares the classified image to another data source that is considered to be accurate or ground truth data. Ground truth can be collected in the field; however, this is time consuming and expensive. Ground truth data can also be derived from interpreting high-resolution imagery, existing classified imagery, or GIS data layers.

The most common way to assess the accuracy of a classified map is to create a set of random points from the ground truth data and compare that to the classified data in a confusion matrix. Although this is a two-step process, you may need to compare the results of different classification methods or training sites, or you may not have ground truth data and are relying on the same imagery that you used to create the classification. To accommodate these other workflows, this process uses three geoprocessing tools: Create Accuracy Assessment Points, Update Accuracy Assessment Points, and Compute Confusion Matrix.

The most common workflow is when you have classified imagery and you want to compare it to ground truth data. The first set of steps creates a set of random points.

1. Open the Create Accuracy Assessment Points tool and set the Target Field to Ground Truth.

The ground truth layer determines the number and placement of the random points according to the sampling strategy.

2. Select a sampling strategy.
   • Random—Generates random accuracy assessment points across the entire input dataset.
   • Stratified Random—Generates a set of accuracy assessment points that is proportional in number to the class area for each class.
   • Equalized Stratified Random—Generates a set of accuracy assessment points where each class has the same number of points.

A table is created listing each random point as a record along with a field for ground truth and a field for the classified image. The Ground Truth field is populated with its value while the Classified field is filled with a null value (-1).

3. Open the Update Accuracy Assessment Points tool.
4. Set the Input Raster or Feature Class data as the classified dataset.
5. Use the output from Create Accuracy Assessment Points tool as the Input Accuracy Assessment Points.
6. Set the Target Field to Classified.
The table is updated to include the value of each point based on the classified data. It now has both fields populated and is ready to be used to compute the confusion matrix.
7. Open the Compute Confusion Matrix geoprocessing tool and use the table generated in the previous step as the input.

After seeing the results of the accuracy assessment, you may need to adjust training samples or classification parameters, or choose a different classifier to get a better result. If this is the case, use the new classified data as the input to the Update Accuracy Assessment Points tool and set the Target Field to Classified and run Compute Confusion Matrix using this output.

If you have updated the ground truth data and need to run the assessment again, there are two options. One is to use the new ground truth data as the input to the Update Accuracy Assessment Points tool and set the Target Field to Ground Truth. This will keep the same set of points that was created the first time you performed the analysis. Alternatively, you can start from the beginning and use the Create Accuracy Assessment Points tool to generate a new set of points.

Another option is to create the points from the classified dataset and manually identify each point to populate the ground truth field. In this scenario, you would use the Create Accuracy Assessment Points, edit the Ground Truth Field, and run the Compute Confusion Matrix tool.