Understanding coordinate management in ArcGIS

In this topic

---

Coordinate grids, coordinate snapping, and coordinate comparison

The resolution and domain extent values determine how the coordinates of a geometry are stored. Resolution values are in the same units as the associated coordinate system. For example, if a spatial reference is using a projected coordinate system with units of meters, the x,y resolution value is defined in meters. For many applications, a resolution value of 0.0001 meter (1/10 mm) is appropriate. You should use a resolution value that is at least 10 times smaller than the accuracy of the data. If a coordinate system is unknown, or for Ms, you need to set appropriate resolution values according to the data without knowing the unit of measure.

The following equation is used to generate integers for feature data sources that persist coordinate values as integers:

In this equation, the left side is the integer value to be persisted. Round is a function that rounds its double-precision, floating-point operand to the nearest 53-bit integer. Snapping a coordinate to a grid means applying the equation (1) previously shown, followed by applying its inverse, to generate a double-precision coordinate rounded to the specified resolution and corresponding to an integer grid point. Personal geodatabases, for example, persist coordinates as double-precision values, but those values have been snapped. Coordinates are not snapped when working with a shapefile.

If you want to compare coordinates, you have several options. If your coordinates have not yet been "seen" by ArcGIS in any way, then the method of comparison is up to you. Coordinates that have been processed by a topological operator (Simplify or Intersect, for example) or that have been stored into a personal, file, or enterprise geodatabase, however, have been snapped, as described previously. In that case, you have the following options for coordinate comparison:

IRelationalOperator.Equals is a tolerance-based comparison operator. It uses the x,y tolerance property of the spatial reference associated with the geometries being compared to determine if two x,y coordinate values are equal. This method does not compare Zs, Ms, or point ID vertex attributes.

IClone.IsEqual is also a tolerance-based comparison operator but compares vertex attributes using the z- and m-tolerances when comparing z- and m-attributes. There is no tolerance for the point ID attribute.

IGeometry.SnapToSpatialReference moves the coordinates of a geometry to their closest grid points (see the previous discussion of snapping). ISpatialReference2.ApplyPrecision is a low-level method that accomplishes the same thing but operates on arrays of point structures (WKSPoints) rather than on geometry objects. ISpatialReference2GEN.ApplyPrecision is available for Java & .NET languages that cannot pass C-style arrays to ArcObjects methods. It is implemented on the SpatialReferenceEnvironment object and delegates to ISpatialReference2.ApplyPrecision. It is important to understand that snapping is a stable operation. The floating-point representation of a coordinate may change the first time it has been snapped but not thereafter.

Use SnapToSpatialReference or ApplyPrecision if you want to compare double-precision quantities exactly (that is, if you want to avoid using the tolerance property). In general, it is difficult to predict if an unsnapped double-precision value will compare exactly with a snapped value, even if they appear to be equal when printed as a sequence of base 10 digits. A good reference for gaining a precise understanding of the behavior of double-precision quantities is "What Every Computer Scientist Should Know About Floating-Point Arithmetic," by David Goldberg, published in the March 1991 issue of Computing Surveys. Copyright 1991, Association for Computing Machinery, Inc.

One recommendation is to keep the minimum domain x-, y-, and z-values as close as possible to the extent of the data. This minimizes the exponent shifting that occurs during the floating-point subtraction operation in the preceding equation (1), and also allows better compression of the integer versions of the coordinates, as described in the following text.

In addition to the integerization of map coordinates, ArcSDE and file geodatabases compress the resulting integer coordinates by removing leading zeros among other things. Coordinates with large numeric values after integerization don’t compress as well as those with smaller numeric values, so coordinates clustered near the upper right corner of the domain extent do not compress as well as those near the lower left, or minimum domain extent.

Coordinate compression and changes to the spatial reference at ArcGIS 9.2

If you've worked with geometries prior to ArcGIS 9.2, the resolution values are the inverse of the precision values. The precision values are also known as the xyUnits, zUnits, and mUnits or the scale factor. At 9.2, the terms resolution and precision are often treated as synonyms. There may be parts of the application programming interface (API) or documentation that still refer to precision or scale factor when they should use the term resolution instead. The word precision also is used to describe a spatial reference with an enhanced grid, as discussed next.

The coordinate values of a geometry must fall within the appropriate x,y; z; and m domain extents. The largest legal map coordinate value is the upper right of the domain extent. This coordinate value must also be representable as an integer according to the preceding equation (1). The domain extents and resolution values are connected. Together they form a grid mesh with the resolution defining the mesh separation, or cell size. Internally, the resolution and domain extents are used to define an integer grid.

At version 9.1 and prior releases, each integer coordinate was allotted 31 bits. At 9.2, 53 bits are provided for data sources created and managed by 9.2. Thus, for a given resolution, the domain extent can be much larger, or alternatively, for a given domain extent, the resolution can be much smaller than is possible for data using a low-precision spatial reference. A data source that stores 53-bit coordinates is always associated with a high-precision spatial reference. Existing datasets that have not had their spatial references upgraded are always associated with low-precision spatial references.

As an example, if the minimum domain value is 0 and the resolution is 1, the maximum domain value for a high-precision data source is 9007199254740990 or 253-2. If the resolution is 0.0001, the maximum domain value is 900719925474.0990. These values are for high-precision spatial references that are new at version 9.2. Prior to this release, spatial references were what is now called low precision. A low-precision spatial reference uses maximum domain values of 2147483647 or 231-1. If the same resolution value of 0.0001 and a minimum domain extent of 0 are used, the largest value a geometry could have is 214748.3647. That is too small for many projected coordinate systems such as universal transverse Mercator (UTM) and State Plane. So, when you work with low-precision spatial references, you must carefully balance the trade-off between domain extent and the resolution or precision values.

If possible, you should always work with high-precision spatial references. By default at version 9.2, a new spatial reference is high precision. If you are editing or creating data for a geodatabase of any type that has not had its spatial reference upgraded, you have to continue to work with low-precision spatial references. New Component Object Model (COM) interfaces are available on the various spatial reference objects to determine whether one is low or high precision. A new interface is provided on the SpatialReferenceEnvironment object to convert between high- and low-precision spatial references using certain assumptions.

In addition to the resolution and domain extent properties of a spatial reference’s coordinate grid, the XY tolerance property is also important. The XY tolerance property is applied to x- and y-coordinates during relational and topological operations. The XY tolerance property of a spatial reference describes how far a point is allowed to move in the x,y plane whenever it is necessary to determine if two points are the same. The minimum allowable tolerance value is twice the resolution (or 2.0/scale factor). The Z tolerance property is used when validating topologies in a geodatabase. Different tolerance values can produce different answers for relational and topological operations. For example, two geometries might be classified as disjoint (no points in common) with the minimum tolerance, but a larger tolerance might cause them to be classified as touching.

To achieve precise and predictable results using the geometry library, it is essential that the spatial reference of geometries in a workflow is well defined. When performing a spatial operation using two or more geometries, for example, an intersection, the coordinate systems of the two geometries must be equal. If the coordinate systems of the geometries are different or undefined, the operation could produce unexpected results. Prior to version 9.2, the resolution (precision) determined the tolerance value used in a geometry operation. Today, a spatial reference should have explicitly defined tolerance values. In two-at-a-time geometry operations, such as the methods found on the IRelationalOperator and ITopologicalOperator interfaces, the tolerance value of the left operand geometry is used. If the tolerance property of the spatial reference is undefined or there is no spatial reference associated with a geometry, a default grid guaranteed to contain the operands, and the minimum allowable tolerance based on the grid, are used.

The spatial reference is not defined when creating an instance of a geometry. It is the developer's responsibility to define a spatial reference that makes sense for the geometry and for the operation. If an existing geometry is coming from a feature class, then usually the spatial reference is well defined and the geometry can be used directly or projected without problems.