Simple Point Plug-In Data Source Example

Object Model Diagram: Click here.

Description: This project implements a plug-in data source for the SimplePoint data format to provide direct read-only ArcGIS support for the format.

Design: Required classes for a plug-in data source

Categories: ESRI Plugin Workspace Factory Helpers, ESRI Workspace Factories, and ESRI Gx Enabled Workspace Factories

Interfaces: IPlugInWorkspaceFactoryHelper, IPlugInWorkspaceHelper, IPlugInDatasetHelper, IPlugInDatasetInfo, IPlugInCursorHelper, and IPlugInFastQueryValues

Imagine that you have a regular supply of text files containing geographic locations, but the data in the files has an unusual format. You would like to use ArcGIS with this data, but you do not want to convert data every time a new file is received. In short, you would like ArcGIS to work with this data directly, just like it does with other supported data formats. This can be done by implementing a plug-in data source.

The data you will work with in this example follows a simple format. An ASCII text file contains data for each new point on a new line. The first six characters are the x-coordinate, the next six characters contain the y-coordinate, and the trailing characters contain an attribute value.

To make a plug-in data source, you must implement four required classes:

As a developer you will typically name these classes with a prefix corresponding to your data source - in the VB6 example they are called SPTWorkspaceFactoryHelper, SPTWorkspaceHelper, SPTDatasetHelper and SPTCursorHelper. In some documentation you will sometimes see these classes referred to generically with the prefix 'PlugIn', for example, a PlugInWorkspaceHelper.

As well as the four required classes, a plug-in data source can have an optional plug-in extension class and possibly several plug-in native type classes. These are not implemented in the example, but will be discussed later.

With each class there are one or more interfaces you need to implement. For detailed help on individual interface members, see the ArcGIS Developer Help.

A workspace factory helper class must implement the IPlugInWorkspaceFactoryHelper interface. This helper class works in conjunction with the existing ArcGIS PlugInWorkspaceFactory coclass.

The PlugInWorkspaceFactory class implements IWorkspaceFactory and uses the plug-in workspace factory helper to get information about the data source and to browse for workspaces - together they act as a workspace factory for the data source.

If you implement a plug-in workspace factory helper with C++, or another language that supports class aggregation, it should aggregate an instance of the existing geodatabase PlugInWorkspaceFactory coclass and register in the ESRI Workspace Factories and ESRI Gx Enabled Workspace Factories component categories. You must implement the workspace factory helper as a singleton object. The need for the singleton is a consequence of the following rule for data sources: datasets must be pointer comparable. That is, there can only be one dataset object for a dataset in each process thread. To ensure this, there must be only one workspace object for each workspace, and thus only one workspace factory that creates workspaces.

In addition to implementing the workspace factory as a singleton, you must maintain a cache of the plug-in workspaces that have been opened, and in each workspace object, a cache of the open datasets. These caches are used to avoid creating a second dataset object, when one already exists for that dataset in the process.

You could choose not to register to ESRI Gx Enabled Workspace Factories. This component category instructs ArcCatalog to create standard user-interface objects for the data source. If you don't register to this category, you will need to implement custom ArcCatalog objects for the data source to be displayed in ArcCatalog. There is more information about why you would adopt this approach later in this section.

Returning to the example, the remaining implementation of IPlugInWorkspaceFactoryHelper is mainly straightforward. The hardest member to implement is often GetWorkspaceString. The workspace string is used as a lightweight representation of the workspace.

Your plug-in is the sole consumer (IsWorkspace and OpenWorkspace) of the strings, so their content is up to you. For many data sources, including the example, the path to the workspace is chosen as the workspace string. Another thing to note about GetWorkspaceString is the FileNames parameter. This parameter may be null, in which case you should call IsWorkspace to determine if the directory is a workspace of your type. If the parameter is not null, you should examine the files in FileNames to determine if the workspace is of your type. You also need to remove any files from the array that belong to your data source. This behavior is comparable to that of IWorkspaceFactory::GetWorkspaceName.

The DataSourceName property is simple to implement - just return a string representing the data source. The example returns "SimplePoint". This is the only text string that should not be localized. You should localize the other strings (for example, by using a resource file) if your plug-in data source could be used in different countries. For simplicity, the example does not localize its strings.

The OpenWorkspace method creates an instance of the next class you must implement, the plug-in workspace helper. You need a way of initializing the workspace helper with the location of the data. The example does this by defining a new interface on the workspace helper, ISPTWorkspaceHelper, which provides a WorkspacePath property so that the location of the workspace can be passed.

For convenience, the new interface is defined in the Visual Basic project rather than with IDL. As described in Creating Type Libraries using IDL, interfaces defined in this way cannot be easily called from Visual C++ clients. However, in this case, there is no problem as the only consumer of the interface is the Visual Basic project.

Plug-in workspace factories may also implement the optional interface IPlugInCreateWorkspace to support creation of workspaces for a plug-in data source. See Implementing copy, rename and delete for plug-in data sources for more details.

Plug-in workspace factories may also implement the optional interface IWorkspaceFactoryFileExtensions to help improve ArcCatalog efficiency. See Improving browse performance in ArcCatalog for plug-in data sources for more details.

A plug-in workspace helper represents a single workspace for datasets of your data source type. The class does not need to be publicly cocreatable, as the plug-in workspace factory helper is responsible for creating it in its OpenWorkspace method.

The class must implement IPlugInWorkspaceHelper; this interface allows browsing of datasets. The most noteworthy member is OpenDataset, which creates and initializes an instance of a plug-in dataset helper.

If the SupportsSQLSyntax property of IPlugInWorkspaceFactoryHelper returns true, your plug-in workspace helper should implement the ISQLSyntax interface. In this case, the workspace object will delegate calls to its ISQLSyntax to the interface on this class. The ArcGIS framework will pass where clauses to the IPlugInDatasetHelper::FetchAll and FetchByEnvelope, and the cursors returned by these functions should contain only rows that match the where clause.

If SupportsSQLSyntax returns false, the ArcGIS framework won't pass where clauses, but will handle them with post-query filtering. The advantage of implementing support for where clauses is that you may be able to process queries on large datasets more efficiently than a post-query filter. The disadvantage is the extra implementation code required. The example returns false for SupportsSQLSyntax and so leaves handling of where clauses to the ArcGIS framework.

A plug-in workspace helper may implement IPlugInMetadata or IPlugInMetadataPath to support metadata. Implement IPlugInMetadata if your data source has its own metadata engine; this interface allows metadata to be set and retrieved as property sets. Otherwise, implement IPlugInMetadataPath; it allows the plug-in to specify a metadata file for each dataset. ArcGIS will then use these files for storing metadata. You should implement one of these interfaces for successful operation of the Export Data command in ArcMap. This command uses the FeatureDataConverter object which relies on metadata capabilities of data sources.

A plug-in workspace helper may also implement the optional interface IPlugInWorkspaceHelper2. See Implementing attribute indexes for plug-in data sources for more details.

A plug-in workspace helper may also implement the optional interface IPlugInLicense. See Implementing license handling for plug-in data sources for more details.

A plug-in dataset helper class must implement the IPlugInDatasetInfo and IPlugInDatasetHelper interfaces. It does not need to be publicly cocreatable, as a plug-in workspace helper is responsible for creating it.

IPlugInDatasetInfo provides information about the dataset so that the user interface can represent it. For example, ArcCatalog uses this interface to display an icon for the dataset. To enable fast browsing, it is important that the class have a low creation overhead. In the example, the SPTDatasetHelper class can be created and all the information for IPlugInDatasetInfo derived without opening the data file.

IPlugInDatasetHelper provides more information about the dataset and methods to access the data. If the dataset is a feature dataset (that is, it contains feature classes), all of the feature classes are accessed via a single instance of this class. Many of the interface members have a ClassIndex parameter that determines which feature class is being referred to.

IPlugInDatasetHelper::Fields defines the columns of the dataset. For the SimplePoint data source, all datasets have just three fields: Object ID, Shape, and a single attribute field, which in the example is arbitrarily named 'Column1'. When implementing Fields you must define the spatial reference of your dataset. In the example, for simplicity, an UnknownCoordinateSystem is chosen. If your spatial reference is a geographic coordinate system, you should put the extent of the dataset into the IGeographicCoordinateSystem2::ExtentHint property before setting the domain of the spatial reference. Setting the domain first can cause problems with projections and export.

All data sources must include an Object ID field. If your data does not have a suitable unique integer field, then you will need to generate a value on the fly. As will be seen later, the example uses the current line number in the text file as the Object ID. Another data source without explicit Object IDs is the shapefile format. In a similar way the ArcGIS framework generates a suitable unique integer automatically for each feature in a shapefile.

There are three similar members of IPlugInDatasetHelper that all open a cursor on the dataset: FetchAll, FetchByEnvelope, and FetchByID. In the example, all these methods cocreate a new plug-in cursor helper and initialize it with various parameters that will control the operation of the cursor. Here is the implementation of FetchByEnvelope.

An ISPTCursorHelper interface has been defined on the SPTCursorHelper class to pass parameters. In the above code, three parameters have been set: the field map will control which attribute values are fetched by the cursor, the query envelope will determine which rows are fetched by the cursor, and the filepath tells the cursor where the data is.

The example is able to ignore some of the FetchByEnvelope parameters as ClassIndex applies only to feature classes within a feature dataset and WhereClause applies only to those data sources supporting ISQLSyntax; strictSearch can be ignored since the example does not use a spatial index to perform its queries, and so always returns a cursor of features that strictly fall within the envelope.

There are other equally valid ways of implementing FetchByEnvelope, FetchById, and FetchAll; with your data source it may be more appropriate to create the cursor helper, then use a postprocess to filter the rows to be returned.

There is one more member of IPlugInDatasetHelper that is worth mentioning. The Bounds property returns the geographic extent of the dataset. Many data sources have the extent recorded in a header file, in which case implementing Bounds is easy. However, in the example, a cursor on the entire dataset must be opened and a minimum-bounding rectangle gradually built. The implementation makes use of IPlugInCursorHelper. Note that it would be quite unusual for another developer to consume the plug-in interfaces in this way, since once your data source is implemented, the normal geodatabase interfaces will work with it (albeit in a read-only manner). Another point to note about the Bounds property is that you must create a new envelope or clone a cached envelope. You can run into problems with projections if your class caches the envelope and passes out pointers to the cached envelope.

There is an optional interface IPlugInDatasetWorkspaceHelper2 which has a FetchByFilter method. If a plug-in implements this interface, this method will be called instead of FetchByEnvelope. FetchByFilter allows filtering by spatial envelope, where clause and by a set of OIDs (FIDSet).

This interface should be implemented like FetchByEnvelope, unless the FIDSet parameter is supplied. If the FIDSet is supplied, it contains a list of OIDs to return. The cursor should return only those rows whose OID is in the FIDSet, and which match the spatial filter and where clause.

A plug-in dataset helper should implement IPlugInFileSystemDataset if the data source is file-based and multiple files make up a dataset. Single-file and folder-based data sources do not need to implement this interface.

A plug-in dataset helper should implement IPlugInRowCount if the RowCountIsCalculated property of the workspace helper returns false. Otherwise, this interface should not be implemented. If you implement this interface, make sure it operates quickly. It should be faster than just opening a cursor on the entire dataset and counting.

A plug-in dataset helper may also implement the optional interfaces IPlugInFileOperations and IPlugInFileOperationsClass. See Implementing copy, rename and delete for plug-in data sources for more details.

A plug-in dataset helper may also implement the optional interfaces IPlugInIndexInfo and IPlugInIndexManager. See Implementing attribute indexes for plug-in data sources for more details.

A plug-in dataset helper may also implement the optional interface IPlugInLicense. See Implementing license handling for plug-in data sources for more details.

The plug-in cursor helper deals with the raw data and is normally the class for which you will write the most code. The cursor helper represents the results of a query on the dataset. The class must implement the IPlugInCursorHelper interface, but does not need to be publicly cocreatable, as the plug-in dataset helper is responsible for creating it.

NextRecord advances the cursor position. In the example, a new line of text is read from the file and stored in a string. As was described in the previous section, the dataset helper defines the way the cursor will operate; this is reflected in the example's implementation of NextRecord. If a record is being fetched by object ID, the cursor is advanced to that record. If a query envelope is specified, the cursor is moved on to the next record with a geometry that falls within the envelope.

With Visual C++ or other suitable languages, you should return S_FALSE (this value cannot be raised by Visual Basic 6). To enable debugging of a Visual Basic 6 implementation, it is useful to choose the 'Break on Unhandled Errors' setting on the General tab of the Options dialog box; this prevents the debugger from stopping whenever an object passes back an error HRESULT.

QueryShape should return the geometry of the feature. In common with many other ArcObjects methods having a name beginning with Query, the object to be returned is already instantiated in memory. VB developers in particular may find it helpful to review the 'Clientside storage members' discussion for more information.

For this PlugInCursorHelper, you only need to set the coordinates of the point feature.

For data sources with complex geometries, you can improve the performance of QueryShape by using a shape buffer. Use IESRIShape::AttachToESRIShape to attach a shape buffer to the geometry. This buffer should then be reused for each geometry.

The ESRI white paper, ESRI Shapefile Technical Description, can be referred to for more information on shape buffers, as shapefiles use the same shape format. You can find the white paper on the ESRI Web site, www.esri.com.

QueryValues returns the attributes of the current record. The field map (specified when the cursor was created) dictates which attributes to fetch. This is designed to improve performance by reducing the amount of data transfer; for example, when features are being drawn on the map, it is likely that only a small subset, or even none of the attribute values, will be required. The return value of QueryValues is interpreted by ArcGIS as the Object ID of the feature.

Implementing IPlugInFastQueryValues

A plug-in cursor helper implemented in Visual C++ may implement IPlugInFastQueryValues. The only method, FastQueryValues, should do the same thing as IPlugInCursorHelper::QueryValues, but as it passes open arrays, you should be able to provide a more efficient implementation.