Geoprocessing framework

The geoprocessing framework is the set of windows and dialog boxes you use to manage and execute tools. This topic focuses on the high-level concepts and ideas behind this framework rather than the mechanics of using it.

The core idea behind geoprocessing is to allow you to quickly and easily turn your ideas into new software that can be executed, managed, modified, documented, and shared with the ArcGIS user community. Software, in this case, means something that instructs ArcGIS to do what you want. A geoprocessing model, for example, is new software built by you with an easy-to-use visual programming language called ModelBuilder.

The main theme of this section is the idea that geoprocessing is a way for you to create new useful software. By doing so, it is hoped you'll have a broader and deeper understanding of how and why to use geoprocessing.

To create software of any kind, the following two essential elements are needed:

• A formal language that operates on the data captured within the system.
• A framework for creating, managing, and executing software based on this language. This includes things such as editors, browsers, and documentation tools.

Geoprocessing's language is its collection of tools. The geoprocessing framework is a small collection of built-in user interfaces for organizing and managing existing tools and creating tools. The basic components of the framework are shown in A quick tour of geoprocessing and consist of the following:

• The Search window, where you find and execute tools, and the Catalog window where you browse to toolboxes to manage or execute your tools
• The tool dialog box for interactively filling out tool parameters and executing the tool
• The Python window for executing a tool by typing in its parameters
• The ModelBuilder window for chaining together sequences of tools
• Methods for creating scripts and adding them to toolboxes.

Geoprocessing models and ModelBuilder

The tool dialog box allows you to execute a single tool. You can think of this as executing a single instruction in a programming language. While single tool execution is practical, the system would not be very useful unless you could string together multiple tools, feeding the output of one into another, just like a programming language.

In the geoprocessing framework, the ModelBuilder window is where you quickly and turn your ideas into software by chaining together elements of the geoprocessing language (the tools) into a sequence. It's important to realize that models are software, since they instruct the computer to do something. The programming language is visual—what you see in ModelBuilder—rather than text-based like a traditional programming language.

The most important thing to note here is that models are tools. They behave exactly like all other tools in the system. You can execute them in the dialog box or in the Python window. Since models are tools, you can embed models within models; several of the system tools provided with ArcGIS are models.

You can make your models complex. You can use any system or custom tools in a model, including other models you've written (since models are tools). You can also use loops and conditions to control the logical flow of a model.

Models can be simple and still be productive. You can create a model that contains a single tool but embeds some of its parameters. For example, the Buffer tool takes six parameters, but for your current set of tasks, you know that three of these parameters will always be the same. Rather than filling in these parameters each time you execute the Buffer tool, you can quickly create a model and set these three parameters, save it as the MyBuffer tool, and use its dialog box rather than the Buffer dialog box. You might only use MyBuffer a few times before deleting it, but it's no loss because it was quick and easy to create and productive for you to do so.

Scripting

You can also use a scripting language to create useful software. A program that uses a scripting language is a script. In software programming, languages can be divided into two basic categories: system languages and scripting languages. System languages are things such as C++ and .NET that are used to create applications from scratch, using low-level primitives and the raw resources of the computer. Scripting languages, such as Python and Perl, are used to bind applications together, using built-in higher-level functions of the computer and masking the nuts and bolts a system language programmer must deal with. Compared to system languages, scripting languages are easier to learn and use—a basic understanding of programming is all that's needed to be productive.

In the geoprocessing framework, scripts are analogous to models in that they can be used to create tools. Models are created with a visual programming language (ModelBuilder), and scripts are created with a text-based language and text editors.

Just like models, scripts are tools. You can introduce a script to a custom toolbox using a step-by-step wizard, and it becomes another tool that you can use in a model or in another script. Several of the system tools are scripts. Technically, you can write a script and not introduce it to a toolbox, in which case it's not a tool but a stand-alone script on disk.

You may want to use scripting for the following reasons:

• You may need advanced programming logic, such as conditional execution and advanced error handling; advanced data structures, such as dictionaries and lists; or more functionality, such as string, math, and file manipulation functions. Many scripting languages have been extended with third-party libraries for things such as advanced math and statistics, web automation, database queries, and advanced system utilities.
• There are some low-level geoprocessing functions available only in scripts. Cursors, for example, allow you to loop through records on a table, reading or writing rows and inserting new rows. There are functions to access the properties of ArcGIS data, such as the extent of a feature class or the various properties of individual fields on a table.
• Scripts can wrap other software—the binding together of applications. For example, you may have a model that outputs a simple text file of parcel owners and addresses affected by a zoning ordinance change, and you want to launch another program that reads this text file and generates official notification letters for the owners of the affected parcels. You could use a script to wrap this letter generation program, introduce this script to a toolbox, and use it directly in the model.
• Scripts can be executed outside ArcGIS. That is, you can execute the script directly from the operating system prompt. (You still must have the ArcPy site package installed on the machine since you need access to the geoprocessing tools.)

Framework for creating and managing software

The geoprocessing framework was built to allow you to quickly turn your ideas into new software that can be managed by the system and shared among users.

Geoprocessing is a language consisting of operators, or tools, that operate on the data within ArcGIS (tables, feature classes, rasters, TINs, and so on), and perform tasks that are necessary for manipulating and analyzing geographic information across a wide range of disciplines.

You can quickly create software in the form of models and scripts. These new tools perform tasks that are not part of the standard ArcGIS package. For example, there is no menu, button, or programming object in ArcGIS that performs the simple Project and Clip model shown in What is geoprocessing.

Tools are managed by the geoprocessing framework, which means you don't have to. This is a subtle but important point. This management includes the following:

• All tools, whether system tools or custom (user-written) tools, can be accessed from their toolbox. Imagine a different situation where models, scripts, and system and custom tools were each accessed by different interfaces and methods—it would be difficult to use and manage. In geoprocessing, all things are created and managed equally, whether they are built-in tools, model tools, or script tools.
• Tools are all documented the same way. Once you create a tool, you can document your tool in the Catalog window so it can be cataloged and searched by the system. Compare this to the alternative of leaving documentation standards and management to the user.
• Tools have the same user interface: the dialog box. These dialog boxes are automatically created based on the tool parameters. You don't have to do any user interface programming. Consider the alternative where user interface design and programming are left to the tool author.

Tools can be easily shared. A toolbox with all its tools and toolsets is either contained in a file on disk with a .tbx extension or in a geodatabase. Anyone with access to the file or geodatabase can run its tools.

The notable point is that your tools become components of the geoprocessing framework, where they have consistent documentation, user interface, methods of access, and methods of sharing.

Geoprocessing and ArcObjects

ArcObjects is the extensive library of low-level programming objects delivered as part of the ArcGIS Software Development Kit (SDK). Developers use ArcObjects to build new applications or extend the existing functionality of ArcGIS applications. (Most system tools and the entire geoprocessing framework were built using ArcObjects.) As with geoprocessing, the ArcObjects SDK can be used to create software.

The ArcObjects SDK and geoprocessing are complementary; neither obsoletes the other. ArcObjects is used to extend ArcGIS with new behavior, while geoprocessing is designed to automate tasks. You use ArcObjects to do things such as add new user interfaces, add custom behavior to feature classes, or create a special cartographic renderer. Geoprocessing is used to create software (models and scripts) that automates tasks within the confines of a framework.

ArcObjects is used with a system programming language, where the programmer must access low-level primitives to implement complex logic and algorithms. This is why ArcObjects contains thousands of objects and requests—to allow the programmer the fine degree of control required. Because ArcObjects is used with a system programming language, it requires more programming knowledge than geoprocessing, with its models and scripts.

Conversely, geoprocessing is a universal capability that can be used and deployed by all GIS users to automate their work, build repeatable and well-defined methods and procedures, and model important geographic processes.