Tools that honor the Parallel Processing Factor environment will divide and perform operations across multiple processes.
Many modern computers include multiple-core CPUs. Spreading a geoprocessing operation across multiple processes can speed up performance by taking advantage of more than one core. The performance benefit of parallel processing varies from tool to tool.
Usage notes
- The value of this environment determines the number of processes across which a tool spreads its operation. Those processes will be divided between hardware cores (processors) built into the machine. The number of hardware cores does not change based on this setting.
- Each tool that honors this environment has a built-in default for the number of processes given a particular machine. You can change this based on your data, operation, and available resources.
- If you specify a percent value (using the % symbol) then the number of processes used will be the specified percentage of the number of cores on the machine, rounded to the nearest integer. For example, on a 4-core machine:
- Setting 50% means the operation will be spread over 2 processes (50% * 4 = 2).
- Setting 66% means the operation will be spread over 3 processes (66% * 4 = 2.64 which rounds to 3).
- Setting 100% means the operation will be spread over all 4 processes (100% * 4 = 4).
When geoprocessing using a SQL Server Express database, you need to limit the number of parallel processes to two.
SQL Server Express allows a maximum of three connections at a time. Each processing CPU requires a connection to the server. As well, the software running the tool, such as ArcGIS for Desktop, counts as one connection process, leaving only two worker connection processes available for parallel processing.
- Specifying more processes than your machine has cores may incur a performance penalty. This is because multiple processes will compete for resources on one core. To specify the Parallel Processing Factor in a way that avoids this competition you can either use a percent value less than 100% or a number of processes less than the number of cores on your machine.
However, for cases where all your processes are I/O bound to a disk or to an enterprise database connection, you may get better performance by specifying more processes than you have cores. For example, the Add Rasters to Mosaic Dataset tool is I/O bound when the mosaic dataset is stored in an enterprise database. Also, the Build Overviews tool is primarily I/O bound to the disk. You can use more processes than your machine has cores by either specifying a percent value greater than 100% or a number of processes greater than the number of cores on your machine. For example, if you have a four-core machine, then specifying 8 or 200% will spread operations over eight processes.
Dialog syntax
Parallel Processing Factor—The number of processes across which a tool will spread its operation.
- blank (empty)—Let each tool determine how many processes to use. This is the default.
- 0—Do not spread operations across multiple processes.
- n—Use the specified number of processes.
- n%—Calculate the number of processes using the specified percentage: Number of processes = number of system cores * n / 100.
Scripting syntax
arcpy.env.parallelProcessingFactor = string
| Parameters | Explanation |
|---|---|
empty string (blank) | Let each tool determine how many processes to use. This is the default. |
0 | Do not spread operations across multiple processes. |
|
n |
Use the specified number of processes. |
n% | Calculate the number of processes using the specified percentage: Number of processes = number of system cores * n / 100. |
Script example
import arcpy
# Use half of the cores on the machine.
arcpy.env.parallelProcessingFactor = "50%"