Tools that honor the Parallel Processing Factor environment will divide and perform operations across multiple processes.
Many modern computers include multiple-core CPUs. Spreading a geoprocessing operation across multiple processes can speed up performance by taking advantage of more than one core. The performance benefit of parallel processing varies from tool to tool.
Usage notes
- The value of this environment determines the number of processes across which a tool spreads its operation. Those processes will be divided between hardware cores (processors) built into the machine. The number of hardware cores does not change based on this setting.
- Each tool that honors this environment has a built-in default for the number of processes given a particular machine. You can change this based on your data, operation, and available resources.
- If you specify a percent value (using the % symbol), the number of processes used will be the specified percentage of the number of cores on the machine, rounded to the nearest integer. For example, on a 4-core machine
- Setting 50% means the operation will be spread over 2 processes (50% * 4 = 2).
- Setting 66% means the operation will be spread over 3 processes (66% * 4 = 2.64 which rounds to 3).
- Setting 100% means the operation will be spread over all 4 processes (100% * 4 = 4).
When geoprocessing using a SQL Server Express database, you need to limit the number of parallel processes to two.
SQL Server Express allows a maximum of three connections at a time. Each processing CPU requires a connection to the server. Additionally, the software running the tool, such as ArcGIS Desktop, counts as one connection process, leaving only two worker connection processes available for parallel processing.
- Specifying more processes than your machine has cores may incur a performance penalty. This is because multiple processes will compete for resources on one core. To specify the Parallel Processing Factor in a way that avoids this competition you can either use a percent value less than 100% or a number of processes less than the number of cores on your machine.
However, for cases where all your processes are I/O bound to a disk or to an enterprise database connection, you may get better performance by specifying more processes than you have cores. For example, the Add Rasters to Mosaic Dataset tool is I/O bound when the mosaic dataset is stored in an enterprise database. Also, the Build Overviews tool is primarily I/O bound to the disk. You can use more processes than your machine has cores by either specifying a percent value greater than 100% or a number of processes greater than the number of cores on your machine. For example, if you have a four-core machine, then specifying 8 or 200% will spread operations over eight processes.
Dialog syntax
Parallel Processing Factor—The number of processes across which a tool will spread its operation.
- blank (empty)—Let each tool determine how many processes to use. This is the default.
- 0—Do not spread operations across multiple processes.
- n—Use the specified number of processes.
- n%—Calculate the number of processes using the specified percentage: Number of processes = number of system cores * n / 100.
Scripting syntax
arcpy.env.parallelProcessingFactor = string
| Parameters | Explanation |
|---|---|
empty string (blank) | Let each tool determine how many processes to use. This is the default. |
0 | Do not spread operations across multiple processes. |
|
n |
Use the specified number of processes. |
n% | Calculate the number of processes using the specified percentage: Number of processes = number of system cores * n / 100. |
Script example
import arcpy
# Use half of the cores on the machine.
arcpy.env.parallelProcessingFactor = "50%"