Batch intersect and multiprocessing with arcpy

Question

I admit what I am working on is way over my head, especially as I've only been dabbling in python for a few months. I am attempting to write a script that allows me to use Intersect analysis on about sixty raster images for each of their sides. To make this more efficient, I would like to use multiprocessing. I have read many tutorials on this, but I have decided to follow the framework that ESRI used in their blog post as it appears to be the least confusing.

What I have written so far is only for two processes, but I will eventually be using a computer that will allow six.

import os
import re
import multiprocessing
import arcpy
def intersect_1():
 inFeatures = ["file1", "file2"]
 intersectOutput = ["intersect_result"]
 clusterTolerance = 1.5
 arcpy.Intersect_analysis(inFeatures, intersectOutput, "", clusterTolerance, "")
def intersect_2():
 inFeatures = ["file2", "file3"]
 intersectOutput = ["intersect_result2"]
 clusterTolerance = 1.5
 arcpy.Intersect_analysis(inFeatures, intersectOutput, "", clusterTolerance, "")
def main():
 workspace = 'c:/Users/Hannah/Desktop/wrs2_d'
 arcpy.env.workspace = workspace
 pool = multiprocessing.Pool()
 pool.map(intersect_1,intersect_2)
 pool.close()
 pool.join()
if __name__ == '__main__':
 main()

These are the errors I continue to get, but my grasp of basic python is so shaky I cannot understand why. Can anyone point me in the right direction?

Traceback (most recent call last):
 File "C:/Users/rs1/Desktop/possol_1.py", line 31, in <module>
main()
 File "C:/Users/rs1/Desktop/possol_1.py", line 25, in main
pool.map(intersect_1,intersect_2)
 File "C:\Python27\ArcGIS10.3\lib\multiprocessing\pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
 File "C:\Python27\ArcGIS10.3\lib\multiprocessing\pool.py", line 304, in map_async
iterable = list(iterable)
TypeError: 'function' object is not iterable

Answer 1

Review the examples again and look at the map function's arguments. pool.map(function, array) is what they pass. You are trying to pass two functions like pool.map(function1, function2).

You need to find a way to pass multiple arguments to the function using the pool and use something like an iterator to help group the arguments, then repeat the second argument in the next group (file1, file2), (file2, file3).

Maybe this thread will help. The answer that stands out to me is imotai's:

def multi_run_wrapper(args):
 return add(*args)
def add(x,y):
 return x+y
if __name__ == "__main__":
 from multiprocessing import Pool
 pool = Pool(4)
 results = pool.map(multi_run_wrapper,[(1,2),(2,3),(3,4)])
 print results

• eruiggy: That makes sense. I figured the problem was something to do with my understanding of how it worked.