Downloading file using URL value in shapefile field accessed via ArcPy cursor

Question

How do I download files through Python using URLs located in a shapefile with ArcPy?

The script needs to read a shapefile, create folders and download images using URLs in more than 1 field/record (all starting with url*).

Answer 1

import os
import arcpy
import shutil
import requests
# since I have no access to INDEX_utm20_PEI_Charlottetown_2008.shp, I am using 
# a test shapefile with 5 fields: 
# id (int), name (string), url1 (string), url2 (string), notes (string)
shapefile = r"data\photos.shp"
photo_location = r"data\photos"
fields = [field.name for field in arcpy.ListFields(shapefile, "url*", "string")]
with arcpy.da.SearchCursor(shapefile, ["id", "name"] + fields) as cursor:
 for row in cursor:
 
 id = row[0] # unique
 name = row[1]
 destination = os.path.join(photo_location, name)
 os.makedirs(destination, exist_ok=True)
 
 for url, field_name in zip(row[2:], fields):
 
 file_path = os.path.join(destination, f"{id}-{name}-{field_name}.jpg")
 
 response = requests.get(url, stream=True)
 if response.status_code == 200:
 with open(file_path, 'wb') as file:
 response.raw.decode_content = True
 shutil.copyfileobj(response.raw, file)

See also:

• https://stackoverflow.com/a/13137873/42659
• https://stackabuse.com/download-files-with-python/
• https://docs.python-requests.org/en/master/

---

The above example does not use arcpy.env.workspace. I avoid using that whenever possible, especially in a a standalone script, or a complex script (using more than one path/workspace). I think this makes the script more explicit and easier to read.