Python code in AWS Lambda to load shapefile to PostgreSQL/PostGIS (RDS)

Question

I am new to GIS implementation. I am trying to develop AWS Lambda code in Python to load shapefile dynamically.

I developed the code after doing some research and it is perfectly working on my local computer.

But the same code is troubling when I am trying to run in AWS Lambda.

I have added libraries (Lambda Layers) for 'OSGEO/GDAL' in AWS Lambda and tested it by calling import and it's working fine.

Following is the code :

import os
import subprocess
import boto3
import urllib.parse
from osgeo import gdal
from osgeo import ogr
s3 = boto3.client('s3')
def lambda_handler(event, context): 
 bucket = event['Records'][0]['s3']['bucket']['name']
 s3key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')
 # input shapefile
 input_shp = ('s3://' + bucket + '/' + s3key)
 # database options
 db_schema = "SCHEMA=public"
 overwrite_option = "OVERWRITE=YES"
 geom_type = "MULTILINESTRING"
 output_format = "PostgreSQL"
 
 # database connection string
 db_connection = """PG:host=<RDS host name> port=5432 user=<RDS User Name> dbname= postgres password=<RDS Password>"""
 
 
 # call ogr2ogr from python
 subprocess.call(["ogr2ogr", "-lco", db_schema, "-lco", overwrite_option, "-nlt", geom_type, "-f", output_format, db_connection, input_shp])

Error Message is :

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: 'ogr2ogr': 'ogr2ogr'

The same code is working fine on my local with only difference that instead of S3 I am providing hard-coded path where shape file is stored on my local machine.

I have used following instructions to load OSGEO/GDAL in AWS lambda Layer :

https://github.com/developmentseed/geolambda

It seems this import doesn't carry ogr2ogr binaries. Any suggestions?

• priyanka suneja: can you share how you got this working.
• Vikas Kumar: @priyankasuneja - It worked when instead of using ogr2ogr, I used 'psycopg2' library. Subprocess call is not supported Aws Lambda.

Answer 1

It worked when instead of ogr2ogr subprocess call, I used 'psycopg2' library to establish connection. It seems subprocess call is not supported in AWS Lambda.