-
Notifications
You must be signed in to change notification settings - Fork 435
Installing asyncpg on OS Amazon Linux #808
-
Hey I'm planning to use asyncpg as a part of my data pipeline infrastructure. All my pipelines run as AWS Lambda functions which themselves run the OS Amazon Linux. https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
I tried installing asyncpg in this environment, but at runtime I'm getting:
[ERROR] Runtime.ImportModuleError: No module named 'asyncpg.protocol.protocol'
I'm adding asyncpg as a dependency by creating a Lambda layer as described here:
https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
I searched around in issues and elsewhere but failed to find this problem described. Anyone aware of the cause of this and perhaps have a suggestion for workaround?
Beta Was this translation helpful? Give feedback.
All reactions
Lambda requires extra hoops to be jumped through for binary packages. See, e.g. this post
Replies: 2 comments 3 replies
-
Lambda requires extra hoops to be jumped through for binary packages. See, e.g. this post
Beta Was this translation helpful? Give feedback.
All reactions
-
I tried another approach than your example. I spun up a server with Amazon Linux, installed asyncpg there and imported those files to my Lambda function but same error as I posted above. Will debug a bit more, if no success I'll attempt the docker solution in your example link.
Beta Was this translation helpful? Give feedback.
All reactions
-
Followed the tutorial in your post, this works! Thank you so much :)
Beta Was this translation helpful? Give feedback.
All reactions
-
I faced the same problem. And I solved it in a way that did not rely on external tools.
As described in official Documentation by AWS, we had to specify the pip install command with --platform and other options.
Also note that the architecture must be correctly specified in the runtime configuration.
Remember to specify asyncpg as Extras if you are using asyncpg with dependencies from other libraries such as the "databases" library.
Beta Was this translation helpful? Give feedback.
All reactions
-
❤️ 1
-
x86_64
pip install \ --platform manylinux2014_x86_64 \ --target=package \ --implementation cp \ --python-version <version> \ --only-binary=:all: --upgrade \ <package_name>
arm64
pip install \ --platform manylinux2014_aarch64 \ --target=package \ --implementation cp \ --python-version <version> \ --only-binary=:all: --upgrade \ <package_name>
Beta Was this translation helpful? Give feedback.