-
Notifications
You must be signed in to change notification settings - Fork 161
python arm build issue still failing in some cases #348
Description
Description:
python lambda function builds using the sam cli (which calls this library) are still failing for certain dependencies
Steps to reproduce the issue:
- setup python project with sam cli
- add py7zr to the requirements.txt of a function
- run command sam build -u
Observed result:
build fails on resolve dependencies
Expected result:
build completes
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
aws codebuild
image
aws/codebuild/amazonlinux2-x86_64-standard:3.0
buildspec.yaml:
version: 0.2
env:
secrets-manager:
DOCKER_PASSWORD: *******
DOCKER_USER: ******
phases:
install:
runtime-versions:
python: 3.9
build:
commands:
- printenv
- pip install --upgrade aws-sam-cli
- docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
- echo $DOCKER_PASSWORD | docker login --username $DOCKER_USER --password-stdin
- sam build -u --debug
- sam package --s3-bucket ******i --output-template-file template-out.yaml
artifacts:
files:
- template-out.yaml
sam build -u --debug output near failure:
2022年03月22日 18:35:43,680 | Building codeuri: /codebuild/output/src531839338/src/lambda/******** runtime: python3.9 metadata: {} architecture: arm64 functions: ['*****']
2022年03月22日 18:35:43,680 | Building to following folder /codebuild/output/src531839338/src/.aws-sam/build****
Fetching public.ecr.aws/sam/build-python3.9:latest-arm64 Docker container image......
2022年03月22日 18:35:44,617 | Mounting /codebuild/output/src531839338/src/lambda/****** as /tmp/samcli/source:ro,delegated inside runtime container
Using the request object from command line argument
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch --exists-action i
Full dependency closure: {brotli==1.0.9(wheel), pycryptodomex==3.14.1(sdist), texttable==1.6.4(wheel), zipfile-deflate64==0.0.0(sdist), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel)}
initial compatible: {brotli==1.0.9(wheel), texttable==1.6.4(wheel), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel)}
initial incompatible: {pycryptodomex==3.14.1(sdist), zipfile-deflate64==0.0.0(sdist)}
Downloading missing wheels: {pycryptodomex==3.14.1(sdist), zipfile-deflate64==0.0.0(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux2014_aarch64 --implementation cp --abi cp39 --dest /tmp/samcli/scratch pycryptodomex==3.14.1
calling pip download --only-binary=:all: --no-deps --platform manylinux2014_aarch64 --implementation cp --abi cp39 --dest /tmp/samcli/scratch zipfile-deflate64==0.0.0
compatible wheels after second download pass: {brotli==1.0.9(wheel), texttable==1.6.4(wheel), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel)}
Build missing wheels from sdists (C compiling True): {pycryptodomex==3.14.1(sdist), zipfile-deflate64==0.0.0(sdist)}
calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/pycryptodomex-3.14.1.tar.gz
calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/zipfile-deflate64-0.2.0.tar.gz
compatible after building wheels (no C compiling): {brotli==1.0.9(wheel), pycryptodomex==3.14.1(wheel), texttable==1.6.4(wheel), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel), zipfile-deflate64==0.2.0(wheel)}
Build missing wheels from sdists (C compiling False): {zipfile-deflate64==0.0.0(sdist)}
calling pip wheel --no-deps --wheel-dir /tmp/samcli/scratch /tmp/samcli/scratch/zipfile-deflate64-0.2.0.tar.gz
compatible after building wheels (C compiling): {brotli==1.0.9(wheel), pycryptodomex==3.14.1(wheel), texttable==1.6.4(wheel), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel), zipfile-deflate64==0.2.0(wheel)}
Final compatible: {brotli==1.0.9(wheel), pycryptodomex==3.14.1(wheel), texttable==1.6.4(wheel), pybcj==0.5.0(wheel), multivolumefile==0.2.3(wheel), pyppmd==0.17.4(wheel), pyzstd==0.15.2(wheel), py7zr==0.18.1(wheel), zipfile-deflate64==0.2.0(wheel)}
Final incompatible: set()
Final missing wheels: {zipfile-deflate64==0.0.0(sdist)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute
package_builder.build_dependencies(
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 156, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 261, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {zipfile-deflate64==0.0.0(sdist)}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflow.py", line 302, in run
action.execute()
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 57, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {zipfile-deflate64==0.0.0(sdist)}
Builder workflow failed
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute
package_builder.build_dependencies(
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 156, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 261, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {zipfile-deflate64==0.0.0(sdist)}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflow.py", line 302, in run
action.execute()
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 57, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {zipfile-deflate64==0.0.0(sdist)}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/main.py", line 118, in main
builder.build(
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/builder.py", line 164, in build
return workflow.run()
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflow.py", line 96, in wrapper
func(self, *args, **kwargs)
File "/usr/local/opt/sam-cli/lib/python3.9/site-packages/aws_lambda_builders/workflow.py", line 309, in run
raise WorkflowFailedError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - {zipfile-deflate64==0.0.0(sdist)}
2022年03月22日 18:43:02,875 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - {zipfile-deflate64==0.0.0(sdist)}"}}
Build Failed
2022年03月22日 18:43:03,179 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '64785e7d-a17c-4f7f-a347-aca7b7adc804', 'installationId': '75a7108d-f9a0-4aa3-ad9a-b637b8ea0647', 'sessionId': 'c7ec421f-a00b-47f1-b1e6-b34189e99768', 'executionEnvironment': 'AWSCodeBuild', 'ci': True, 'pyversion': '3.9.5', 'samcliVersion': '1.41.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 667631, 'exitReason': 'BuildInsideContainerError', 'exitCode': 1}}]}
2022年03月22日 18:43:03,217 | Telemetry response: 200
Error: PythonPipBuilder:ResolveDependencies - {zipfile-deflate64==0.0.0(sdist)}
[Container] 2022年03月22日 18:43:03 Command did not exit successfully sam build -u --debug exit status 1
[Container] 2022年03月22日 18:43:03 Phase complete: BUILD State: FAILED
[Container] 2022年03月22日 18:43:03 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build -u --debug. Reason: exit status 1
[Container] 2022年03月22日 18:43:03 Entering phase POST_BUILD
[Container] 2022年03月22日 18:43:03 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2022年03月22日 18:43:03 Phase context status code: Message:
[Container] 2022年03月22日 18:43:03 Expanding base directory path: .
[Container] 2022年03月22日 18:43:03 Assembling file list
[Container] 2022年03月22日 18:43:03 Expanding .
[Container] 2022年03月22日 18:43:03 Expanding file paths for base directory .
[Container] 2022年03月22日 18:43:03 Assembling file list
[Container] 2022年03月22日 18:43:03 Expanding template-out.yaml
[Container] 2022年03月22日 18:43:03 Skipping invalid file path template-out.yaml
[Container] 2022年03月22日 18:43:03 Phase complete: UPLOAD_ARTIFACTS State: FAILED
[Container] 2022年03月22日 18:43:03 Phase context status code: CLIENT_ERROR Message: no matching artifact paths found