Skip to content

Navigation Menu

Sign in
Sign up

Bug: git+https requirements fail to install when using python3.12 container #675

Open

Description

Description:

sam build fails when requirements.txt contains git+https links, python3.12 only

Steps to reproduce:

  1. SAM init hello world example
  2. Runtime: python3.12; Architectures: - arm64
  3. Add to requirements.txt:
gendercomputer@git+https://github.com/tue-mdse/genderComputer.git
  1. sam build --use-container --debug

Observed result:

Fetching public.ecr.aws/sam/build-python3.12:latest-arm64 Docker container image......
2024年08月14日 13:41:13,759 | Mounting .../sam-app/hello_world as /tmp/samcli/source:ro,delegated, inside runtime container 
Using the request object from command line argument
...
 Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch --exists-action i
pip stdout: b"Collecting gendercomputer@ git+https://github.com/tue-mdse/genderComputer.git (from -r /tmp/samcli/source/requirements.txt (line 1))\n Cloning https://github.com/tue-mdse/genderComputer.git to /tmp/pip-download-cnqo230n/gendercomputer_83fc8aa3dae241db97874f2c91a268bd\n Resolved https://github.com/tue-mdse/genderComputer.git to commit f6267615517913e53cb0b882b248f1c2e11b8bbc\n Installing build dependencies: started\n Installing build dependencies: finished with status 'done'\n Getting requirements to build wheel: started\n Getting requirements to build wheel: finished with status 'done'\n Preparing metadata (pyproject.toml): started\n Preparing metadata (pyproject.toml): finished with status 'done'\nCollecting unidecode==1.3.2 (from gendercomputer@ git+https://github.com/tue-mdse/genderComputer.git->-r /tmp/samcli/source/requirements.txt (line 1))\n Downloading Unidecode-1.3.2-py3-none-any.whl.metadata (13 kB)\nCollecting nameparser==1.0.6 (from gendercomputer@ git+https://github.com/tue-mdse/genderComputer.git->-r /tmp/samcli/source/requirements.txt (line 1))\n Downloading nameparser-1.0.6-py2.py3-none-any.whl.metadata (5.9 kB)\nDownloading nameparser-1.0.6-py2.py3-none-any.whl (23 kB)\nDownloading Unidecode-1.3.2-py3-none-any.whl (235 kB)\n \xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81\xe2\x94\x81 235.7/235.7 kB 5.9 MB/s eta 0:00:00\nSaved /tmp/samcli/scratch/gendercomputer-0.1.zip\nSaved /tmp/samcli/scratch/nameparser-1.0.6-py2.py3-none-any.whl\nSaved /tmp/samcli/scratch/Unidecode-1.3.2-py3-none-any.whl\nSuccessfully downloaded gendercomputer nameparser unidecode\n"
pip stderr: b"WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.\nPlease see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.\nTo avoid this problem you can invoke Python with '-m pip' instead of running pip directly.\n Running command git clone --filter=blob:none --quiet https://github.com/tue-mdse/genderComputer.git /tmp/pip-download-cnqo230n/gendercomputer_83fc8aa3dae241db97874f2c91a268bd\n\n[notice] A new release of pip is available: 24.0 -> 24.2\n[notice] To update, run: pip install --upgrade pip\n"
Non zero rc (1) from the setup.py egg_info command: b'Traceback (most recent call last):\n File "<string>", line 1, in <module>\nModuleNotFoundError: No module named \'setuptools\'\n'
Using fallback location for PKG-INFO file in package directory: /tmp/tmpdamop0ow/gendercomputer
PythonPipBuilder:ResolveDependencies raised unhandled exception
Traceback (most recent call last):
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflow.py", line 374, in run
 action.execute()
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 71, in execute
 package_builder.build_dependencies(
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 157, in build_dependencies
 self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 262, in build_site_packages
 wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 286, in _download_dependencies
 deps = self._download_all_dependencies(requirements_filename, directory)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 370, in _download_all_dependencies
 Package(directory, filename, self.python_exe)
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 522, in __init__
 self._name, self._version = self._calculate_name_and_version()
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 564, in _calculate_name_and_version
 name, version = info_fetcher.get_package_name_and_version(sdist_path)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 706, in get_package_name_and_version
 pkg_info_filepath = self._get_pkg_info_filepath(package_dir)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 618, in _get_pkg_info_filepath
 raise UnsupportedPackageError(self._osutils.basename(package_dir))
aws_lambda_builders.workflows.python_pip.packager.UnsupportedPackageError: Unable to retrieve name/version for package: gendercomputer
Builder workflow failed
Traceback (most recent call last):
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflow.py", line 374, in run
 action.execute()
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 71, in execute
 package_builder.build_dependencies(
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 157, in build_dependencies
 self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 262, in build_site_packages
 wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 286, in _download_dependencies
 deps = self._download_all_dependencies(requirements_filename, directory)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 370, in _download_all_dependencies
 Package(directory, filename, self.python_exe)
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 522, in __init__
 self._name, self._version = self._calculate_name_and_version()
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 564, in _calculate_name_and_version
 name, version = info_fetcher.get_package_name_and_version(sdist_path)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 706, in get_package_name_and_version
 pkg_info_filepath = self._get_pkg_info_filepath(package_dir)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 618, in _get_pkg_info_filepath
 raise UnsupportedPackageError(self._osutils.basename(package_dir))
aws_lambda_builders.workflows.python_pip.packager.UnsupportedPackageError: Unable to retrieve name/version for package: gendercomputer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/__main__.py", line 117, in main
 builder.build(
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/builder.py", line 172, in build
 return workflow.run()
 ^^^^^^^^^^^^^^
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflow.py", line 116, in wrapper
 func(self, *args, **kwargs)
 File "/usr/local/opt/lambda-builders/lib/python3.12/site-packages/aws_lambda_builders/workflow.py", line 385, in run
 raise WorkflowUnknownError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowUnknownError: PythonPipBuilder:ResolveDependencies - Unable to retrieve name/version for package: gendercomputer
2024年08月14日 13:41:17,306 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - Unable to retrieve name/version for
package: gendercomputer"}} 
2024年08月14日 13:41:17,634 | Exception raised during the execution 
Build Failed

Expected result:

Build successful.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

Notice No module named \'setuptools\' in the logs.
The issue is not reproducible when changing Runtime to python3.11.

{
 "version": "1.121.0",
 "system": {
 "python": "3.12.5",
 "os": "macOS-14.6.1-arm64-arm-64bit"
 },
 "additional_dependencies": {
 "docker_engine": "27.1.1",
 "aws_cdk": "Not available",
 "terraform": "1.9.4"
 },
 "available_beta_feature_env_vars": [
 "SAM_CLI_BETA_FEATURES",
 "SAM_CLI_BETA_BUILD_PERFORMANCE",
 "SAM_CLI_BETA_TERRAFORM_SUPPORT",
 "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
 ]
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      AltStyle によって変換されたページ (->オリジナル) /