Fix container-related misconfigurations in release build tasks #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

per1234 merged 2 commits into arduino:main from per1234:fix-release-tasks

Nov 13, 2024

Merged

Fix container-related misconfigurations in release build tasks #201

per1234 merged 2 commits into arduino:main from per1234:fix-release-tasks

Nov 13, 2024

Conversation

per1234

Copy link

Contributor

@per1234 per1234 commented Nov 12, 2024 •

edited

Loading

Please check if the PR fulfills these requirements

The PR has no duplicates (please search among the Pull Requests
before creating one)
The PR follows our contributing guidelines
[N/A] Tests for the changes have been added (for bug fixes / features)

What kind of change does this PR introduce?

Bug fix.

What is the current behavior?

DistTasks.yml contains the tasks used to produce the release builds of the project for each of the host targets. The builds are produced in Docker containers.

Some regressions were introduced in these task at the time the project's Go version was bumped to 1.21.5 (which includes bumping the versions of the images used by the tasks: 1ccd378 / #174).

Non-existent container tag referenced in the Linux_ARMv6 task.
Repository configuration incompatible for use in a container with modern Git versions due to "dubious ownership" (go vcs stamping will break with new git elastic/golang-crossbuild#232 ).

These regressions would cause the failure of the "Release" workflow run when it is triggered by a new release of the project.

What is the new behavior?

Use Debian 11 in `Linux_ARMv6` release build task

The Linux_ARMv6 task must use a specific version of Debian in the container:

arduino-language-server/DistTasks.yml

Lines 147 to 174 in 5444eb5

# We are experiencing the following error with ARMv6 build:

# # github.com/arduino/arduino-cli

# net(.text): unexpected relocation type 296 (R_ARM_V4BX)

# panic: runtime error: invalid memory address or nil pointer dereference

# [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x51ae53]

# goroutine 1 [running]:

# cmd/link/internal/loader.(*Loader).SymName(0xc000095c00, 0x0, 0xc0000958d8, 0x5a0ac)

# /usr/local/go/src/cmd/link/internal/loader/loader.go:684 +0x53

# cmd/link/internal/ld.dynrelocsym2(0xc000095880, 0x5a0ac)

# /usr/local/go/src/cmd/link/internal/ld/data.go:777 +0x295

# cmd/link/internal/ld.(*dodataState).dynreloc2(0xc007df9800, 0xc000095880)

# /usr/local/go/src/cmd/link/internal/ld/data.go:794 +0x89

# cmd/link/internal/ld.(*Link).dodata2(0xc000095880, 0xc007d00000, 0x60518, 0x60518)

# /usr/local/go/src/cmd/link/internal/ld/data.go:1434 +0x4d4

# cmd/link/internal/ld.Main(0x8729a0, 0x4, 0x8, 0x1, 0xd, 0xe, 0x0, 0x0, 0x6d7737, 0x12, ...)

# /usr/local/go/src/cmd/link/internal/ld/main.go:302 +0x123a

# main.main()

# /usr/local/go/src/cmd/link/main.go:68 +0x1dc

# Error: failed building for linux/armv6: exit status 2

# This seems to be a problem in the go builder 1.16.x that removed support for the R_ARM_V4BX instruction:

# https://github.com/golang/go/pull/44998

# https://groups.google.com/g/golang-codereviews/c/yzN80xxwu2E

# Until there is a fix released we must use a recent gcc for Linux_ARMv6 build, so for this

# build we select the debian10 based container.

The Debian version is defined via the image tag. Previously, Debian 10 was used, and a tag of the Go 1.18.3 image was available for this Debian version. However, the maintainers of the image did not produce a Debian 10 variant of the Go 1.21.5 image, so the use of that tag caused the task to fail:

https://github.com/per1234/arduino-language-server/actions/runs/11772916519/job/32789011179#step:4:36

Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10' locally
docker: Error response from daemon: manifest for docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10 not found: manifest unknown: manifest unknown.
See 'docker run --help'.
task: Failed to run task "dist:Linux_ARMv6": exit status 125

A Debian 11 variant of the image is available, and this version of Debian is also suitable for release builds. So the solution is to update the image tag referenced in the task to the Debian 11 tag.

Configure repository for compatibility with modern Git versions in release build containers

As a security measure (see CVE-2022-24765), starting from 2.30.3 Git requires the repository folder to be owned by the operating system user's account. Due to it having been checked out outside the container, the repository does not meet this requirement inside the container. An older version of Git was installed in the Go 1.18.3 Docker image, so this was not a problem before the bump, but a newer version is used in the Go 1.21.5 containers, which causes some tasks to fail (elastic/golang-crossbuild#232):

https://github.com/per1234/arduino-language-server/actions/runs/11772976565/job/32789161736#step:4:148

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for linux/armv6: exit status 1
failed building for linux/armv6: exit status 1
task: Failed to run task "dist:Linux_ARMv6": exit status 1

https://github.com/per1234/arduino-language-server/actions/runs/11773011125/job/32789254088#step:4:146

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for darwin/amd64: exit status 1
failed building for darwin/amd64: exit status 1
task: Failed to run task "dist:macOS_64bit": exit status 1

https://github.com/per1234/arduino-language-server/actions/runs/11773011125/job/32789254192#step:4:161

error obtaining VCS status: exit status 128
	Use -buildvcs=false to disable VCS stamping.
Error: failed building for darwin/arm64: exit status 1
failed building for darwin/arm64: exit status 1
task: Failed to run task "dist:macOS_ARM64": exit status 1

The solution is to configure Git to allow the use of the repository, despite the "dubious ownership" of its folder. This is done via the safe.directory Git configuration variable.

This approach is already in use in Arduino CLI's release build tasks: arduino/arduino-cli@5a5ae94 / arduino/arduino-cli#2103

Other information

In order to facilitate the review of this pull request, I performed a demonstration release with the proposed changes in my fork:

https://github.com/per1234/arduino-language-server/actions/runs/11789922878

https://github.com/per1234/arduino-language-server/releases/tag/0.0.0-rc.12

per1234 added 2 commits

November 11, 2024 19:00

@per1234


 Use Debian 11 in Linux_ARMv6 release build task

0fb9206

`DistTasks.yml` contains the tasks used to produce the release builds of the project for each of the host targets. The
builds are produced in Docker containers.
A regression was introduced in the `Linux_ARMv6` task at the time the project's Go version was bumped to 1.21.5. This
task must use a specific version of Debian in the container, which is defined via the image tag. Previously, Debian 10
was used, and a tag of the Go 1.18.3 image was available for this Debian version. However, the maintainers of the image
did not produce a Debian 10 variant of the Go 1.21.5 image, so the use of that tag caused the task to fail:
```
Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10' locally
docker: Error response from daemon: manifest for docker.elastic.co/beats-dev/golang-crossbuild:1.21.5-armel-debian10 not found: manifest unknown: manifest unknown.
See 'docker run --help'.
task: Failed to run task "dist:Linux_ARMv6": exit status 125
```
A Debian 11 variant of the image is available, and this version of Debian is also suitable for release builds. So the
solution is to update the image tag referenced in the task to the Debian 11 tag.

@per1234


 Configure repository for compatibility with modern Git versions in re...

a14dd93

...lease build containers
`DistTasks.yml` contains the tasks used to produce the release builds of the project for each of the host targets. The
builds are produced in Docker containers.
A regression was introduced in several of the tasks at the time the project's Go version was bumped to 1.21.5. As a
security measure (see CVE-2022-24765), starting from 2.30.3 Git requires the repository folder to be owned by the
operating system user's account. Due to it having been checked out outside the container, the repository does not meet
this requirement inside the container. An older version of Git was installed in the Go 1.18.3 Docker image, so this was
not a problem before the bump, but a newer version is used in the Go 1.21.5 containers, which causes some tasks to fail:
```
error obtaining VCS status: exit status 128
 Use -buildvcs=false to disable VCS stamping.
Error: failed building for linux/armv6: exit status 1
failed building for linux/armv6: exit status 1
task: Failed to run task "dist:Linux_ARMv6": exit status 1
```
The solution is to configure Git to allow the use of the repository, despite the "dubious ownership" of its folder. This
is done via the `safe.directory` Git configuration variable.