Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit a6c4adb

Browse files
Add global LIMA_BATS_ALL_TESTS_RETRIES variable
It will rerun all tests up to this number on failure. Also provides a `flaky` function to set the retries to LIMA_BATS_FLAKY_TESTS_RETRIES to allow a greater number of retries to known flaky tests. Signed-off-by: Jan Dubois <jan.dubois@suse.com>
1 parent 282d5d2 commit a6c4adb

File tree

2 files changed

+14
-9
lines changed

2 files changed

+14
-9
lines changed

‎.github/workflows/test.yml‎

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -362,14 +362,9 @@ jobs:
362362
with:
363363
template: templates/k8s.yaml
364364
- name: "Run BATS k8s tests"
365-
# Wish we could use BATS_TEST_RETRIES=3 as an environment variable here,
366-
# but bats does not seem to support it.
367-
uses: nick-fields/retry@ce71cc2ab81d554ebbe88c79ab5975992d79ba08 # v3.0.2
368-
with:
369-
timeout_minutes: 30
370-
retry_on: error
371-
max_attempts: 3
372-
command: ./hack/bats/lib/bats-core/bin/bats --timing ./hack/bats/extras/k8s.bats
365+
run: ./hack/bats/lib/bats-core/bin/bats --timing ./hack/bats/extras/k8s.bats
366+
env:
367+
LIMA_BATS_ALL_TESTS_RETRIES: 3
373368

374369
colima:
375370
name: "Colima tests (QEMU, Linux host)"

‎hack/bats/helpers/load.bash‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,16 @@ set -o errexit -o nounset -o pipefail
88
# The upstream PR https://github.com/bats-core/bats-core/pull/1118 is still open, so our submodule points to the PR commit.
99
export BATS_RUN_ERREXIT=1
1010

11+
# BATS_TEST_RETRIES must be set for the individual test and cannot be imported from the
12+
# parent environment because the BATS test runner sets it to 0 before running the test.
13+
BATS_TEST_RETRIES=${LIMA_BATS_ALL_TESTS_RETRIES:-0}
14+
15+
# Known flaky tests should call `flaky` inside the @test to allow retries up to
16+
# LIMA_BATS_FLAKY_TESTS_RETRIES even when the LIMA_BATS_ALL_TESTS_RETRIES is lower.
17+
flaky() {
18+
BATS_TEST_RETRIES=${LIMA_BATS_FLAKY_TESTS_RETRIES:-$BATS_TEST_RETRIES}
19+
}
20+
1121
# Don't run the tests in ~/.lima because they may destroy _config, _templates etc.
1222
export LIMA_HOME=${LIMA_BATS_LIMA_HOME:-$HOME/.lima-bats}
1323

@@ -71,4 +81,4 @@ assert_output_lines_count() {
7181
declare -A -g TEST_CONTAINER_IMAGES=(
7282
["nginx"]="ghcr.io/stargz-containers/nginx:1.19-alpine-org"
7383
["coredns"]="public.ecr.aws/eks-distro/coredns/coredns:v1.12.2-eks-1-31-latest"
74-
)
84+
)

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /