-
-
Couldn't load subscription status.
- Fork 2.6k
How to properly disassemble selenium/video to generate evidence in gitlab-ci? #2258
-
I have the following test scenario:
I have a gitlab that is using runner that I have configured on my machine, so the configuration is something simple, it is being done as follows:
concurrent = 1 check_interval = 0 shutdown_timeout = 0 [session_server] session_timeout = 1800 [[runners]] name = "myname" url = "https://gitlab.com" id = myid token = "mytoken" token_obtained_at = 2023年05月31日T11:15:39Z token_expires_at = 0001年01月01日T00:00:00Z executor = "docker" [runners.cache] MaxUploadedArchiveSize = 0 [runners.docker] tls_verify = false image = "alpine" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 2000000000
And my gitlab-ci.yml file, which is basically my pipelnie, looks like this
stages: - "e2e" e2e: image: python stage: e2e services: - name: selenium/standalone-chrome alias: browser - name: selenium/video alias: video variables: FF_NETWORK_PER_BUILD: "true" SE_VIDEO_FOLDER: "/cache" SE_VIDEO_FILE_NAME: 'video.mp4' DISPLAY_CONTAINER_NAME: browser before_script: - python3 -m venv .venv - source .venv/bin/activate - pip3 install poetry - poetry install --only main - cd project/ script: - python test.py after_script: - mkdir -p ${CI_PROJECT_DIR}/evidency - cp -r /cache/video.mp4 ${CI_PROJECT_DIR}/evidency artifacts: when: on_failure paths: - ./evidency expire_in: 1 week
Using this way, I'm getting the audio file to be correctly sent to my project folder so that I can use it as evidence and later visualization in case of errors.
However, the file is coming in at 256KB, which basically means that there is no content in this video file. From what I've investigated, I've come to the conclusion that the video container that runs inside Runner wasn't having a "soft drop", I don't know what expression to use for that.
But basically, while on my machine I could use a docker compose file to do "docker compose up -d" and then "docker compose down" which gives the container time to shut down and compile the video correctly, when this scenario goes to gitlab's Runner this no longer happens, as it is shut down abruptly.
In the image below you can see the exact moment when the local container is closed and selenium/video has time to correctly create the .mp4 file
Now, on the other hand, when the selenium/video container that is running in the gitlab runner is shut down, it shuts down dramatically and consequently doesn't mount the video correctly.
Has anyone had this problem and managed to solve it in any way other than "docker:dind"?
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 2 comments 7 replies
-
May I know video image tag is used in this setup?
Beta Was this translation helpful? Give feedback.
All reactions
-
Im not using any tag.
The example that i'm put here is exactly what i'm using.
I think this problem is because the container of selenium video is not shutdown propertly.
But i dont have any idea what i need to change in my gitlabci. :(
Beta Was this translation helpful? Give feedback.
All reactions
-
So, do you know the last version that worked properly in your env?
Our CI also has tests for video recording and checking video integrity, it passed https://github.com/SeleniumHQ/docker-selenium/actions/runs/9121853320
Beta Was this translation helpful? Give feedback.
All reactions
-
@alexandremariano4, I would suggest a thing, can you try and confirm
Remove this SE_VIDEO_FOLDER: "/cache" from environment. That means unset it
Correct this step in after_script cp -r /videos/video.mp4 ${CI_PROJECT_DIR}/evidency
Beta Was this translation helpful? Give feedback.
All reactions
-
I used the image provided (6.1.1), and also made a change.
I didn't know this before, but now I know that "/builds" is shared between all containers, so I used it to correctly capture the artifact and no need change nothing in runner config.toml , and the result I'm getting is unfortunately still the same.
stages: - "e2e" e2e: image: python stage: e2e cache: [] services: - name: selenium/standalone-chrome alias: browser - name: selenium/video:ffmpeg-6.1.1-20240517 alias: video variables: FF_NETWORK_PER_BUILD: "true" SE_VIDEO_FOLDER: "/builds" SE_VIDEO_FILE_NAME: 'chromevideo.mp4' DISPLAY_CONTAINER_NAME: browser before_script: - python3 -m venv .venv - source .venv/bin/activate - pip3 install poetry - poetry install --only main - cd project/ script: - python teste.py after_script: - mkdir -p ${CI_PROJECT_DIR}/evidency - mv -f /builds/chromevideo.mp4 ${CI_PROJECT_DIR}/evidency artifacts: when: always paths: - ${CI_PROJECT_DIR}/evidency expire_in: 1 week
Sempre o vídeo é gravado e mantém desta forma:
So, if I try for some reason to download it to run, the file simply doesn't open, which leads me to believe that the service (selenium/video) is not having time to correctly generate the video.
I did some research on something like:
SIGKILL/SIGTERM/SIGINT.
And from what I saw, it seems that when I send the SIGINT command which is composed of "kill -2 $PID", the container is closed as seen locally, reporting a "keyboard interrupt", but there is no way to send this command, so I imagine it would be otherwise.
Beta Was this translation helpful? Give feedback.
All reactions
-
Just for note, i don't know if this is relevant , my 'test.py' file where i run the script is like this
from selenium.webdriver import Remote, ChromeOptions from time import sleep browser_option = ChromeOptions() #Selenium Grid browser_option.page_load_strategy = 'eager' browser_option.set_capability('browserName','chrome') browser_option.set_capability('platformName','LINUX') brw = Remote( command_executor='http://browser:4444/wd/hub', options=browser_option ) brw.get('https://google.com.br') brw.maximize_window() print(brw.title) brw.quit() raise Exception
This code raises a Exception just for validate if his really saving a video with a condition that is a failure in my code
Beta Was this translation helpful? Give feedback.
All reactions
-
I had a similar issue in (selenium-video:ffmpeg-7.0.1-20240820) resulting in corrupted video. It seems that the problem is really with the shutdown of the video service container.
What I did as a workaround was to manually kill the video container inside the script step of my job, after my tests are finished:
variables: FF_NETWORK_PER_BUILD: 1 DISPLAY_CONTAINER_NAME: chrome SE_FRAME_RATE: 22 SE_VIDEO_RECORD_STANDALONE: true FILE_NAME: chrome_${CI_COMMIT_REF_SLUG}_${CI_COMMIT_SHORT_SHA}.mp4 VIDEO_FOLDER: /cache/videos services: - name: selenium/standalone-chrome:127.0-chromedriver-127.0-20240820 alias: chrome - name: selenium-video:ffmpeg-7.0.1-20240820_force-recording alias: chrome_video artifacts: paths: - ${CI_PROJECT_DIR}/${FILE_NAME} - /builds/sistemas/myproject/target/surefire-reports when: always script: - | # ... mvn $MAVEN_CLI_OPTS test || true # Stops the video container so the .mp4 is finished and writed to the disk VIDEO_SERVICE_CONTAINER_ID=$(docker ps -f "label=com.gitlab.gitlab-runner.job.id=${CI_JOB_ID}" -f name=selenium-video -q) docker stop ${VIDEO_SERVICE_CONTAINER_ID} && docker rm ${VIDEO_SERVICE_CONTAINER_ID} cp -v ${VIDEO_FOLDER}/${FILE_NAME} ${CI_PROJECT_DIR}/${FILE_NAME}
It works for now, but the video actually has a long blackscreen at the start, because the container starts to record as soon as it starts. So it's definitlly not a long term solution.
Beta Was this translation helpful? Give feedback.
All reactions
-
@pitervergara, can you try video image tag 4.24.0-20240830 to verify it can be resolved?
Beta Was this translation helpful? Give feedback.