Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

How to deals with lot of test +300 #4210

Horsty80 started this conversation in General
Discussion options

I'm asking how you deals with a package of e2e test like +300 tests.

This is the actual amount of test that i maintain.
I have 2 approach on how to launch them.

First - All test in 8 workers - that we call all job & duration of ~15min
codeceptjs run-workers 8 --plugins allure

Second - 30 jenkins jobs that run only 10 tests with grep option - that we call split job & duration of ~2/3min per job
codeceptjs run-workers 2 --plugins allure --grep

Depending on which approach i use, i didn't get the same result and the same % of green run.

With the second split job i have a score of +95% of green run that is awesome.
No flaky, no failed job, no timeout, everything is ok.
The only issue is that all split job don't know the result of other so i can not get an aggregate result (except by go to the page and check visually all green marker)

With the first all job it's more complicated. I have ~10% of green run, all other are red or timeout.
The main issue is that the run are stuck i don't know why and then timeout.
Sometimes, 299 test are green and only 1 are red because browser has not starting
The guilty test is not the same and change everytime.
It's like, when i get more and more test, try to rening each in the same job cause issue. I don't know how to explain that.

Have you experimented same thing ?

You must be logged in to vote

Replies: 1 comment 1 reply

Comment options

Do they work fine, when executed with 1 worker?
As bad solution, can suggest https://codecept.io/basics/#retry-scenario

You must be logged in to vote
1 reply
Comment options

I will try tomorrow, but it will be very long run ^^ not the target

I have already add retry on Scénario / Feature / Before between 3 and 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants

AltStyle によって変換されたページ (->オリジナル) /