-
Notifications
You must be signed in to change notification settings - Fork 159
Description
Cypress Testing queries are recommended to use with Chai should assertions, I'm guessing because of the way the chained promises work (but very willing to be corrected!). E.g. from https://testing-library.com/docs/cypress-testing-library/intro/:
cy.findByRole('button', {name: /ButtonText/i}).should('exist')
When these findByRole queries fail in Cypress right now, you get standard error messages with the roles found/not found on screen.
Chai expect assertions allow you to supply a custom error message when expect assertions fail (https://www.chaijs.com/api/bdd/) but should doesn't give the same capability.
Correctly IMO, the way Testing Library queries work as above actively discourages the use of conditional testing, which aligns with Cypress guidance about it being generally a no-no.
However, I have come across a situation where we use Cypress tests for validations of deployments of our functionality to certain environments, and if an element does not exist on the page, I'd like to give some guidance to the tester about how to "reset" the state of the data so that we could try running the test again.
Here's a pretty contrived example - imagine we want to write a test on a food delivery website, and as part of that test, we rely on some dishes to exist on the site. If the dish doesn't exist, the roles that can be found on screen are outputted in the error. At that point, a tester can infer that the dish is missing by looking at the Test Replay and API requests etc (for those lucky enough to have Cypress Cloud) or the videos of the failure, and at that point try to figure out what went wrong with the data load.
If we had an extra errorMessage option, or something similar, we could put more active guidance in our tests for how to handle failures, e.g.:
cy.findByRole('link', {name: "Chicken Chow Mein", errorMessage: "Chicken Chow Mein does not exist on this menu. Please follow the guide at https://example.com to set up this dish and try to run the test again."}).should('not.exist')
My expectation here would be that the new errorMessage would be outputted in the error logs, pre-pended to the standard error messages from Testing Library.
Having custom error messages like this helps us to avoid conditional testing while also giving concrete guidance to the user running the test about how to proceed next in the case of failure.
Interested to hear some feedback here and if folks have any suggestions on how they've handled a scenario like this! Thank you!