Currently when a single test in it() block fails Cypress halts pletely.
I want Cypress to continue running subsequent assertions within the test, regardless if a previous assertion failed or not (but I still want to mark the failed tests so I know which one failed).
I tried to intercept the fail event in beforeEach:
beforeEach(() => {
Cypress.on('fail', (error, runnable) => {
cy.log('this single test failed, but continue other tests');
// don't stop!
// throw error; // marks test as failed but also makes Cypress stop
});
But it appears I cannot use any cy mands inside this handler because when I do it returns an error due to Cypress weird internal promise logic:
CypressError: Cypress detected that you returned a promise from a mand while also invoking one or more cy mands in that promise.
The mand that returned the promise was:
cy.wait()
The cy mand you invoked inside the promise was:
cy.log()
Because Cypress mands are already promise-like, you don't need to wrap them or return your own promise.
Cypress will resolve your mand with whatever the final Cypress mand yields.
The reason this is an error instead of a warning is because Cypress internally queues mands serially whereas Promises execute as soon as they are invoked. Attempting to reconcile this would prevent Cypress from ever resolving.
If I leave the Cypress.on('fail') block empty all tests are going to be marked as passed even if they fail.
If I unment throw error Cypress will halt pletely on any failed test.
Currently when a single test in it() block fails Cypress halts pletely.
I want Cypress to continue running subsequent assertions within the test, regardless if a previous assertion failed or not (but I still want to mark the failed tests so I know which one failed).
I tried to intercept the fail event in beforeEach:
beforeEach(() => {
Cypress.on('fail', (error, runnable) => {
cy.log('this single test failed, but continue other tests');
// don't stop!
// throw error; // marks test as failed but also makes Cypress stop
});
But it appears I cannot use any cy mands inside this handler because when I do it returns an error due to Cypress weird internal promise logic:
CypressError: Cypress detected that you returned a promise from a mand while also invoking one or more cy mands in that promise.
The mand that returned the promise was:
cy.wait()
The cy mand you invoked inside the promise was:
cy.log()
Because Cypress mands are already promise-like, you don't need to wrap them or return your own promise.
Cypress will resolve your mand with whatever the final Cypress mand yields.
The reason this is an error instead of a warning is because Cypress internally queues mands serially whereas Promises execute as soon as they are invoked. Attempting to reconcile this would prevent Cypress from ever resolving.
https://on.cypress.io/returning-promise-and-mands-in-another-mand
If I leave the Cypress.on('fail') block empty all tests are going to be marked as passed even if they fail.
If I unment throw error Cypress will halt pletely on any failed test.
- 2 Which version are you using? Cypress doesn't fail-early (and never did AFAIK), so this behavior you're describing is weird. Can you share a reproducible repo? – dwelle Commented Nov 1, 2019 at 11:41
3 Answers
Reset to default 2My way to ensure subsequent tests are done, and the failed test is marked as failure is - to put every it case in different file, and if needed to group them - I group them in a separate subfolder.
It has improved readability of reports and cypress tests run time, since before that I sometimes had problems with cypress not clearing its state between tests and we had memory leaks.
If you throw error in your test it will halt the script in the same way it does when there is an issue with the code. This is functioning as expected. You may need to revisit your test logic and consider adding some stubs/spies that wouldn't set the exit code to 1.
There is a lack of this feature(see this issue:https://github./cypress-io/cypress/issues/518). If you read through it you will find some code snippets that mention throwing an error to stop the test runner. You are doing this with the opposite intent.
If you must throw the error and NOT have the test runner bail, you need to catch it.
Is there a reason why don't you want to place tests in the different it() blocks?
If there is not, then definitely do it. Makes reports much more readable and ensures clean state before each test.
If you need to persist the application state between it()'s, consider using cy.session() (if your cypress version is new enough). Otherwise, there are some methods for getting, saving, and later using cookies. There are different options, so if you need further help, please describe your issue more.
Some useful links: cy.session() cy.getCookie()