-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
/
Copy pathretry_failing_tests.feature
67 lines (55 loc) · 1.65 KB
/
retry_failing_tests.feature
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
Feature: Retry failing tests
Retry gives you a way to get through flaky tests that usually pass after a few runs.
This gives a development team a way forward other than disabling a valuable test.
- Specify max retry count in option
- Output information to the screen
- Output retry information in test report
Questions:
use a tag for flaky tests? Global option to retry any test that fails?
Background:
Given a scenario "Fails-once" that fails once, then passes
And a scenario "Fails-twice" that fails twice, then passes
And a scenario "Solid" that passes
And a scenario "Fails-forever" that fails
Scenario: Retry once, so Fails-once starts to pass
When I run `cucumber -q --retry 1 --format summary`
Then it should fail with:
"""
7 scenarios (5 failed, 2 passed)
"""
And it should fail with:
"""
Fails-forever
Fails-forever ✗
Fails-forever ✗
Solid
Solid ✓
Fails-once feature
Fails-once ✗
Fails-once ✓
Fails-twice feature
Fails-twice ✗
Fails-twice ✗
"""
Scenario: Retry twice, so Fails-twice starts to pass too
When I run `cucumber -q --retry 2 --format summary`
Then it should fail with:
"""
9 scenarios (6 failed, 3 passed)
"""
And it should fail with:
"""
Fails-forever
Fails-forever ✗
Fails-forever ✗
Fails-forever ✗
Solid
Solid ✓
Fails-once feature
Fails-once ✗
Fails-once ✓
Fails-twice feature
Fails-twice ✗
Fails-twice ✗
Fails-twice ✓
"""