[CI] Enable "Rerun Failed Tests" for Small Tests #4586
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4586 +/- ##
==========================================
- Coverage 85.95% 83.42% -2.54%
==========================================
Files 565 565
Lines 33930 33868 -62
==========================================
- Hits 29166 28254 -912
- Misses 4764 5614 +850 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
tomaszwojcikowski
force-pushed
the
tomasz/MIM-2364-rerun-flaky-tests
branch
from
631e9c9 to
1c3de3d
Compare
tomaszwojcikowski
changed the title
tomaszwojcikowski
changed the title
|
dynamic_domains_mssql_mnesia_28 / odbc_mssql_mnesia / a560aaf elasticsearch_and_cassandra_28 / elasticsearch_and_cassandra_mnesia / a560aaf small_tests_28 / small_tests / a560aaf small_tests_28_arm64 / small_tests / a560aaf small_tests_27 / small_tests / a560aaf ldap_mnesia_28 / ldap_mnesia / a560aaf dynamic_domains_mysql_redis_28 / mysql_redis / a560aaf ldap_mnesia_27 / ldap_mnesia / a560aaf internal_mnesia_28 / internal_mnesia / a560aaf dynamic_domains_pgsql_mnesia_28 / pgsql_mnesia / a560aaf pgsql_cets_28 / pgsql_cets / a560aaf dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / a560aaf mysql_redis_28 / mysql_redis / a560aaf cockroachdb_cets_28 / cockroachdb_cets / a560aaf pgsql_mnesia_28 / pgsql_mnesia / a560aaf mssql_mnesia_28 / odbc_mssql_mnesia / a560aaf pgsql_mnesia_27 / pgsql_mnesia / a560aaf dynamic_domains_mssql_mnesia_28 / odbc_mssql_mnesia / a560aaf |
|
elasticsearch_and_cassandra_28 / elasticsearch_and_cassandra_mnesia / 0638a02 small_tests_27 / small_tests / 0638a02 small_tests_28 / small_tests / 0638a02 small_tests_28_arm64 / small_tests / 0638a02 ldap_mnesia_28 / ldap_mnesia / 0638a02 ldap_mnesia_27 / ldap_mnesia / 0638a02 graphql_server_SUITE:admin_cli:clustering_tests:join_successful{error,{#{what => invalid_response_code,
response_code => {exit_status,1},
expected_type => ok},
[{graphql_helper,assert_response_code,2,
[{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
{line,258}]},
{graphql_helper,get_ok_value,2,
[{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
{line,241}]},
{graphql_server_SUITE,join_successful,1,
[{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
{line,153}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1303}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1235}]}]}}dynamic_domains_mssql_mnesia_28 / odbc_mssql_mnesia / 0638a02 dynamic_domains_mysql_redis_28 / mysql_redis / 0638a02 dynamic_domains_pgsql_mnesia_28 / pgsql_mnesia / 0638a02 dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / 0638a02 internal_mnesia_28 / internal_mnesia / 0638a02 pgsql_cets_28 / pgsql_cets / 0638a02 pgsql_mnesia_27 / pgsql_mnesia / 0638a02 mysql_redis_28 / mysql_redis / 0638a02 cockroachdb_cets_28 / cockroachdb_cets / 0638a02 pubsub_SUITE:dag+collection_config:disable_persist_items_leaf_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_disable_persist_items_leaf_test_3687@localhost/res1">>,
escalus_tcp,<0.115350.0>,
[{event_manager,<0.115348.0>},
{server,<<"localhost">>},
{username,<<"alicE_disable_persist_items_leaf_test_3687">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.115348.0>},
{server,<<"localhost">>},
{username,
<<"alicE_disable_persist_items_leaf_test_3687">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,<<"alice_disable_persist_items_leaf_test_3687">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,<<"alicE_disable_persist_items_leaf_test_3687">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"8821e3d795e972fe">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,444}]},
{pubsub_tools,receive_and_check_response,4,
[{file,"/home/circleci/project/big_tests/tests/pubsub...pubsub_SUITE:tree+last_item_cache:send_last_published_item_no_items_test{error,
{timeout_when_waiting_for_stanza,
[{escalus_client,wait_for_stanza,
[{client,
<<"alice_send_last_published_item_no_items_test_3749@localhost/res1">>,
escalus_tcp,<0.116807.0>,
[{event_manager,<0.116804.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3749">>},
{resource,<<"res1">>}],
[{event_client,
[{event_manager,<0.116804.0>},
{server,<<"localhost">>},
{username,
<<"alicE_send_last_published_item_no_items_test_3749">>},
{resource,<<"res1">>}]},
{resource,<<"res1">>},
{username,
<<"alice_send_last_published_item_no_items_test_3749">>},
{server,<<"localhost">>},
{host,<<"localhost">>},
{port,5222},
{auth,fun escalus_auth:auth_plain/2},
{wspath,undefined},
{username,
<<"alicE_send_last_published_item_no_items_test_3749">>},
{server,<<"localhost">>},
{password,<<"matygrysa">>},
{stream_id,<<"320a9fcc46b4e29b">>}]},
5000],
[{file,
"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
{line,136}]},
{pubsub_tools,receive_response,3,
[{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
{line,444}]},
{pubsub_tools,receive_and_c...mssql_mnesia_28 / odbc_mssql_mnesia / 0638a02 pgsql_mnesia_28 / pgsql_mnesia / 0638a02 ldap_mnesia_27 / ldap_mnesia / 0638a02 |
tomaszwojcikowski
force-pushed
the
tomasz/MIM-2364-rerun-flaky-tests
branch
from
0638a02 to
a560aaf
Compare
|
Would it be possible for you to address these few items?
|
tomaszwojcikowski
changed the title
|
chrzaszcz
left a comment
chrzaszcz
left a comment
There was a problem hiding this comment.
Looks good. I added one comment.
| fi | ||
| - store_test_results: | ||
| when: always | ||
| path: small_tests_test_results |
There was a problem hiding this comment.
Why do we use inconsistent names?
- For small tests:
small_tests_test_results - For big tests:
big_tests_junit_report.xml
Could we make this at least a bit more consistent?
This PR enables CircleCI's "Rerun failed tests" for small tests.