ran testcases by multi-threads, then allure raised /usr/local/lib/python3.9/site-packages/allure_commons/reporter.py:33: KeyError

[ae86sen]

i ran testcases by multi-threads, then allure raised the error.

version:
python: 3.9.13
allure: 2.14.0
allure-pytest: 2.10.0
pytest: 7.1.2

[skhomuti]

@ae86sen could you please provide code examples and how exactly you run tests by multi-threads?

[ae86sen]

@ae86sen 你能不能提供代码例子，以及你到底是如何通过多线程运行测试的？

run.py

import pytest from concurrent.futures 
import ThreadPoolExecutor, wait, ALL_COMPLETED


def main_task(args: list): 
    ""pytest"" 
    pytest.main(args)

if __name__ == '__main__': 
    marks = ['-m hpc', '-m ehpc'] 
    thread_count = len(mark) tp = ThreadPoolExecutor(max_workers=thread_count)
    _ = [tp.submit(main_task, arg) for arg in marks] 
    wait(_, return_when=ALL_COMPLETED) 
    tp.shutdown()

testcase_demo.py

@pytest.mark.hpc 
def test_hpc(): 
    pass

@pytest.mark.ehpc
 def test_hpc(): 
    pass

It doesn't have to raised “keyerror” ,but i haven't found the necessary conditions,It seems to be related to the thread isolation of allure?

[ae86sen]

it seems to have something to do with the ”@allure.step“.
there is a new code example:
test_demo.py

import time

import allure
import pytest


@allure.title("hello")
@pytest.mark.d1
@allure.step('123')
def test_hpc():
    time.sleep(1)

@allure.title("hello")
@pytest.mark.d2
@allure.step('123')
def test_hpc2():
    time.sleep(1)

@allure.title("hello")
@pytest.mark.d3
@allure.step('123')
def test_hpc3():
    time.sleep(1)


@allure.title("hello")
@pytest.mark.d4
@allure.step('123')
def test_hpc4():
    time.sleep(1)

run.py

from concurrent.futures import ThreadPoolExecutor, wait, ALL_COMPLETED

import pytest


def main_task(args: str):
    params = ["-s", "--alluredir=report"]
    params.append(args)
    print(params)
    pytest.main(params)


def main():
    marks = ["-m d1", "-m d2", "-m d3", "-m d4"]
    tp = ThreadPoolExecutor(max_workers=len(marks))
    _ = [tp.submit(main_task, m) for m in marks]
    wait(_, return_when=ALL_COMPLETED)
    tp.shutdown()


if __name__ == '__main__':
    main()

when i ran run.py, raised the error:

[ae86sen]

i tried to run with v2.8.24 of allure-pytest, it's no problem.

[ae86sen]

[delatrie]

Hi, @ae86sen
Thanks for complete examples and highlights.

The thing is that we have two slightly different scenarios related to multithreading with allure:

1. Running tests concurrently in a multithreaded environment (your scenario)
2. Running steps concurrently inside a test (see Allure step in thread not show in report. #563)

The changes you are referring to were introduced to deal with (2).

I'll take a look on how to add support for (1). We also have issue with an async concurrency that needs to be addressed as well.

As for now, I advise you (if possible) to utilize xdist, i.e. to use parallel execution in separated processes instead of running tests concurrently in multiple threads. If you don't need to share a common state across your tests that should do the job.

[ae86sen]

thanks！Looking forward to your good news~