Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AioZipStream does not support async generators #3

Open
mojimi opened this issue Aug 28, 2020 · 5 comments
Open

AioZipStream does not support async generators #3

mojimi opened this issue Aug 28, 2020 · 5 comments

Comments

@mojimi
Copy link

mojimi commented Aug 28, 2020

The following example does not work :

        async def files_generator():
            for filename in ['test1.txt', 'test2.txt', 'test3.txt']:
                yield {
                    'stream': await storage.get_file_stream({'path' : filename}, 5),
                    'name': filename
                }

        zs = AioZipStream(files_generator())
@kbbdy
Copy link
Owner

kbbdy commented Sep 23, 2020

Hi. I can't say what's wrong without knowing what is storage.get_file_stream method and what it returns. You also not added any error log, so I can't help. Remember that stream generator should return chunks of binary data.

@kingmad
Copy link

kingmad commented Sep 23, 2020

@kbbdy AioZipStream does not support async generator as an input. Changing for loop to async for in AioZipStream.stream#78 will probably do the job in this particular case

@kbbdy
Copy link
Owner

kbbdy commented Sep 23, 2020

Ok, thanks. I will check that.

@mojimi
Copy link
Author

mojimi commented Sep 23, 2020

Yes exactly what kingmad said, sorry I should've explained it better.

Basically by not working with an async generator it won't free up memory and/or the asyncio loop in some cases

@ChristianMeisenbichler
Copy link

ChristianMeisenbichler commented Oct 1, 2020

I would also like to use an async iterator for the source of the files list maybe implement a stream_async_iter method ? Something where the loop looping over the files (AioZipStream.stream#78) is async for?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants