Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Zlib inputs left hanging at the middle of a block #2514

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

etcimon
Copy link
Contributor

@etcimon etcimon commented Jan 5, 2021

Handle inputs left hanging at the middle of a block

Handle inputs left hanging at the middle of a block
@s-ludwig
Copy link
Member

Wow, so this is really a case that needs to be handled? So many small fixes were necessary already here, because the zlib API is so brittle.

Based on the documentation, it seems like the input should always be fully drained, as long as enough output space is provided:

The detailed semantics are as follows. inflate performs one or both of the following actions:

  • Decompress more input starting at next_in and update next_in and avail_in accordingly. If not all input can be processed (because there is not enough room in the output buffer), then next_in and avail_in are updated accordingly, and processing will resume at this point for the next call of inflate().
  • Generate more output starting at next_out and update next_out and avail_out accordingly. inflate() provides as much output as possible, until there is no more input data or no more space in the output buffer (see below about the flush parameter).

...or at least not having enough output space is the only reason given. So if this indeed needs to be done anyway, I would suggest to only do it if there is still something in the input buffer and inflate did not write anything to the output buffer - just to avoid adding unnecessary memmove overhead in the regular cases.

@s-ludwig
Copy link
Member

So I looked at this some more and tested with different kinds of inputs (different levels of compressibility), as well as input streams that only yielded one byte at a time. I couldn't make the existing code fail that way, unfortunately.

Do you remember where it failed for you? Did it fail an assertion (e.g. assert(m_zstream.avail_out != m_outbuffer.peekDst.length || m_zstream.avail_in != avins)), or did it hang in an infinite loop?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants