fix: read wrong disk buffer cause high memory #2307
+141
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
||
| // if encodedInfoSize > BUFFER_DATA_MAX_SIZE, then read encodedInfoSize bytes from file, and validate fields | ||
| if (encodedInfoSize > BUFFER_DATA_MAX_SIZE) { | ||
| if (!PartialReadAndValidateKeyFields(fin, encodedInfoSize, pbMeta, filename)) { |
Collaborator
|
请确保flusher指针有效，请修复写入时的问题 |
Collaborator
|
我也是担心这种处理并不解决根本问题，比如可能修复了80%的可能，但是仍然不是最优方案 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
核心问题是这一段
SenderQueueItem是被DiskBuffer从发送队列clone保存的，但其中的flusher是一个指针。当force stop DiskBuffer的时候，flusher可能已经被释放了，但C++访问一个被释放的类的属性是未定义的，通常是可以获取的。但也没办法判断flusher是否已释放，所以只能在读取时拦截。
修复方案
问题一：超大，导致内存暴涨
通过protobuf流式读取。当size超大时，预读取前16KB，判断其中的元信息的几个字段是否超长，排除数据本身超大导致的错判。
问题二：aliuid长度合法，但内容错误
aliuid一定为纯数字组成。校验其每位是否为数字。