[Bug] Occasionally encountering IllegalReferenceCountException when releasing ShuffleIndexResult #1628
Closed
2 of 3 tasks
Assignees
Labels
help wanted
Extra attention is needed
Comments
|
@EnricoMi Could you help to find out the reason here? I still have no clue. I've encountered this issue again this time: https://github.com/apache/incubator-uniffle/actions/runs/9486586733/job/26141279578?pr=1780 |
|
Maybe duplicate of #1818 |
|
I think they are different. This is caused by calling |
Sorry, it looks like the error stack is different |
|
Could you help to look at this issue too? |
Sure, I will try to fix it later. |
|
|
|
I think it is feasible, you can do it. Although setting the internal buffer to null might have a positive impact on GC, this impact is usually minimal. |
|
After adding the following log in NettyManagedBuffer, I got some related logs from https://github.com/wForget/incubator-uniffle/actions/runs/9676598458 and did not find any repeated calls.
|
|
Could you add some logs in |
The |
|
If you are sure that it is Because we use Could you please add some logs in |
|
I seem to have found two threads that repeatedly release it: |
|
I didn't see the connection between these two release stacktrace logs, as they seem to be unrelated. The first stack contains |
They seem to be the same object. |
|
Could we temporarily handle it in the following way? public static void releaseByteBuffer(ByteBuffer byteBuffer) {
if (byteBuffer == null || !byteBuffer.isDirect()) {
return;
}
try {
PlatformDependent.freeDirectBuffer(byteBuffer);
} catch (Throwable t) {
LOGGER.warn("Failed to release direct buffer:" + t.getMessage());
}
} |
3 tasks
maobaolong
added a commit
to maobaolong/incubator-uniffle
that referenced
this issue
Nov 4, 2024
…se exception to pass flaky test (apache#2179) ### What changes were proposed in this pull request? (Please outline the changes and how this PR fixes the issue.) ### Why are the changes needed? - Pass the flaky test - Avoid memory leak after double release the `ShuffleIndexResult` Fix: apache#1628 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Flaky test can be fixed. (cherry picked from commit 4c14b97)
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Code of Conduct
Search before asking
Describe the bug
https://github.com/apache/incubator-uniffle/actions/runs/9282054580/job/25539265104?pr=1752
org.apache.uniffle.test.RepartitionWithHadoopHybridStorageRssTest
Error stacktrace:
Reason: Unknown
Priority: Minor
Affects Version(s)
master
Uniffle Server Log Output
No response
Uniffle Engine Log Output
No response
Uniffle Server Configurations
No response
Uniffle Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: