ICU-22934 clean up error handling in PR#3230

[FrankYFTang]

Checklist

• Required: Issue filed: https://unicode-org.atlassian.net/browse/ICU-22934
• Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
• Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Issue accepted (done by Technical Committee after discussion)
• Tests included, if applicable
• API docs and/or User Guide docs changed or added, if applicable

[aheninger]

Please add a unit test that triggers the recursion depth failures. It's not immediately obvious that there won't be memory leaks or other problems when they occur.

In functions with a newly added UErrorCode in PR #3230, double check the error handling. RBBINode::cloneTree(), for instance,

• at line 216, set a U_MEMORY_ALLOCATION_ERROR and return immediately if the new fails.
• make sure that you return a nullptr right away, and don't leak, in the event of a failure returned from the recursive calls at lines 220 and 226

In RBBINode::flattenSets(), maybe make the setRefNode variables into localPointers, so you can more easily return immediately if the cloneTree() calls set an error.

[jira-pull-request-webhook]

Notice: the branch changed across the force-push!

• icu4c/source/common/rbbinode.cpp is different
• icu4c/source/common/rbbiscan.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

[FrankYFTang]

Please add a unit test that triggers the recursion depth failures.

I have the problem to do that. The fuzzer found test case is > 400k in a file.

[jira-pull-request-webhook]

Notice: the branch changed across the force-push!

• icu4c/source/common/rbbinode.cpp is different
• icu4c/source/common/rbbiscan.cpp is different
• icu4c/source/common/rbbisetb.cpp is different
• icu4c/source/common/rbbitblb.cpp is different

View Diff Across Force-Push

~ Your Friendly Jira-GitHub PR Checker Bot

[aheninger]

Please add a unit test that triggers the recursion depth failures.

I have the problem to do that. The fuzzer found test case is > 400k in a file.

Hi Frank,
I've started to look at creating test cases, but have run into some behavior that I don't fully understand. I'll continue with it and get you something workable. Unfortunately I will be busy for most of the coming week, so it won't happen right away.

[FrankYFTang]

Please add a unit test that triggers the recursion depth failures.

I have the problem to do that. The fuzzer found test case is > 400k in a file.

Hi Frank, I've started to look at creating test cases, but have run into some behavior that I don't fully understand. I'll continue with it and get you something workable. Unfortunately I will be busy for most of the coming week, so it won't happen right away.

ping