-
-
Couldn't load subscription status.
- Fork 311
Implement fallback for systems without qsort_r #5931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
2b359ca to
601012d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be better if you can use matrix for fbsd versions (15.0, 14.3, and 14.2):
https://github.com/vmactions/freebsd-vm
4483b40 to
38da26d
Compare
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| freebsd-version: ['13.5', '14.3', '15.0'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Beginning with 14.3 will be sufficient, as 13.5 has reached EOL.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See my comment on #5800 (comment) though. FreeBSD switched flipped argument ordering between 13 and 14, so it may not be a bad idea to test both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, let's test 13.5.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| #ifndef HDqsort_r | ||
| #ifdef H5_HAVE_DARWIN | ||
| #ifdef H5_HAVE_QSORT_REENTRANT | ||
| #if defined(H5_HAVE_DARWIN) || (defined(__FreeBSD__) && __FreeBSD__ < 14) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the near future we should probably move this into something we try to check at configure time. I imagine there are other platforms that use this signature and checking at configure time should get rid of platform-specific #ifdefs and also avoid needing the (probably minor) overhead of a fallback function. Maybe something like https://gitlab.gnome.org/GNOME/glib/-/blob/2.30.0/configure.ac?ref_type=tags#L589-627.
|
What the PR does: |
src/H5system.c
Outdated
| qsort(base, nel, size, HDqsort_fallback_wrapper); | ||
|
|
||
| /* Clear the thread-local storage */ | ||
| H5TS_key_set_value(HDqsort_fallback_key, NULL); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HDF5 code always checks H5TS_key_set_value return values, for example:
src/H5TSrec_rwlock.c:
if (H5_UNLIKELY(H5TS_key_set_value(lock->rec_read_lock_count_key, (void *)count) < 0))
src/H5TSint.c:
if (H5_UNLIKELY(H5TS_key_set_value(H5TS_thrd_info_key_g, tinfo_node))) {
// Error handling
}
But
H5TS_key_create(&HDqsort_fallback_key, NULL); // No check
H5TS_key_set_value(HDqsort_fallback_key, &ctx); // No check
H5TS_key_set_value(HDqsort_fallback_key, NULL); // No check
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comments
|
src/H5system.c
Outdated
| /* Ensure the TLS key is initialized */ | ||
| ret = H5TS_once(&HDqsort_fallback_key_once, HDqsort_fallback_key_init); | ||
| if (H5_UNLIKELY(ret < 0)) { | ||
| assert(false && "Failed to initialize TLS key for qsort fallback"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As this is in main library code, we need to do better here than asserting on a condition that has the potential to occur in debug builds and simply ignoring errors in release builds. Since the signature of this function currently doesn't allow propagating of errors, either the interface needs reworked to support that, or we should just abandon this variant of the function, using the global variable approach only, and document the thread-safety issues until we actually need a thread-safe version.
.github/workflows/freebsd.yml
Outdated
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| freebsd-version: ['14.3', '15.0'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should add 13.5 back, as long as we plan to test that
|
|
||
| - name: Force qsort fallback by modifying ConfigureChecks.cmake | ||
| run: | | ||
| sed -i.bak '/CHECK_FUNCTION_EXISTS (qsort_r _HAVE_QSORT_R_TMP)/,/^endif ()$/{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with testing the fallback function, but it probably doesn't need a completely separate workflow run everywhere just to test something that won't be used in most places. Instead, I think this should probably be driven by testing on some platform we support or want/plan to support where we know the fallback function is going to be used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of comments; dealing with the lack of error handling in the fallback function is the most important IMO
src/H5system.c
Outdated
|
|
||
| /* Assert that initialization succeeded - cannot propagate errors from here */ | ||
| if (H5_UNLIKELY(ret < 0)) { | ||
| assert(false && "Failed to create TLS key for qsort fallback"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We must avoid assertions in main library code for cases that can realistically occur, even if that means ignoring errors (though ideally we don't do that either). In this case though, it looks like ret isn't assigned anyway, so this doesn't seem to be a useful check, unless ret should have been assigned from H5TS_key_create.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, I'd be much more in favor of simply dropping the thread-safe variant unless there's a good use case, and requiring qsort_r to exist for thread-safe builds, rather than bending over backwards a bit to add in error handling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right, ret should have been assigned from the key creation. I've removed this, since like the comment says if key creation fails, that'll be detected immediately afterwards in HDqsort_fallback().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, I'd be much more in favor of simply dropping the thread-safe variant unless there's a good use case, and requiring
qsort_rto exist for thread-safe builds, rather than bending over backwards a bit to add in error handling.
What exactly do you mean by requiring qsort_r to exist for threadsafe builds? I think that having the entire threadsafe build be impossible on systems that don't provide qsort_r directly seems too severe. As it stands, a threadsafe build should acquire the API lock on entry to the dataset routines that trigger qsort_r at tree create time anyway, so this should still be safe.
If the API lock isn't enough to handle the cases where the user wants a threadsafe build AND their system doesn't provide qsort_r, then I'd be more inclined to either fallback to not using the r-tree, or to acquire a special lock at tree creation time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's reasonable to require more modern features as the library progresses forward, but for the time being we could simply document the issues for now. For concurrency builds (as opposed to thread-safe builds), this may eventually become an issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case, the current commit should be ready, unless there's somewhere else that this should be documented.
Also includes a new workflow to test FreeBSD, which has a different qsort signature and was previously untested
Resolves #5896
Important
Implement
qsort_rfallback for unsupported systems and add FreeBSD CI workflow.HDqsort_fallback()inH5system.cfor systems withoutqsort_r/qsort_s, using thread-local storage or global variable.HDqsort_rmacro inH5private.hto use fallback ifqsort_r/qsort_sis unavailable.freebsd.ymlworkflow to test FreeBSD versions 13.5, 14.3, 15.0.test-qsort-fallback.ymlworkflow to testqsort_rfallback on Ubuntu, macOS, Windows.ConfigureChecks.cmaketo check forqsort_randqsort_sand setH5_HAVE_QSORT_REENTRANT.This description was created by
for c7ffe3a. You can customize this summary. It will automatically update as commits are pushed.