-
Notifications
You must be signed in to change notification settings - Fork 3k
Add support for named Elasticsearch client (multi-tenant) #51422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add support for named Elasticsearch client (multi-tenant) #51422
Conversation
|
/cc @gsmet (elasticsearch,hibernate-search), @loicmathieu (elasticsearch) |
| or some Dev Services but not all of them, | ||
| you can disable this feature for a specific Dev Service by setting the configuration property | ||
| xref:elasticsearch-dev-services.adoc#quarkus-elasticsearch-rest-client_quarkus-elasticsearch-devservices_quarkus-elasticsearch-devservices-reuse[`quarkus.elasticsearch.devservices.reuse`] | ||
| xref:elasticsearch-dev-services.adoc#quarkus-elasticsearch-rest-client_quarkus-elasticsearch-devservices-reuse[`quarkus.elasticsearch.devservices.reuse`] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the achhors for config devservices properties changed here... but I'm not sure what exactly contributed to that 😕 , or if maybe it's ok to have the new anchor moving forward ?
| import io.smallrye.config.WithParentName; | ||
| import io.smallrye.config.WithUnnamedKey; | ||
|
|
||
| @ConfigMapping(prefix = "quarkus.elasticsearch-java") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since we didn't have the config properties for this extension before this ... just pointing your attention to the new prefix for the config group 🙂
| Set<DotName> clientTypeNames, Set<DotName> configAnnotationNames) { | ||
| Set<String> referencedNames = new HashSet<>(); | ||
| // Always start with the default: | ||
| referencedNames.add(ElasticsearchClientBeanUtil.DEFAULT_ELASTICSEARCH_CLIENT_NAME); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While we try to find injection points and collect the client names from the identifier qualifiers, I think that we should always add the default one. The idea here is that, if we discover the injection point for the default one -- then yeah it's redundant, but doesn't hurt, while if we don't discover it ... because it's only retrieved programmatically, or because it is only used in some other synthetic bean like a Java ES client... we'd be forcing user to add some build-time property for the default client to get things working. But I'd prefer if the default one just worked.
| /** | ||
| * This property is to help Quarkus identify any low-level Elasticsearch REST clients | ||
| * that possibly are not explicitly injected in the application code with their corresponding identifier, e.g. | ||
| * {@code @Inject @Identifier("client-name") RestClient client;}. | ||
| * In such cases, this property is required to help Quarkus detect all the necessary clients at build time. | ||
| * The value of this configuration property is ignored. | ||
| */ | ||
| @WithDefault("true") | ||
| boolean forceDiscovery(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note this build time property ... we only use it so that the name of the client is collected during the build time and we know that there may be such client used somewhere... since all of the config properties for these ES clients are runtime scoped, I couldn't think of a better alternative than having this "fake"
Telling users to explicitly set health.enabled just so that we can discover the client didn't sit well with me... 😖 🙂
| for (String name : ElasticsearchClientProcessorUtil.collectReferencedClientNames(indexBuildItem, registrationPhase, | ||
| Set.of(REST_CLIENT), | ||
| Set.of(ELASTICSEARCH_CLIENT_CONFIG_ANNOTATION))) { | ||
| references.produce(new ElasticsearchLowLevelClientReferenceBuildItem(name)); | ||
| clientNames.add(name); | ||
| } | ||
|
|
||
| // Because we may have not found some of the injections e.g. programmatic ones, | ||
| // or the ones we add in synthetic beans, so we say that there should be at least some build-time prop | ||
| // that we can work things out from: | ||
| for (String clientName : config.clients().keySet()) { | ||
| if (clientNames.add(clientName)) { | ||
| references.produce(new ElasticsearchLowLevelClientReferenceBuildItem(clientName)); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see this "mix": we try to find the client names from the injection points, but we also check for the build time properties, where the user can pass us those "fake" force-discovery properties for the clients they are using in a programmatic manner only.
| @Inject | ||
| @Identifier("another-es") | ||
| RestClient restClient; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you'll notice these "pointless" rest client injections in some of the Hibernate Search tests...
those are here to let dev services start an ES container for us to get away from the maven plugin.
with the following changes, once we add the way to pass the configured rest client to Hibernate Search these will go away and we would just configure Search with the correct clients instead.
This comment has been minimized.
This comment has been minimized.
0ed9528 to
de1bf08
Compare
Status for workflow
|
|
🎊 PR Preview e25d7c6 has been successfully built and deployed to https://quarkus-pr-main-51422-preview.surge.sh/version/main/guides/
|
Status for workflow
|
| Status | Name | Step | Failures | Logs | Raw logs | Build scan |
|---|---|---|---|---|---|---|
| ❌ | JVM Tests - JDK 17 | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | JVM Tests - JDK 21 | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | JVM Tests - JDK 21 Semeru | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | JVM Tests - JDK 25 | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | Native Tests - Data6 | Build |
Failures | Logs | Raw logs | 🔍 |
| ❌ | JVM Integration Tests - JDK 17 | Build |
Failures | Logs | Raw logs | 🚧 |
| ✔️ | JVM Integration Tests - JDK 17 Windows | Logs | Raw logs | 🚧 | ||
| ❌ | JVM Integration Tests - JDK 21 | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | JVM Integration Tests - JDK 21 Semeru | Build |
Failures | Logs | Raw logs | 🚧 |
| ❌ | JVM Integration Tests - JDK 25 | Build |
Failures | Logs | Raw logs | 🚧 |
Full information is available in the Build summary check run.
You can consult the Develocity build scans.
Failures
⚙️ JVM Tests - JDK 17 #
- Failing: extensions/hibernate-search-orm-outbox-polling/deployment
📦 extensions/hibernate-search-orm-outbox-polling/deployment
❌ io.quarkus.hibernate.search.orm.outboxpolling.test.configuration.ConfigPropertiesTest. - History - More details - Source on GitHub
java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:350)
at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:703)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: jakarta.persistence.PersistenceException: [PersistenceUnit: pu1] Unable to build Hibernate SessionFactory
⚙️ JVM Tests - JDK 21 #
- Failing: extensions/hibernate-search-orm-outbox-polling/deployment
📦 extensions/hibernate-search-orm-outbox-polling/deployment
❌ io.quarkus.hibernate.search.orm.outboxpolling.test.configuration.ConfigPropertiesTest. - History - More details - Source on GitHub
java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:350)
at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:703)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: jakarta.persistence.PersistenceException: [PersistenceUnit: pu1] Unable to build Hibernate SessionFactory
⚙️ JVM Tests - JDK 21 Semeru #
- Failing: extensions/hibernate-search-orm-outbox-polling/deployment
📦 extensions/hibernate-search-orm-outbox-polling/deployment
❌ io.quarkus.hibernate.search.orm.outboxpolling.test.configuration.ConfigPropertiesTest. - History - More details - Source on GitHub
java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at java.base/java.lang.reflect.Method.invoke(Method.java:586)
at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:350)
at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:703)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: jakarta.persistence.PersistenceException: [PersistenceUnit: pu1] Unable to build Hibernate SessionFactory
⚙️ JVM Tests - JDK 25 #
- Failing: extensions/hibernate-search-orm-outbox-polling/deployment
📦 extensions/hibernate-search-orm-outbox-polling/deployment
❌ io.quarkus.hibernate.search.orm.outboxpolling.test.configuration.ConfigPropertiesTest. - History - More details - Source on GitHub
java.lang.RuntimeException: Failed to start quarkus
at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
at io.quarkus.runtime.Application.start(Application.java:101)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:350)
at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:703)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
Caused by: jakarta.persistence.PersistenceException: [PersistenceUnit: pu1] Unable to build Hibernate SessionFactory
⚙️ Native Tests - Data6 #
- Failing: integration-tests/hibernate-search-orm-elasticsearch-tenancy
📦 integration-tests/hibernate-search-orm-elasticsearch-tenancy
❌ io.quarkus.it.hibernate.search.orm.elasticsearch.multitenancy.fruit.HibernateSearchTenancyFunctionalityInGraalITCase.test - History - More details - Source on GitHub
java.lang.RuntimeException: java.lang.IllegalStateException: Unable to determine the status of the running process. See the above logs for details
at io.quarkus.test.junit.QuarkusIntegrationTestExtension.throwBootFailureException(QuarkusIntegrationTestExtension.java:345)
at io.quarkus.test.junit.QuarkusIntegrationTestExtension.beforeEach(QuarkusIntegrationTestExtension.java:113)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.IllegalStateException: Unable to determine the status of the running process. See the above logs for details
at io.quarkus.test.common.LauncherUtil.waitForCapturedListeningData(LauncherUtil.java:86)
at io.quarkus.test.common.DefaultNativeImageLauncher.start(DefaultNativeImageLauncher.java:97)
⚙️ JVM Integration Tests - JDK 17 #
- Failing: integration-tests/hibernate-search-orm-elasticsearch
📦 integration-tests/hibernate-search-orm-elasticsearch
❌ Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test (default-test) on project quarkus-integration-test-hibernate-search-orm-elasticsearch:
See /home/runner/_work/quarkus/quarkus/integration-tests/hibernate-search-orm-elasticsearch/target/surefire-reports for the individual test results.
See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
There was an error in the forked process
⚙️ JVM Integration Tests - JDK 21 #
- Failing: integration-tests/hibernate-search-orm-elasticsearch
📦 integration-tests/hibernate-search-orm-elasticsearch
❌ Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test (default-test) on project quarkus-integration-test-hibernate-search-orm-elasticsearch:
See /home/runner/_work/quarkus/quarkus/integration-tests/hibernate-search-orm-elasticsearch/target/surefire-reports for the individual test results.
See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
There was an error in the forked process
⚙️ JVM Integration Tests - JDK 21 Semeru #
- Failing: integration-tests/hibernate-search-orm-elasticsearch
📦 integration-tests/hibernate-search-orm-elasticsearch
❌ Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test (default-test) on project quarkus-integration-test-hibernate-search-orm-elasticsearch:
See /home/runner/_work/quarkus/quarkus/integration-tests/hibernate-search-orm-elasticsearch/target/surefire-reports for the individual test results.
See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
There was an error in the forked process
⚙️ JVM Integration Tests - JDK 25 #
- Failing: integration-tests/hibernate-search-orm-elasticsearch
📦 integration-tests/hibernate-search-orm-elasticsearch
❌ Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test (default-test) on project quarkus-integration-test-hibernate-search-orm-elasticsearch:
See /home/runner/_work/quarkus/quarkus/integration-tests/hibernate-search-orm-elasticsearch/target/surefire-reports for the individual test results.
See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
There was an error in the forked process
Flaky tests - Develocity
⚙️ JVM Tests - JDK 17 Windows
📦 extensions/micrometer-opentelemetry/deployment
❌ io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod_AsyncFailed - History
Stream has no elements-java.lang.IllegalArgumentException
java.lang.IllegalArgumentException: Stream has no elements
at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lambda$lastReading$2(MetricDataFilter.java:213)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lastReading(MetricDataFilter.java:213)
at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lastReadingDataPoint(MetricDataFilter.java:231)
at io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod_AsyncFailed(MicrometerTimedInterceptorTest.java:150)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:532)
❌ io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod_UniFailed - History
Stream has no elements-java.lang.IllegalArgumentException
java.lang.IllegalArgumentException: Stream has no elements
at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lastReadingDataPoint(MetricDataFilter.java:236)
at io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod_UniFailed(MicrometerTimedInterceptorTest.java:202)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:532)
at io.quarkus.test.QuarkusUnitTest.interceptTestMethod(QuarkusUnitTest.java:446)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
⚙️ Maven Tests - JDK 21 Semeru
📦 integration-tests/devmode
❌ io.quarkus.test.devui.DevUIGrpcSmokeTest.testTestService - History
Too many recursions, message not returned for id [1817098938]-java.lang.RuntimeException
java.lang.RuntimeException: Too many recursions, message not returned for id [1817098938]
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:175)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
at io.quarkus.devui.tests.DevUIJsonRPCTest.objectResultFromJsonRPC(DevUIJsonRPCTest.java:178)
⚙️ MicroProfile TCKs Tests
📦 tcks/microprofile-lra
❌ org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsUnavailable - History
Expecting the metric Compensated callback was called Expected: a value equal to or greater than <1> but: <0> was less than <1>-java.lang.AssertionError
java.lang.AssertionError:
Expecting the metric Compensated callback was called
Expected: a value equal to or greater than <1>
but: <0> was less than <1>
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.eclipse.microprofile.lra.tck.TckRecoveryTests.assertMetricCallbackCalled(TckRecoveryTests.java:210)
at org.eclipse.microprofile.lra.tck.TckRecoveryTests.testCancelWhenParticipantIsUnavailable(TckRecoveryTests.java:195)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

This one
I'll add a few inline comments for things I'm not sure about... 🫣 🙂
Aaaand ... there'll be more changes once we get this + Hibernate Search 8.2 update in. In particular the combination of these two will allow us to: