-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Devservices Keycloak breaks integration tests #45793
Comments
/cc @geoand (devservices), @pedroigor (keycloak), @sberyozkin (keycloak), @stuartwdouglas (devservices) |
@constantin-ungureanu-github Can you please create a basic reproducer ? There are a few integration tests running in Quarkus with the Keycloak devservices |
https://github.com/constantin-ungureanu-github/quarkus-oidc-example Let me know if you need anything more. |
Thanks @constantin-ungureanu-github, that is helpful, it is a shared container case. #42509 related to it but as it happens, #42509 is currently blocked, hopefully not for too long. |
I wonder if the shared container is the root cause. It fails regardless if it is shared or not (quarkus.keycloak.devservices.shared=true/false). The cause seems to be that when running IT tests, it sets the '--hostname-port' that with Keycloak version 26+ is no longer available (has been removed completely), and that Keycloak container fails to start. However, a more serious issue, that it seems it is a different problem, is that the management port and quarkus-oidc don't work together. This is not regarding only the tests, but the resulting container fails to start in a Kubernetes environment where the health probes are used. Also tested to directly access the health checks when the management port is used, they just don't work anymore. The sample code provided is built in this way and can be verified. Please have these issues fixed. Let me know if you need more details. |
I made some updates to the repository, to add running capabilities. Just run docker compose up or podman compose up and the container should pop-up, alongside a preconfigured Keycloak. I found the root cause for the management port not working, that is only when "quarkus.http.non-application-root-path=/". The default is /q and I wanted to remove that altogether. Commenting out this change and I have back the metrics and the health checks. Now the "quarkus.http.non-application-root-path=/" works fine without using quarkus-oidc, so the issue resides in the combination of using all those in this configuration. Please fix all these issues. |
Hi @constantin-ungureanu-github I look at your reproducer For Keycloak issue I come up with some solution. Will discribe them later in the text. For liveness check. Why test are not passing is because the quarkus test framework not exposing the management port. As workaround you can use When running your docker compose I'm able to see http://localhost:9000/q/health/live.
There are two option and I'm not sure which is correct so here is needed input of @sberyozkin as he have much more knowledge about how thing work with Quarkus. Also I didn't encounter the blocking issue as mentioned in #42509 as blocking issue seems only affecting windows not linux. 1st option is to replace The keycloak command would be
Maybe there is some other option. I come with these option when playing with it. Here is a command which is executed when the integration test are executed
I checked with 3.8 which didn't have hostname v2 by default and the We cant just remove the
|
Hi @jedla97 ! I updated the repository https://github.com/constantin-ungureanu-github/quarkus-oidc-example as per your suggestions. Instead of using quarkus.http.non-application-root-path I use quarkus.management.root-path and it is indeed working as expected !! For the original issue, of the integration tests failing due to usage of quarkus-oidc, you guys for sure know better. I hope to see this fixed soon. Thanks for your replies! |
Hi @jedla97, many thanks for your investigation... What I do recall is that when Awhile back we had a dedicated integration test, created by @geoand, I was periodically doing some minor tweaks to get it passing, and then later it was removed due to some cleanup, and unfortunately I never got to restoring it, as far as I recall @iocanel was suggesting we did not need to have an image generated to test this shared network case... Would you like to have a look at restoring that test in the next couple of weeks and CC @geoand, @iocanel and myself whenever you have it working ? |
Hi @sberyozkin I can look at restoring them. Thanks for the links! For this issue are we want to provide the fix now so we have at least working solution (maybe not for all case but at least fix the startup of Keycloak) or are we want fixt it with test? |
I don't think that the removed tests are related this issue. The removed tests were maven invoker tests, which pretty much tested the image building. If the removed modules included Quarkus integration tests that we need, then we could propably restore them, but please not as invoker tests. |
@jedla97 Thanks, my understanding that restoring the actual integration test would require your suggested 2nd option to get this test passing on @iocanel The removed tests were testing the shared network case related to this issue. So, given that those tests need that image which was built during the maven invoker run, can you recommend how to setup the test so that the shared network case is tested ? |
It now seems that with quarkus 3.18.x a keycloak 26.x.x image is required for the dev service (otherwise my application fails to start while testing)
With a keycloak image 26.x.x, my test run through, but then my shared network tests fail again. I'll try to poke around a little bit today, but it certainly would be appreciated if someone with expertise would have a look, as it is most likely a simple config issue. |
@sberyozkin @constantin-ungureanu-github I found that this works for me: #46141 A keycloak expert should probably verify that 🙂 What is still missing is a test that is run with (gradle) |
My understanding is that the problem was fixed with #46141 ? Should we close this issue? |
@constantin-ungureanu-github should be fixed with 3.18.3. Can you confirm? |
Hi everybody, Line 532 in 58ff745
KEYCLOAK_QUARKUS_HOSTNAME is missing the port, by doing so the issuer is always wrong in tokens generated during integration tests :
The correct issuer should be (in my case) http://keycloak-441zi:8080/realms/quarkus Of course every IT is failing for this ... help ? |
Also, this is confimed by documentation : https://quarkus.io/guides/security-jwt#integration-testing-security-annotation I expect the correct issue value to be ${keycloak.url}/realms/quarkus, that in my case is http://keycloak-441zi:8080/realms/quarkus |
Sorry I forgot to mention this happens in the case of required shared network ... |
@amoscatelli I'll have a look, I did work for me without any issues in a shared network. Do you authenticate within the backend? |
@HerrDerb thank you. During my tests I am able to obtain a JWT from Keycloak using KeycloakTestClient (for example the JWT I pasted above). But the JWT is invalid for the quarkus microservice because the issuer mismatches. How are you setting the mp.jwt.verify.issuer property in your tests ? |
Example code :
|
Probably the 2nd option what I mentioned will be needed insted of setting
I'll check it later today |
Hi @HerrDerb and @amoscatelli I think it should work so I created the #46710 I checked it with reproducer what @constantin-ungureanu-github provided and the test are passing. Can you try this patch and check if it work on your side? Also @amoscatelli I checked with your example code which you provided but don't know what happening in |
@jedla97 thank you, it seems ok to me : |
@jedla97 tested an seems to work for me too. I am actually wondering and a bit confused as I thought I would have tried that too 😅 anyway, it works 😀👍 |
Thanks for working to solve this issue. I made some updates to the code to be able to use in the tests a different realm than the default one, but the keycloak client is always trying to use the client "quarkus-app", that was not in my realm and I couldn't find a proper way to specify the name of the client. Not sure if there is one, specifying the service name quarkus.keycloak.devservices.service-name doesn't seem to make the keycloak client to use the client with this name. Not a big deal, since I could modify in the realm used for testing the name of the client to be exactly "quarkus-app", but probably devservices keycloak client should have this option to specify the client and secret to use for the custom realm. However, when I try to build also with native mode enabled it hangs in the native integration tests: 2025-03-11 16:42:36,558 INFO [io.qua.dev.key.KeycloakDevServicesProcessor] (build-6) Dev Services for Keycloak started. I suspected that the issue is that I'm trying to use a custom realm, so alternative is to only use the default configuration. So reverting the usage of custom realm (commenting out #quarkus.keycloak.devservices.realm-path=quarkus-realm.json) but the error still persists. Commenting all additional settings to keycloak and the error still persists. One additional detail is that I have activated the swagger/openapi, maybe this has an influence overall. An additional, more important issue I have observed in the past (can't reproduce it here since the native IT tests fail before) is that when I want to use random management port in tests, it is always 0. |
Hi, @constantin-ungureanu-github looking at this and the problem seems to be that your dockerfiles not use ubi9 images. for
I try it with this and it seems to work except that management endpoint whenruning integration tests ( Also I check if the quarkus update will update the docker files and they still same. I'll create issues for both of these problems. But in meantime can you try update docker files (https://github.com/constantin-ungureanu-github/quarkus-oidc-example/tree/main/src/main/docker) as I mention above |
Thanks @jedla97 for the update. The keycloak issues indeed seems to have been fixed. The random management test port issue is now reproducible, I even put a log to see the URL I'm getting for the live probe when trying to use a random management port. I cannot use the default management test port 9001 because this specific port it is blocked on my work machine, so a random port would be exactly fitted for this. This issue with the random management port on native integration tests it would be nice if it is fixed for the 3.20 LTS version. [INFO] Running org.acme.LivenessCheckIT ... |
@constantin-ungureanu-github could you clarify how we can reproduce the issue ^. Ideally having a reproducer we can start would help. |
To reproduce the issue of the random managitest port being always 0 in
native integration test just run the build native as described in the
readme with the latest source code. the repository is available in initial
comments.
…On Tue, Mar 11, 2025, 18:26 Guillaume Smet ***@***.***> wrote:
@constantin-ungureanu-github
<https://github.com/constantin-ungureanu-github> could you clarify how we
can reproduce the issue ^. Ideally having a reproducer we can start would
help.
—
Reply to this email directly, view it on GitHub
<#45793 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB7K56KOGYMEXXZBEQUIG4L2T4MENAVCNFSM6AAAAABVU6R3D6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJVGE2DSNJRHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
[image: gsmet]*gsmet* left a comment (quarkusio/quarkus#45793)
<#45793 (comment)>
@constantin-ungureanu-github
<https://github.com/constantin-ungureanu-github> could you clarify how we
can reproduce the issue ^. Ideally having a reproducer we can start would
help.
—
Reply to this email directly, view it on GitHub
<#45793 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB7K56KOGYMEXXZBEQUIG4L2T4MENAVCNFSM6AAAAABVU6R3D6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMJVGE2DSNJRHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@constantin-ungureanu-github @gsmet I create the issue about it #46737 I attached the @constantin-ungureanu-github reproducer and add simple mine. |
Describe the bug
With the quarkus-oidc in place, the restriction to the REST API are added.
The @QuarkusTest run okay. all of them.
The @QuarkusIntegrationTest do no run okay.
So adding quarkus-oidc to a quarkus project, adding few REST API restrictions (as in security-keycloak-authorization-quickstart demo), and @QuarkusTest runs fine, both with/withouth oidc restrictions, also healthchecks.
Same tests run as @quarkusIntegrationTest fail. (build native but is not related to the build as I understand)
The difference observed is that --hostname-port is sent when running the integration tests. Please remove this because was deprecated (25) and with keycloak 26+ was removed and now the keycloak is failing to start.
Workaround found for this was to:
quarkus.keycloak.devservices.service-name=keycloak
quarkus.keycloak.devservices.start-command=start --http-enabled=true --hostname=keycloak --hostname-strict=false --spi-user-profile-declarative-user-profile-config-file=/opt/keycloak/upconfig.json --features=hostname:v1
However, this works only with quarkus less than 3.18 (CR1), and it fails still in case of healthchecks (integration tests only). Sure, the keyloack can be downgraded but that still doesn't completely fix the issues.
For keycloak the v1 is replaced with v2 (--features=hostname:v2) . However, that does not work anymore, container fails to start.
The 3.17.7 with the workaround fails for the API that don't include oidc restrictions and also for the healthcheck IT tests.
This is what I get from the keycloak container (when running IT tests that fail):
Changes detected in configuration. Updating the server image.
Updating the configuration and installing your custom providers, if any. Please wait.
2025-01-22 12:56:41,951 INFO [io.qua.dep.QuarkusAugmentor] (main) Quarkus augmentation completed in 6640ms
Server configuration updated and persisted. Run the following command to review the configuration:
Next time you run the server, just run:
� kc.sh start --http-enabled=true --hostname-debug=true --hostname=localhost --spi-user-profile-declarative-user-profile-config-file=/opt/keycloak/upconfig.json --hostname-port=53483 --optimized
Unknown option: '--hostname-port'
Possible solutions: --hostname, --hostname-admin, --hostname-backchannel-dynamic, --hostname-strict, --hostname-debug
Try 'kc.sh start --help' for more information on the available options.
Expected behavior
All test run nicely.
Actual behavior
Integration tests fails
I see there is a difference that the --hostname-port option has been removed
How to Reproduce?
Run integration tests involving devservices keycloak and this pops up immediately.
Also test this when endpoints are healtchecks and there are no restrictions and when the management port is used (with healthchecks on that port).
Output of
uname -a
orver
No response
Output of
java -version
java 21
Quarkus version or git rev
3.18 CR1 and also lower (3.17)
Build tool (ie. output of
mvnw --version
orgradlew --version
)maven 3.9.9
Additional information
I see for the version 3.18 there were numerous improvements related to OIDC, so would be nice to have the fix for quarkus-oidc running IT tests in 3.18. If needed, more details can be provided, but if you just run IT tests with OIDC this should pop-up.
The text was updated successfully, but these errors were encountered: