Skip to content

Conversation

@apuig
Copy link

@apuig apuig commented Dec 8, 2025

Attempt on optimizing the GitLab Docker image, increased timeouts, and incorporated defensive retries. Wherever possible, use Git directly instead of GitLabApi.

Thanks to @raul-arabaolaza and others for their previous work in #1425

Aim to fix:
#1461
#1365

Testing done

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests that demonstrate the feature works or the issue is fixed

@apuig
Copy link
Author

apuig commented Dec 12, 2025

I'm marking this as ready for review to gather feedback.
The build is consistently passing on ci.jenkins and the test is robust with proper awaits/retries.

However, there are some limitations to acknowledge:

  • Configuration hell: The Dockerfile uses aggressive optimization settings to minimize startup time. These may be fragile and could break with future GitLab updates.

  • Performance impact: Even with optimizations, the GitLab image bundles all required services (Redis, PostgreSQL, Nginx, Rails app with multiple processes), resulting in slow and resource-intensive startup (~2m locally / ~8m on CI).

    This positions GitLabPluginTest as the 2nd most expensive test in the suite at ~37 minutes total runtime, behind only JobDslPluginTest (~51 min)

UPDATE: reusing the container ~23 minutes

@apuig apuig marked this pull request as ready for review December 12, 2025 18:08
@apuig apuig requested a review from a team as a code owner December 12, 2025 18:08
@apuig apuig changed the title Testing re-enable GitLab Re-enable GitLabPluginTest Dec 12, 2025
Copy link
Member

@jglick jglick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not familiar with the original, but looks reasonable.

@timja
Copy link
Member

timja commented Dec 12, 2025

  • Performance impact: Even with optimizations, the GitLab image bundles all required services (Redis, PostgreSQL, Nginx, Rails app with multiple processes), resulting in slow and resource-intensive startup (~2m locally / ~8m on CI).

    This positions GitLabPluginTest as the 2nd most expensive test in the suite at ~37 minutes total runtime, behind only JobDslPluginTest (~51 min)

Is it worth keeping this test in here in this case? that seems really expensive to run...

@jglick
Copy link
Member

jglick commented Dec 12, 2025

We can always decide to disable it by default in CI, while keeping the option to run it on demand to check for regressions when some are suspected.

Copy link
Member

@jtnord jtnord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub won't let me post any review comments but there is at least issue of double time elasticity (don't use elastic time in a wait for) and there appear to be stream leaks (due to unclosed streams).

The idiom Jesse was looking for to ignore exceptions in a wait is .ignoring(Exception..). Using it makes the code both easier to follow and if the timeout fails and the last call resulting an exception you get told about it

I will try adding inline comments again next week, hopefully GH will have recovered.

- prefer duration
- use `.ignoring` idiom to improve debugability
- avoid direct elastic time usage in `Wait`
- visit the job page to verify its existence
- use logger
- remove container cleanup
- improve readability
@apuig
Copy link
Author

apuig commented Dec 19, 2025

that seems really expensive to run...

Last run (reusing the container), finished in 23 minutes, with the GitLab image ready in ~4 minutes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants