fix: flyteadmin doesn't shutdown servers gracefully #6289
Conversation
Signed-off-by: Ryan Lo <[email protected]>
Signed-off-by: Ryan Lo <[email protected]>
Signed-off-by: Ryan Lo <[email protected]>
Code Review Agent Run Status
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6289 +/- ##
==========================================
+ Coverage 50.37% 58.46% +8.09%
==========================================
Files 1170 937 -233
Lines 92851 71121 -21730
==========================================
- Hits 46774 41582 -5192
+ Misses 41931 26387 -15544
+ Partials 4146 3152 -994
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
lowc1012
changed the title
|
@lowc1012 , is this ready for review? |
Code Review Agent Run #55c786Actionable Suggestions - 3
Filtered by Review RulesBito filtered these suggestions based on rules created automatically for your feedback. Manage rules.
Review Details
AI Code Review powered by |
Changelist by BitoThis pull request implements the following key changes.
|
| go func() { | ||
| err = srv.Serve(tls.NewListener(conn, srv.TLSConfig)) | ||
| if err != nil && err != http.ErrServerClosed { | ||
| logger.Errorf(ctx, "Failed to start HTTP/2 Server: %v", err) | ||
| } | ||
| }() |
There was a problem hiding this comment.
The HTTP/2 server is now started in a goroutine, but the function immediately proceeds to wait for shutdown signals. This could lead to a race condition where the shutdown sequence begins before the server is fully initialized. Consider adding a small delay or a readiness check before proceeding to the shutdown logic.
Code suggestion
Check the AI-generated fix before applying
@@ -558,6 +558,13 @@
go func() {
err = srv.Serve(tls.NewListener(conn, srv.TLSConfig))
if err != nil && err != http.ErrServerClosed {
logger.Errorf(ctx, "Failed to start HTTP/2 Server: %v", err)
}
}()
+
+ // Give the server a moment to start before proceeding to shutdown logic
+ time.Sleep(100 * time.Millisecond)
+
+ // Log that the server has started
+ logger.Infof(ctx, "HTTP/2 Server started successfully on %s", cfg.GetHostAddress())
Code Review Run #55c786
Should Bito avoid suggestions like this for future reviews? (Manage Rules)
- Yes, avoid them
Signed-off-by: Ryan Lo <[email protected]>
Signed-off-by: Ryan Lo <[email protected]>
Code Review Agent Run #e9c2cdActionable Suggestions - 0Review Details
|
|
@eapolinario please help review it, thank you |
Tracking issue
#6274
Why are the changes needed?
What changes were proposed in this pull request?
How was this patch tested?
Labels
Please add one or more of the following labels to categorize your PR:
This is important to improve the readability of release notes.
Setup process
Screenshots
Check all the applicable boxes
Related PRs
Docs link
Summary by Bito
This PR implements graceful shutdown mechanisms for flyteadmin servers by adding signal handling and timeout-based contexts. It enhances error handling in gRPC server startup and ensures both HTTP and gRPC servers shut down properly when receiving termination signals, preventing resource leaks and improving service reliability. A redundant comment was also removed to improve code readability.Unit tests added: False
Estimated effort to review (1-5, lower is better): 1