Fix image generation request timeout#1231
Conversation
|
Thanks @szkocot! im wondering though do we really need the timeout at all? Why not just set it to 0 and skip the env var or CLI option? |
|
@jeremyfowers Defaulting to 0 avoids surprising long-running jobs, but keeping the option is still useful for production or shared deployments where operators may want a safety limit (hung backends, resource control, load balancing, etc.). It’s basically flexibility at no cost for users who don’t need it. |
|
Looks like @superm1 is merging #1272 which is highly related. @szkocot is it ok for you to just set the new global timeout to 0 for you use case, or do you still want a specific change for image gen? If you want something specific for image gen, please change this PR so that it has just the env var, not the CLI arg, and the imagegen timeout var supercedes the global timeout for image gen. |
|
Yes, sure. |
345af67 to
23c0a45
Compare
#1422 makes sure that the global timeout applies to image generation, which solves the core issue. |
Summary
Fixes #1222.
Image generation requests for
sd-cppcould time out after 10 minutes even while the backend was still actively generating the image.This change removes the hardcoded timeout behavior and adds a configurable server-side setting for image request timeouts.
Changes
--image-request-timeout <seconds>LEMONADE_IMAGE_REQUEST_TIMEOUT=<seconds>image_request_timeoutto0meaning no overall timeoutBehavior Before
Stable Diffusion requests used a fixed 10 minute timeout in the backend forwarding layer. Long-running image generation jobs could fail
with a network timeout even though
sd-serverwas still processing the request.Behavior After
Stable Diffusion image requests now use a configurable timeout, with the default set to no overall timeout.
Examples: