Feature: gateway timeouts #78

superbeeny · 2025-01-27T18:40:27Z

New design note for gateway timeouts

Signed-off-by: Nick Beenham <[email protected]>

willdavsmith · 2025-02-04T19:12:15Z

resources/2025-01-gateway-timeouts.md

+we plan to do in the future, but are out of scope of this design, list
+them here. Provide a brief explanation on why this is a non-goal.
+-->
+None


Is it out of scope to provide more httpproxy punch-through options? Is this pattern extensible to other routing configuration users may want to provide in the future?

Will add note to doc to limit scope to requirements

ytimocin · 2025-02-04T19:11:28Z

resources/2025-01-gateway-timeouts.md

+team. Do not provide the design details in this, section - there is a
+dedicated section for that later in the document.
+-->
+The purpose of this feature is to allow the user to configure a timeout on the gateway for an application. This will allow the user to specify how long the gateway should wait for a response from the application before timing out. This will be useful in scenarios where the application may take a long time to respond, or where the user wants to ensure that the application responds within a certain time frame.


What is the main difference between health/readiness probes and this? Is this for an endpoint that would take a long time to process something?

Yes, my explicit use case is for LLM responses

willdavsmith · 2025-02-04T19:14:23Z

resources/2025-01-gateway-timeouts.md

+        destination: 'http://${ui.name}:3000'
+      }
+      {
+        {


syntax is kind of confusing. if I'm reading it right, this timeoutPolicy is applied to the /api route. why is timeoutPolicy not present at the top level along with path and destination?

this is a typo that I will fix

nithyatsu · 2025-02-04T19:15:56Z

resources/2025-01-gateway-timeouts.md

+team. Do not provide the design details in this, section - there is a
+dedicated section for that later in the document.
+-->
+The purpose of this feature is to allow the user to configure a timeout on the gateway for an application. This will allow the user to specify how long the gateway should wait for a response from the application before timing out. This will be useful in scenarios where the application may take a long time to respond, or where the user wants to ensure that the application responds within a certain time frame.


Are the timeouts configurable per route?

willdavsmith · 2025-02-04T19:17:03Z

resources/2025-01-gateway-timeouts.md

+      }
+      {
+        {
+          timeoutPolicy: {


Is there a way (or a need) to apply global timeout policies for all routes?

Not necessary for this PR but worth some further investigation

kachawla · 2025-02-04T19:20:41Z

resources/2025-01-gateway-timeouts.md

+  **idle:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.
+  **idleConnection:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.
+
+This maps to the contour timeoutPolicy - https://github.com/projectcontour/contour/blob/main/apis/projectcontour/v1/httpproxy.go#L1126


Do we need to consider how this design will evolve if we support other ingress controllers options in future?

+1 as removing Contour as a hard dependency is on the roadmap: radius-project/roadmap#58

This will need refactoring, will check with k8s gateway API for similarities

kachawla · 2025-02-04T19:21:19Z

resources/2025-01-gateway-timeouts.md

+  **idle:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.
+  **idleConnection:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.


nit: Definitions for both idle and idleConnection are same..

Will update and correct definitions

nithyatsu · 2025-02-04T19:24:05Z

resources/2025-01-gateway-timeouts.md

+The solution would update the specification for the gateway resource in the bicep file to include a timeoutPolicy object. This object would contain the following properties:
+  **response:** The amount of time the gateway should wait for a response from the application before timing out.
+  **idle:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.
+  **idleConnection:** The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.


idle and idleConnection seem to have same meaning, should we clarify it better when we write the user docs?

Will update

smatochkin · 2025-02-04T19:32:59Z

Any thoughts on the future maintenance of the spec with the possible gateway implementation changes? The suggested contract change is tied to Contour specific implementation of the gateway. If there is a plan to switch from Contour specific API to more generic GatewayAPI, then the suggested contact will be hard to migrate.

brooke-hamilton

🚀 This is good - agree with checking on the K8s gateway api, as well.

superbeeny · 2025-02-05T19:35:15Z

Adding a new comment for timeouts comparison with the gateway api specification. Route Timeouts have 2 optional fields:

request defined as

Request specifies the maximum duration for a gateway to respond to an HTTP request. If the gateway has not been able to respond before this deadline is met, the gateway MUST return a timeout error.

and backendRequest defined as

BackendRequest specifies a timeout for an individual request from the gateway to a backend. This covers the time from when the request first starts being sent from the gateway to when the full response has been received from the backend.

Is the consensus to update to match the gateway specification?

github-actions · 2025-02-20T00:21:50Z

This pull request is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.

superbeeny · 2025-02-20T16:21:47Z

Can I get a review please @kachawla , @zachcasper @sylvainsf

Signed-off-by: Nick Beenham <[email protected]>

willdavsmith

LGTM, great doc!

Signed-off-by: Nick Beenham <[email protected]>

superbeeny · 2025-03-11T17:10:28Z

@willtsai is anything else needed here to approve and merge?

new design note for gateway timeouts

Loading
Loading status checks…

f075f3f

Signed-off-by: Nick Beenham <[email protected]>

superbeeny requested review from a team as code owners January 27, 2025 18:40

adding terms to words

Loading
Loading status checks…

e46f655

Signed-off-by: Nick Beenham <[email protected]>

superbeeny changed the title ~~new design note for gateway timeouts~~ Feature: gateway timeouts Jan 27, 2025

superbeeny mentioned this pull request Jan 28, 2025

Make gateway http timeouts configurable radius-project/roadmap#60

Open

willdavsmith reviewed Feb 4, 2025

View reviewed changes

ytimocin reviewed Feb 4, 2025

View reviewed changes

willdavsmith reviewed Feb 4, 2025

View reviewed changes

nithyatsu reviewed Feb 4, 2025

View reviewed changes

willdavsmith reviewed Feb 4, 2025

View reviewed changes

kachawla reviewed Feb 4, 2025

View reviewed changes

nithyatsu reviewed Feb 4, 2025

View reviewed changes

brooke-hamilton reviewed Feb 4, 2025

View reviewed changes

github-actions bot added the Stale label Feb 20, 2025

superbeeny added 2 commits February 20, 2025 11:32

tweaking to align with gateway API spec

Loading
Loading status checks…

ee5fb9a

Signed-off-by: Nick Beenham <[email protected]>

adding words

Loading
Loading status checks…

51cff8c

Signed-off-by: Nick Beenham <[email protected]>

github-actions bot removed the Stale label Feb 21, 2025

willdavsmith previously approved these changes Mar 11, 2025

View reviewed changes

superbeeny dismissed willdavsmith’s stale review via a801346 March 11, 2025 15:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: gateway timeouts #78

Feature: gateway timeouts #78

superbeeny commented Jan 27, 2025

willdavsmith Feb 4, 2025

superbeeny Feb 4, 2025

ytimocin Feb 4, 2025

superbeeny Feb 4, 2025

willdavsmith Feb 4, 2025

superbeeny Feb 4, 2025

nithyatsu Feb 4, 2025

superbeeny Feb 4, 2025

willdavsmith Feb 4, 2025

superbeeny Feb 4, 2025

kachawla Feb 4, 2025 •

edited

Loading

willtsai Feb 4, 2025

superbeeny Feb 4, 2025

kachawla Feb 4, 2025

superbeeny Feb 4, 2025

nithyatsu Feb 4, 2025

superbeeny Feb 4, 2025

smatochkin commented Feb 4, 2025

brooke-hamilton left a comment

superbeeny commented Feb 5, 2025

github-actions bot commented Feb 20, 2025

superbeeny commented Feb 20, 2025

willdavsmith left a comment

superbeeny commented Mar 11, 2025

		idle: The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.
		idleConnection: The amount of time the gateway should wait for a response from the application before timing out when the connection is idle.

Feature: gateway timeouts #78

Are you sure you want to change the base?

Feature: gateway timeouts #78

Conversation

superbeeny commented Jan 27, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kachawla Feb 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smatochkin commented Feb 4, 2025

brooke-hamilton left a comment

Choose a reason for hiding this comment

superbeeny commented Feb 5, 2025

github-actions bot commented Feb 20, 2025

superbeeny commented Feb 20, 2025

willdavsmith left a comment

Choose a reason for hiding this comment

superbeeny commented Mar 11, 2025

kachawla Feb 4, 2025 •

edited

Loading