HTTP: add ability to return early / skip future callbacks

[mpwarres]

(This is somewhat related to #63, not sure if it's better to fold into that or keep as a separate issue)

ext_proc filter supports the notion of a processing mode override, in which an ext_proc ProcessingResponse for e.g. request headers can say: "skip future callbacks for this request", avoiding the cost of RPCs for request/response body.

For Proxy-Wasm, we might consider adding something analogous, for example a new proxy_action_t value indicating remaining non-lifecycle callbacks can be skipped, or allowing proxy_done to be called outside of proxy_on_done.

[PiotrSikora]

I was mentally bundling this with #63 (see: Done), but it doesn't hurt to have an explicit issue for that. Thanks!

[martijneken]

Note that there are two related flavors of such a feature:

1. Controlling future execution phases (a la mode_override, dynamic control over future events)
2. Early return from the current phase (i.e. "I'm done", early return from streaming callbacks as in body processing)

In the proposal linked above, I'm not sure how EndStream and Done map to these concepts.

[PiotrSikora]

EndStream is a request/response modification that adds end_stream flag that potentially wasn't there and finishes request or response (e.g. forwarding only HTTP headers upstream), but doesn't prevent future callbacks.

Done prevents future callbacks for given request/response, which isn't as elaborate as ext_proc's mode_override, but perhaps that's sufficient?

[martijneken]

Thanks Piotr. Checking my understanding of the proposed proxy_on_http_request_headers/body next_action values:

• Close means break the user stream -- presumably it should usually be paired with proxy_send_http_response?
• EndStream means truncate the user stream -- if used in the HTTP request direction, what happens to HTTP response?
• Done means the same as ext_proc half close -- send me no more data (any direction) but continue the user stream

I think adding a HalfDone or DirectionDone status would be helpful here. This would mean "send me no more data in this direction", and if it's used on the HTTP request path would still send data on the HTTP response path. This status could be used twice (once on request, once on response) and it is much simpler to reason about than mode_override IMO.