Chunked response from function is blocked

[ryskiwt]

Dear maintainers,

I created an issue to of-watchdog repository (openfaas/of-watchdog#49).
And I think openfaas itself also has similar issues.

---

I wrote my template that is similar to https://github.com/openfaas-incubator/python-flask-template/tree/master/template/python3-flask.

The different from the original is that my template uses stream response (or, chunked response) as follows.

@app.route("/", defaults={"path": ""}, methods=["POST", "GET"])
@app.route("/<path:path>", methods=["POST", "GET"])
def main_route(path):
    ...
    def gen():
        yield "1"
        time.sleep(1)
        yield "2"
        time.sleep(1)
        yield "3"

    return Response(gen())

When I invoke a function from the template, the function does not return response chunk by chunk.
It blocks 3 seconds, and then, returns whole the response (1, 2 and 3) all at once.

Expected Behaviour

Function returns response chunk by chunk.

Current Behaviour

Function blocks 3 seconds, and then, returns whole the response all at once.

Possible Solution

The following code may block until whole the response returned.

faas/gateway/handlers/forwarding_proxy.go

Line 130 in 04d240a

io.CopyBuffer(w, res.Body, nil)

And I found a hint to solve this issue in httputil package's ReverseProxy.copyBuffer method.

https://golang.org/src/net/http/httputil/reverseproxy.go#L335

copyBuffer method reads response little by little and writes to maxLatencyWriter

https://golang.org/src/net/http/httputil/reverseproxy.go#L366

and maxLatencyWriter flushes periodically.

https://golang.org/src/net/http/httputil/reverseproxy.go#L418

Steps to Reproduce (for bugs)

1. fetch this template https://github.com/openfaas-incubator/python-flask-template/tree/master/template/python3-flask
2. edit the template as described above
3. create new function from the template
4. build, push, deploy and invoke the function

Your Environment

Docker version 18.09.1, build 4c52b90
Docker swarm
Linux (vagrant, vm.box = "bento/centos-7.4")

Thank you.

[alexellis]

Hi,

Thanks for your interest in OpenFaaS.

I see that returning chunked-encoding is actually supported and not blocked, it just gets buffered before being returned to you.

Function blocks 3 seconds, and then, returns whole the response all at once.

In this example your client should be blocked for two seconds, not 3 given you have sleep(1) only twice.

The example given is a contrived example. Can you explain your business use-case which specifically needs a chunked-encoding to stream to the caller? Just to be up-front about this, it's unlikely to change, but I'd be curious to know more.

If you have long-running functions my advice would be to use the asynchronous processing in OpenFaaS.

https://docs.openfaas.com/deployment/troubleshooting/#asynchronous-functions

I'll close the issue, but please keep commenting.

Alex