port_forward panics when connection to pod is lost

[bsamuels453]

Getting a panic here that seems to be caused by some sort of race condition:

attacknet/pkg/kubernetes/port_forward.go

Line 120 in 6175dcd

if err = portForward.ForwardPorts(); err != nil {

No pods were restarted/failed during the connection, so this may be root caused by some tcp error that causes the connection to close. Should be handled by our retry logic, but instead the k8s go client panics.

The following channel is closed when PortForward() is called despite it has already being closed: https://github.com/kubernetes/client-go/blob/8c4efe8d079e405329f314fb789a41ac6af101dc/tools/portforward/portforward.go#L227-L229

What's interesting is the above code is the only place where the channel is ever closed. I suspect what's happening is the portforward object is not re-usable if it fails to connect - we'll have to re-build it for each dial attempt.

Stacktrace:

runtime.fatalpanic (panic.go:1188) runtime
runtime.gopanic (panic.go:1017) runtime
runtime.closechan (chan.go:365) runtime
portforward.(*PortForwarder).forward (portforward.go:228) k8s.io/client-go/tools/portforward
portforward.(*PortForwarder).ForwardPorts (portforward.go:200) k8s.io/client-go/tools/portforward
kubernetes.(*KubeClient).StartPortForwarding.func1 (port_forward.go:120) attacknet/cmd/pkg/kubernetes
kubernetes.(*KubeClient).StartPortForwarding.func3 (port_forward.go:130) attacknet/cmd/pkg/kubernetes
runtime.goexit (asm_arm64.s:1197) runtime
 - Async Stack Trace
kubernetes.(*KubeClient).StartPortForwarding (port_forward.go:118) attacknet/cmd/pkg/kubernetes

The console has the following messages, which may not be relevant but are worth including:

E0222 10:01:08.820313   85675 portforward.go:419] error closing listener: close tcp4 127.0.0.1:57176: use of closed network connection
E0222 10:01:08.820338   85675 portforward.go:419] error closing listener: close tcp6 [::1]:57176: use of closed network connection