[BUG] Network Timeout in CI Docker Workflows #14682
Labels
Arch: all
Issues that apply to all architectures
Area: Build system
OS: Linux
Issues related to Linux (building system, etc)
Type: Bug
Something isn't working
Description / Steps to reproduce the issue
Something super strange about Network Timeouts (pic below) in our CI Docker Workflows at GitHub Actions. Here's an example...
First Run fails while downloading something from GitHub...
Second Run fails again, while downloading NimBLE from GitHub...
Third Run succeeds. Why do we keep seeing these errors: GitHub Actions with Docker, can't connect to GitHub itself?
Is there a Concurrent Connection Limit for GitHub HTTPS Connections?
We see 4 Concurrent Connections to GitHub HTTPS...
risc-v-05 at 00:41:06
xtensa-02 at 00:41:17
xtensa-01 at 00:41:34
risc-v-02 at 00:41:58
The Fifth Connection failed: arm-02 at 00:42:52
Should we use a Caching Proxy Server for curl?
$ export https_proxy=https://1.2.3.4:1234 $ curl https://github.com/...
Is something misconfigured in our Docker Image? But the exact same Docker Image runs fine on our own Build Farm. It doesn't show any errors.
Is GitHub Actions starting our Docker Container with the wrong MTU (Network Packet Size)? 🤔
GitHub Actions with Smaller MTU Size
Docker MTU issues and solutions
Meanwhile I'm running a script to Restart Failed Jobs on our NuttX Mirror Repo: restart-failed-job.sh
These Timeout Errors will cost us precious GitHub Minutes. The remaining jobs get killed, and restarting these killed jobs from scratch will consume extra GitHub Minutes. (The restart below costs us 6 extra GitHub Runner Hours)
How do we Retry these Timeout Errors?
Can we have Restartable Builds?
Doesn't quite make sense to kill everything and rebuild from scratch (arm6, arm7, riscv7) just because one job failed (xtensa2)
Or xtensa2 should wait for others to finish, before it declares a timeout and croaks?
(See the Complete Log)
On which OS does this issue occur?
[OS: Linux]
What is the version of your OS?
Ubuntu LTS at GitHub Actions
NuttX Version
master
Issue Architecture
[Arch: all]
Issue Area
[Area: Build System]
Verification
The text was updated successfully, but these errors were encountered: