Clarify and document the semantics of outbound_http.max_concurrent_requests

[rylev]

The max_concurrent_requests config option is easy to misread as a ceiling on concurrent request throughput, but the actual semantic is subtler and worth documenting explicitly.

What it actually controls

The limit is implemented as a cap on the number of simultaneously open TCP connections. A semaphore permit is acquired when a TCP connection is established and held for the lifetime of that connection. This means:

• max_concurrent_requests = N guarantees that at least N concurrent requests can always be in flight, because at least N connections are available.
• The actual concurrent request capacity may be much higher than N: HTTP/2 connections multiplex many requests over a single connection, so N connections can carry far more than N simultaneous requests.
• HTTP/1.1 connections carry one request at a time, so for HTTP/1.1-only workloads the limit behaves closer to a direct request ceiling.

A better way to read the setting: it is a guaranteed minimum concurrency level, not a maximum.

Why this matters

Operators setting max_concurrent_requests = 10 expecting to hard-cap throughput at 10 requests may be surprised that an H2-speaking upstream allows far more than 10 requests simultaneously. Conversely, operators who want to protect against connection exhaustion (file descriptors, kernel buffers) need to understand that they are limiting connections, not requests.

Inconsistency with the legacy fermyon:spin/http interface

The deprecated fermyon:spin/http interface (reqwest-based) implements the limit differently: the permit is held only for the duration of execute(), releasing it before the underlying TCP connection returns to reqwest's internal pool. This means for that interface the limit genuinely does cap concurrent request count, but idle TCP connections in reqwest's pool are not counted against the limit at all. The two interfaces therefore have different semantics for the same config key.

Proposed fix

• Update documentation (runtime config reference and runtime config docs) to describe the connection-lifetime semantic and the "minimum guaranteed concurrency" framing.
• Rename the internal RuntimeConfig::max_concurrent_connections field to max_concurrent_requests to match the public TOML key (currently they disagree), or add a comment explaining the relationship.
• Decide and document the intended behavior for the fermyon:spin/http path: either align it with the connection-lifetime semantic (e.g. by disabling reqwest's pool when the limit is set) or explicitly note the divergence given that the interface is deprecated.