Support HTTP.jl 2.0 alongside 1.x (#96)

* Support HTTP.jl 2.0 alongside 1.x

Add compatibility for HTTP.jl 2.0's reworked public API while keeping
1.x working. A load-time feature flag (_HTTP_V2) and small shims in the
HTTP.jl client backend bridge the differences:

- statustext: HTTP.Messages removed -> fall back to Response.reason
- header lookup: HTTP.Header / HTTP.header(::Vector) removed -> plain
  case-insensitive lookup over the request headers
- content_type returns a String on 2.0 (was a Pair on 1.x)
- TimeoutError.readtimeout -> .timeout_ns; ConnectError.error -> .cause
- timeout milliseconds forced to Integer so the reason string still
  matches the `\d+ milliseconds` regex in is_longpoll_timeout
- read(io, n) removed -> readbytes!; drop the verbose kwarg unsupported
  by 2.0 HTTP.open; map readtimeout -> read_idle_timeout on 2.0

server.jl: HTTP.Forms.Multipart -> HTTP.Multipart (same type on 1.x).

Tests/fixtures updated for both versions, and the timeout server fixture
uses HTTP.listen! on 2.0 (serve!(...; stream=true) on 1.x).

CI now tests both HTTP majors via an http matrix dimension; a step pins
the HTTP compat per cell. Julia 1.6 + HTTP 2 is excluded (HTTP 2.0
requires Julia >= 1.10).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix CI HTTP pin: scope sed to [compat] section

The unscoped sed matched the HTTP UUID line in [deps] too, rewriting it
to a bare version and breaking resolution (Malformed UUID string).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Detect HTTP major via pkgversion instead of submodule sniffing

Use pkgversion(HTTP) >= v"2" (guarded for Julia < 1.9, where only HTTP
1.x can be installed) rather than checking for the internal HTTP.Messages
submodule. Explicit about intent and not coupled to an internal name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Normalize HTTP 2.x DNSError reason to match 1.x

HTTP.jl 2.x introduced a dedicated HTTP.DNSError type for name-resolution
failures. It stringifies as "HTTP.DNSError(...)", whereas 1.x wrapped a
Sockets.DNSError in a ConnectError that stringified as "DNSError: ...".
This broke the client's reason-string contract (tests assert the reason
starts with "DNSError").

Add a version-guarded _http_error_message specialization that normalizes the
2.x DNSError into "DNSError: could not resolve host "<host>"". Elided under
1.x via @static if _HTTP_V2, so 1.x behavior is unchanged.

* Fix generated server code for HTTP 2: HTTP.headers(req) removed

HTTP.jl 2.x removed the 1-arg HTTP.headers(req) form that returns all
headers; only the 2-arg lookup headers(req, key) remains. Generated server
read-handlers used Dict{String,String}(HTTP.headers(req)) to collect header
params, so any endpoint with a header param (e.g. delete_pet's api_key)
threw a MethodError -> HTTP 500 under HTTP 2.

Use req.headers, which works on both 1.x and 2.x.

NOTE: this is generated code. The matching change belongs upstream in the
openapi-generator Julia server template; these checked-in fixtures are
hand-patched to match until that lands.

* Make allany verbose tests HTTP 2 compatible

Two HTTP.jl 2.x behavior changes broke test_debug:

1. verbose=true output: 1.x (and the curl backend) write the raw exchange to
   stderr; 2.x writes a '[http] ... via h1' format to stdout. The test
   captured stderr and asserted 'HTTP/1.1 200 OK', so under 2.x the pipe got
   no data and readavailable() blocked forever (hung the whole suite).
   Capture the right stream per version, assert the matching format, and
   close the pipe write-end before reading so an empty stream yields EOF
   instead of hanging.

2. response body framing: 1.x servers chunk the body ('27\r\n{json}\r\n0...'),
   2.x sends it unframed with Content-Length. The custom-verbose test parsed
   the JSON via split(str,'\n')[2] (assuming a chunk-size line) -> BoundsError
   under 2.x. Extract the JSON object by brace span instead, framing-agnostic.

* Resolve server header params case-insensitively for HTTP.jl 2.x

HTTP.jl 2.x canonicalizes incoming request header field names (e.g.
"api_key" arrives as "Api_key"), whereas 1.x preserves the sent case.
The server's get_param did a case-sensitive Dict lookup, so header
params silently resolved to nothing under HTTP 2.x — a required header
param would 400, an optional one would be dropped.

Add a case-insensitive fallback in get_param (src/server.jl) that fires
only when the exact lookup misses. Header field names are case-
insensitive per RFC, so this is spec-correct and version-agnostic; it
also fixes already-generated servers without regeneration. Query/path
param dicts are not canonicalized, so exact match always wins for them.

Add regression tests in test/param_deserialize.jl covering canonicalized
key resolution, exact-match precedence, missing/required params, and the
to_param end-to-end path.

* Raise wait_server timeout to 90s for slow CI server startup

Server processes launched via run_server use --pkgimages=no, forcing a
full recompile on each launch. Under HTTP.jl 2.x (slower precompile) on
cold CI runners this regularly exceeds the 20s wait, so the server tests
time out and get skipped, and timed-out-but-still-launching processes
orphan the port for the next testset. Bump the wait to 90s.

* bump patch version

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: tan <tanmaykm@gmail.com>

Author: 3 people

.github/workflows/ci.yml

@@ -6,7 +6,7 @@ on:
   pull_request:
 jobs:
   test:
-    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
+    name: Julia ${{ matrix.version }} - HTTP ${{ matrix.http }} - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ github.event_name }}
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -15,10 +15,17 @@ jobs:
           - '1.6'
           - '1' # automatically expands to the latest stable 1.x release of Julia
           - 'nightly'
+        http:
+          - '1'
+          - '2'
         os:
           - ubuntu-latest
         arch:
           - x64
+        exclude:
+          # HTTP.jl 2.0 requires Julia >= 1.10
+          - version: '1.6'
+            http: '2'
     steps:
       - uses: actions/checkout@v4
       - uses: julia-actions/setup-julia@v1
@@ -35,6 +42,8 @@ jobs:
             ${{ runner.os }}-test-${{ env.cache-name }}-
             ${{ runner.os }}-test-
             ${{ runner.os }}-
+      - name: Constrain HTTP.jl to v${{ matrix.http }}
+        run: sed -i -E '/^\[compat\]/,/^\[/{s/^HTTP = .*/HTTP = "${{ matrix.http }}"/}' Project.toml
       - uses: julia-actions/julia-buildpkg@v1
       - uses: julia-actions/julia-runtest@v1
       - uses: julia-actions/julia-processcoverage@v1

Project.toml

@@ -4,7 +4,7 @@ keywords = ["Swagger", "OpenAPI", "REST"]
 license = "MIT"
 desc = "OpenAPI server and client helper for Julia"
 authors = ["JuliaHub Inc."]
-version = "0.2.2"
+version = "0.2.3"
 
 [deps]
 Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
@@ -21,7 +21,7 @@ p7zip_jll = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
 
 [compat]
 Downloads = "1"
-HTTP = "1"
+HTTP = "1, 2"
 JSON = "0.20, 0.21, 1"
 LibCURL = "0.6, 1"
 MIMEs = "0.1, 1"

src/client/httplibs/juliaweb_http.jl

@@ -22,9 +22,60 @@
 #   - HTTPRequestError <: AbstractHTTPLibError
 # =============================================================================
 
+# HTTP.jl 2.0 reworked much of the public API; we support both 1.x and 2.x and
+# branch on the major version at load time. `pkgversion` exists from Julia 1.9;
+# on older Julia only HTTP 1.x can be installed (HTTP 2.0 needs Julia >= 1.10).
+const _HTTP_V2 = isdefined(Base, :pkgversion) && something(pkgversion(HTTP), v"1") >= v"2"
+
+# Status reason text: `HTTP.Messages.statustext` on 1.x, `Response.reason` on 2.x.
+function _http_statustext(raw::HTTP.Response)
+    if isdefined(HTTP, :Messages)
+        return HTTP.Messages.statustext(raw.status)
+    elseif hasproperty(raw, :reason) && !isempty(raw.reason)
+        return raw.reason
+    else
+        return string(raw.status)
+    end
+end
+
+# Case-insensitive header lookup over the request header collection (a Dict).
+# Avoids `HTTP.Header`/`HTTP.header(::Vector, ...)`, both removed in 2.0.
+function _header_value_ci(headers, key::AbstractString)
+    lk = lowercase(key)
+    for (k, v) in headers
+        lowercase(String(k)) == lk && return String(v)
+    end
+    return nothing
+end
+
+# Form content type: `HTTP.content_type` returns a `Pair` on 1.x, a `String` on 2.x.
+_http_form_content_type(body) = (ct = HTTP.content_type(body); ct isa Pair ? ct[2] : ct)
+
+# Timeout budget in ms: `TimeoutError.readtimeout` (s) on 1.x, `.timeout_ns` on 2.x.
+# Keep this an integer — the reason string is later matched against `\d+ milliseconds`.
+_http_timeout_ms(e::HTTP.TimeoutError) =
+    hasproperty(e, :readtimeout) ? e.readtimeout * 1000 : e.timeout_ns ÷ 1_000_000
+
+# Underlying cause of a connect error: `.error` on 1.x, `.cause` on 2.x.
+_http_connect_cause(e::HTTP.ConnectError) = hasproperty(e, :cause) ? e.cause : e.error
+
+# Message for a generic HTTP error. HTTP 2.x introduces a dedicated `HTTP.DNSError`
+# (a subtype of HTTP.HTTPError) for name-resolution failures, where 1.x instead wraps a
+# `Sockets.DNSError` inside a ConnectError. The two stringify differently
+# (`"HTTP.DNSError(...)"` vs `"DNSError: ..."`); normalize the 2.x form so the surfaced
+# reason starts with "DNSError" on both, keeping the message stable across versions.
+_http_error_message(error::HTTP.HTTPError) = string(error)
+@static if _HTTP_V2
+    _http_error_message(error::HTTP.DNSError) = "DNSError: could not resolve host \"$(error.hostname)\""
+end
+
+# Inactivity-timeout keyword: 1.x calls it `readtimeout`; 2.0 renamed it to
+# `read_idle_timeout` (`readtimeout` still works but emits a deprecation warning).
+_http_read_timeout_kw(timeout) = _HTTP_V2 ? (; read_idle_timeout=timeout) : (; readtimeout=timeout)
+
 function get_response_property(raw::HTTP.Response, name::Symbol)
     if name === :message
-        return HTTP.Messages.statustext(raw.status)
+        return _http_statustext(raw)
     else
         return getproperty(raw, name)
     end
@@ -40,26 +91,27 @@ struct HTTPRequestError <: AbstractHTTPLibError
     response::Union{Nothing,HTTP.Response}
 
     function HTTPRequestError(error::HTTP.TimeoutError, bytesread::Int, response::Union{Nothing,HTTP.Response})
-        message = "Operation timed out after $(error.readtimeout*1000) milliseconds with $(bytesread) bytes received"
+        message = "Operation timed out after $(_http_timeout_ms(error)) milliseconds with $(bytesread) bytes received"
         new(message, error, response)
     end
 
     function HTTPRequestError(error::HTTP.TimeoutError, response::Union{Nothing,HTTP.Response})
-        message = "Operation timed out after $(error.readtimeout*1000) milliseconds"
+        message = "Operation timed out after $(_http_timeout_ms(error)) milliseconds"
         new(message, error, response)
     end
 
     function HTTPRequestError(error::HTTP.ConnectError)
-        message = if isa(error.error, CapturedException)
-            string(error.error.ex)
+        cause = _http_connect_cause(error)
+        message = if isa(cause, CapturedException)
+            string(cause.ex)
         else
-            string(error.error)
+            string(cause)
         end
         new(message, error, nothing)
     end
 
     function HTTPRequestError(error::HTTP.HTTPError)
-        message = string(error)
+        message = _http_error_message(error)
         new(message, error, nothing)
     end
 end
@@ -99,8 +151,7 @@ function prep_args(::Val{:http}, ctx::Ctx)
     headers = ctx.header
     body = nothing
 
-    header_pairs = [convert(HTTP.Header, p) for p in headers]
-    content_type_set = HTTP.header(header_pairs, "Content-Type", nothing)
+    content_type_set = _header_value_ci(headers, "Content-Type")
     if !isnothing(content_type_set)
         content_type_set = lowercase(content_type_set)
     end
@@ -146,7 +197,7 @@ function prep_args(::Val{:http}, ctx::Ctx)
                 body_dict[_k] = _v
             end
             body = HTTP.Form(body_dict)
-            headers["Content-Type"] = content_type_set = HTTP.content_type(body)[2]
+            headers["Content-Type"] = content_type_set = _http_form_content_type(body)
         end
 
         if ctx.body !== nothing
@@ -213,7 +264,7 @@ end
 
 function _http_request(ctx, method, url, headers, body, timeout, bytesread, captured_response, output)
     captured_response[] = http_response = HTTP.request(method, url, headers, body;
-                           readtimeout=timeout,
+                           _http_read_timeout_kw(timeout)...,
                            connect_timeout=timeout ÷ 2,
                            retry=false,
                            redirect=true,
@@ -229,22 +280,30 @@ end
 function _http_streaming_request(ctx, method, url, headers, body, timeout, bytesread, captured_response, output, stream_to)
     http_response = nothing
 
+    # HTTP.jl 2.0's `HTTP.open` does not accept a `verbose` keyword; only pass it on 1.x.
+    open_kwargs = merge(_http_read_timeout_kw(timeout),
+                        (; connect_timeout=timeout ÷ 2,
+                           retry=false,
+                           redirect=true,
+                           status_exception=false))
+    if !_HTTP_V2
+        open_kwargs = merge(open_kwargs, (; verbose=get(ctx.client.clntoptions, :verbose, false)))
+    end
+
     @sync begin
         @async begin
             try
-                HTTP.open(method, url, headers;
-                                      readtimeout=timeout,
-                                      connect_timeout=timeout ÷ 2,
-                                      retry=false,
-                                      redirect=true,
-                                      status_exception=false,
-                                      verbose=get(ctx.client.clntoptions, :verbose, false)) do io
+                HTTP.open(method, url, headers; open_kwargs...) do io
                     write(io, body)
                     captured_response[] = http_response = startread(io)
                     try
+                        # `read(io, n)` is unavailable on 2.0 streams; read into a reusable
+                        # buffer with `readbytes!`, which works on both 1.x and 2.x.
+                        buf = Vector{UInt8}(undef, 8192)  # 8KB chunks
                         while !eof(io)
-                            data = read(io, 8192)  # Read 8KB chunks
-                            bytesread[] += write(output, data)
+                            n = readbytes!(io, buf)
+                            n == 0 && break
+                            bytesread[] += write(output, view(buf, 1:n))
                         end
                     finally
                         close(output)

src/server.jl

@@ -61,13 +61,27 @@ end
 
 function get_param(source::Dict, name::String, required::Bool)
     val = get(source, name, nothing)
+    if isnothing(val)
+        # HTTP header field names are case-insensitive, and some HTTP.jl versions
+        # canonicalize incoming request header names (e.g. "api_key" -> "Api_key").
+        # Fall back to a case-insensitive match so header params resolve regardless
+        # of the HTTP.jl version. The exact lookup above wins first, so query/path
+        # params (whose dicts are not canonicalized) are unaffected.
+        lname = lowercase(name)
+        for (k, v) in source
+            if lowercase(k) == lname
+                val = v
+                break
+            end
+        end
+    end
     if required && isnothing(val)
         throw(ValidationException("required parameter \"$name\" missing"))
     end
     return val
 end
 
-function get_param(source::Vector{HTTP.Forms.Multipart}, name::String, required::Bool)
+function get_param(source::Vector{HTTP.Multipart}, name::String, required::Bool)
     ind = findfirst(x -> x.name == name, source)
     if required && isnothing(ind)
         throw(ValidationException("required parameter \"$name\" missing"))
@@ -140,7 +154,7 @@ function to_param(T, source::Dict, name::String; required::Bool=false, collectio
     end
 end
 
-function to_param(T, source::Vector{HTTP.Forms.Multipart}, name::String; required::Bool=false, collection_format::Union{String,Nothing}=",", multipart::Bool=false, isfile::Bool=false)
+function to_param(T, source::Vector{HTTP.Multipart}, name::String; required::Bool=false, collection_format::Union{String,Nothing}=",", multipart::Bool=false, isfile::Bool=false)
     param = get_param(source, name, required)
     if param === nothing
         return nothing

test/client/allany/runtests.jl

@@ -86,14 +86,26 @@ function test_debug(httplib::Symbol)
         )
         api = M.DefaultApi(client)
 
+        # HTTP.jl 2.x routes `verbose=true` output to stdout in a "[http] ... via h1"
+        # format, whereas 1.x (and the Downloads/curl backend) write the raw exchange to
+        # stderr. Capture the right stream and assert on the matching format per version.
+        use_stdout = httplib === :http && OpenAPI.Clients._HTTP_V2
         pipe = Pipe()
-        redirect_stderr(pipe) do
+        redirect = use_stdout ? redirect_stdout : redirect_stderr
+        redirect(pipe) do
             pet = M.AnyOfMappedPets(mapped_cat)
             api_return, http_resp = echo_anyof_mapped_pets_post(api, pet)
             @test pet_equals(api_return, pet)
         end
-        out_str = String(readavailable(pipe))
-        @test occursin("HTTP/1.1 200 OK", out_str)
+        # Close the write end so the read sees EOF; without this, `readavailable` blocks
+        # forever when nothing was written to the captured stream.
+        close(pipe.in)
+        out_str = read(pipe, String)
+        if use_stdout
+            @test occursin("[http]", out_str) && occursin("200", out_str)
+        else
+            @test occursin("HTTP/1.1 200 OK", out_str)
+        end
     end
 
     if httplib === :downloads
@@ -150,7 +162,10 @@ function test_debug(httplib::Symbol)
                 write(iob, message)
             end
             data_in_str = String(take!(iob))
-            data_in_str = strip(split(data_in_str, "\n")[2])
+            # The curl backend reports the raw response body. HTTP.jl 1.x servers send it
+            # chunk-framed ("27\r\n{json}\r\n0\r\n\r\n"); 2.x sends an unframed body with a
+            # Content-Length. Extract the JSON object itself so the parse works either way.
+            data_in_str = data_in_str[findfirst('{', data_in_str):findlast('}', data_in_str)]
             data_in_json = JSON.parse(data_in_str)
             @test data_in_json["pet_type"] == "cat"
             @test data_in_json["hunts"] == true

test/client/utilstests.jl

@@ -68,16 +68,20 @@ end
 function test_longpoll_exception_check()
     resp = OpenAPI.Clients.Downloads.Response("http", "http://localhost", 200, "no error", [])
 
+    # HTTP.jl 1.x and 2.0 have different `TimeoutError`/`ConnectError` constructors.
+    mk_timeout_err() = OpenAPI.Clients._HTTP_V2 ? HTTP.TimeoutError("read_idle", 20_000_000) : HTTP.TimeoutError(20)
+    mk_connect_err() = HTTP.ConnectError("http://localhost", ErrorException("dns error"))
+
     not_longpoll_timeouts = [
         OpenAPI.Clients.Downloads.RequestError("http://localhost", 500, "not timeout error", resp),
-        OpenAPI.Clients.HTTPRequestError(HTTP.TimeoutError(20), 20, nothing),
-        OpenAPI.Clients.HTTPRequestError(HTTP.TimeoutError(20), nothing),
-        OpenAPI.Clients.HTTPRequestError(HTTP.ConnectError("http://localhost", "dns error")),
+        OpenAPI.Clients.HTTPRequestError(mk_timeout_err(), 20, nothing),
+        OpenAPI.Clients.HTTPRequestError(mk_timeout_err(), nothing),
+        OpenAPI.Clients.HTTPRequestError(mk_connect_err()),
     ]
 
     longpoll_timeouts = [
         OpenAPI.Clients.Downloads.RequestError("http://localhost", 200, "Operation timed out after 300 milliseconds with 0 bytes received", resp), # timeout error
-        OpenAPI.Clients.HTTPRequestError(HTTP.TimeoutError(20), 20, HTTP.Response(200, "hello")),
+        OpenAPI.Clients.HTTPRequestError(mk_timeout_err(), 20, HTTP.Response(200, "hello")),
     ]
 
     @test OpenAPI.Clients.is_longpoll_timeout("not an exception") == false

test/deep_object/deep_server.jl

@@ -6,7 +6,7 @@ using .DeepServer: register, FindPetsByStatus200Response
 
 const server = Ref{Any}(nothing)
 
-function find_pets_by_status(::HTTP.Messages.Request, param::DeepServer.FindPetsByStatusStatusParameter)
+function find_pets_by_status(::HTTP.Request, param::DeepServer.FindPetsByStatusStatusParameter)
     return FindPetsByStatus200Response(param)
 end