`oas-schema-check` intermittently reports a false `const` type error (data race, since v0.29.3)

[gnuletik]

Since v0.29.3, the oas-schema-check rule intermittently reports a false
error for a perfectly valid const:

a.yaml:27:21  ✗ error  `const` value type does not match schema type [string]  oas-schema-check

The schema is valid OpenAPI 3.1 — status is type: string with const: ok,
and "ok" is a string, so there is nothing to flag. The error only appears on
some runs of the exact same input, which makes this a concurrency data race,
not a real validation finding:

• linting the offending spec alone trips it rarely (~1% of runs);
• linting a second, unrelated spec before it in the same invocation trips it
  often (~30% of runs);
• GOMAXPROCS=1 makes it disappear entirely;
• v0.29.2 never reproduces it.

v0.29.3 is the release that bumped libopenapi to v0.38.0 and "adapted the
rules to the lazy schema cache", so a shared/unsynchronized lazy schema cache is
the most likely culprit.

Affected versions

Version	Result
v0.29.2	not affected
v0.29.3	affected (libopenapi v0.38.0 + lazy schema cache landed here)
v0.29.4	affected (tested)

Reproduced on both linux/amd64 (~30% hit rate) and darwin/arm64
(~1.7% hit rate — same bug, just needs more iterations to observe).

$ vacuum version
0.29.4

Reproduction files

a.yaml — the valid spec containing the const:

openapi: "3.1.0"
info:
  title: A
  version: "1.0"
  description: spec A
servers:
  - url: https://a.example.com
tags:
  - name: a
    description: tag a
paths:
  /health:
    get:
      description: health
      operationId: checkHealth
      tags: [a]
      responses:
        "200":
          description: ok
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: string
                    const: ok
                    example: ok

b.yaml — any second, unrelated valid spec (its content is not important;
its presence in the same invocation is what raises the race's hit rate):

openapi: "3.1.0"
info:
  title: B
  version: "1.0"
  description: spec B
servers:
  - url: https://b.example.com
tags:
  - name: b
    description: tag b
paths:
  /x:
    get:
      description: get x
      operationId: getX
      tags: [b]
      responses:
        "200":
          description: ok
          content:
            application/json:
              schema:
                type: string
                example: hi

Steps to reproduce

The race is intermittent, so loop the command and count how often the false
error appears:

fails=0
for i in $(seq 1 300); do
  out="$(vacuum lint --errors b.yaml a.yaml 2>&1 || true)"
  case "$out" in *oas-schema-check*) fails=$((fails+1)) ;; esac
done
echo "$fails/300 runs hit the false const error"

Note: vacuum exits non-zero when it reports the error, so capture the output
(|| true) and grep the captured text. Piping vacuum ... | grep under
set -o pipefail would hide the failures behind vacuum's own exit code.

Full reproduce.sh that also runs the alone / GOMAXPROCS=1 / 0.29.2
comparisons:

#!/usr/bin/env bash
set -euo pipefail

VACUUM="${VACUUM:-vacuum}"        # set VACUUM=/path/to/vacuum to test a specific build
N="${N:-300}"                     # iterations (race is intermittent; needs a loop)
cd "$(dirname "$0")"

echo "vacuum: $($VACUUM version 2>/dev/null | head -1)"
echo "GOMAXPROCS=${GOMAXPROCS:-<default>}"
echo

count() { # files... -> prints "<fails>/<N>"
  local f=0 out
  for _ in $(seq 1 "$N"); do
    out="$("$VACUUM" lint --errors "$@" 2>&1 || true)"
    case "$out" in *oas-schema-check*) f=$((f+1)) ;; esac
  done
  echo "$f/$N"
}

echo "a.yaml alone .............. $(count a.yaml)"
echo "b.yaml a.yaml batched ..... $(count b.yaml a.yaml)"

Run it against the broken and the last-good build:

VACUUM=./vacuum-0.29.4 N=200 ./reproduce.sh
VACUUM=./vacuum-0.29.4 N=200 GOMAXPROCS=1 ./reproduce.sh
VACUUM=./vacuum-0.29.2 N=200 ./reproduce.sh

Observed results (vacuum 0.29.4, linux/amd64)

run	false oas-schema-check hits
a.yaml alone	~2/200 (~1%)
b.yaml a.yaml, default concurrency	~61/200 (~30%)
b.yaml a.yaml, GOMAXPROCS=1	0/200
b.yaml a.yaml, vacuum 0.29.2	0/200

A full failing run (vacuum lint --details --errors b.yaml a.yaml):

 > a.yaml
 ----------------------------------------------------------------------------------------------------------

Location      Severity   Message                                                                          Rule
────────────  ─────────  ───────────────────────────────────────────────────────────────────────────────  ────────────────
a.yaml:27:21  ✗ error     `const` value type does not match schema type [string]                           oas-schema-check

 category              ✗ errors      ▲ warnings    ● info
 Schemas               1             0             0
 total                 1             0             0

 | ✗ Failed with 1 errors, 0 warnings and 0 informs.

The same invocation on a passing run reports 0 errors for a.yaml.

Expected

oas-schema-check should pass deterministically. const: ok is a valid string
value for type: string, and the result must never depend on goroutine
scheduling or on which other files happen to share the invocation.

Likely cause / suggested next step

GOMAXPROCS=1 eliminating the failure strongly indicates an unsynchronized
concurrent access — almost certainly in the lazy schema cache introduced in
v0.29.3. A -race build of vacuum run on a single vacuum lint b.yaml a.yaml
should flag the racy read/write directly.

[gnuletik]

Should be fixed with pb33f/libopenapi#594

[amai2012]

I think that might explain strange errors in my pipeline which may disappear on a re-run.

[amai2012]

Looking forward to a new release ;-)

[daveshanley]

not sure why this was autoclosed. I did not do that.

[daveshanley]

Fixed in v0.25.9