Skip to content

Large JSON integers lose precision in formatted output #397

Description

@natalie-o-perret

Description

Formatted or filtered JSON responses can change large integer values before an
output formatter sees them. Generic JSON decoding currently stores numbers as
float64, which cannot represent every integer above the IEEE-754 safe range.

The example below uses
2^53 + 1 = 9007199254740993.

Why Revisit #130

Encoding opaque identifiers as strings remains the safest API design when all
JSON consumers must interoperate with JavaScript. CBOR is also a good option
when the API supports a format with native large-integer handling.

This issue does not dispute that guidance. Restish cannot control the upstream
API contract, however. When a server sends a valid JSON integer, formatted
output should not silently change its value. The v2 content pipeline now makes
it practical to preserve plain integer tokens while retaining the existing
float64 behavior for safely representable numbers and decimals.

Reproduction

Run this fictional API locally:

from http.server import BaseHTTPRequestHandler, HTTPServer

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != "/records/current":
            self.send_error(404)
            return
        body = b'{"id":9007199254740993}\n'
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *_args):
        pass

HTTPServer(("127.0.0.1", 8765), Handler).serve_forever()

The wire response contains the exact value:

$ curl -sS http://127.0.0.1:8765/records/current
{"id":9007199254740993}

Explicit JSON formatting changes it:

$ restish get -o json http://127.0.0.1:8765/records/current
{
  "id": 9007199254740992
}

Expected Behavior

JSON integers should retain their exact value through response decoding,
filtering, and formatting:

{
  "id": 9007199254740993
}

Environment

Reproduced with Restish v2.3.0 and current main at 6305246.

Related to #130, which documented the same underlying behavior before the v2
content and output pipeline was introduced.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions