Skip to content

refactor: use HTTP style start/end ranges #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alanshaw merged 2 commits into from
Aug 28, 2026
Merged

refactor: use HTTP style start/end ranges #7

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1c3970b
refactor: use HTTP style start/end ranges
alanshaw Aug 26, 2026
eed9021
fix: guard RangeReader.Read against nil sr to avoid panic
Copilot Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 12 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,11 @@ func roundTripExternalCEK(data []byte) ([]byte, error) {

Because chunks are sealed independently, any plaintext byte range can be
decrypted without fetching or decrypting the rest of the object.
`fee.DecryptRange` takes the stored blob as an `io.ReaderAt` plus its exact size,
unwraps the CEK just as `fee.Decrypt` does, and returns a reader over exactly the
requested bytes:
`fee.DecryptRange` takes the stored blob as an `io.ReaderAt` plus its exact
size and the inclusive plaintext range `[start, end]` — HTTP `Range` semantics,
where an `end` past the last byte clamps and an unsatisfiable range is
`aesstream.ErrRange` — unwraps the CEK just as `fee.Decrypt` does, and returns
a reader over exactly the requested bytes:

```go
import (
Expand All @@ -271,21 +273,16 @@ import (
)

// serveRange answers an HTTP range request straight from an encrypted object.
func serveRange(w http.ResponseWriter, f *os.File, size int64, u fee.RecipientUnwrapper, off, length int64) error {
r, err := fee.DecryptRange(f, size, u, off, length)
func serveRange(w http.ResponseWriter, f *os.File, size int64, u fee.RecipientUnwrapper, start, end int64) error {
r, err := fee.DecryptRange(f, size, u, start, end)
if err != nil {
return err // aesstream.ErrRange here means a 416
}

// Len is the requested length clamped to the object; Size is the whole
// Len is the range length after end clamps to the object; Size is the whole
// object's plaintext size. Both are known before any ciphertext is read.
if r.Len() == 0 {
w.Header().Set("Content-Range", fmt.Sprintf("bytes */%d", r.Size()))
w.WriteHeader(http.StatusRequestedRangeNotSatisfiable)
return nil
}
w.Header().Set("Content-Length", strconv.FormatInt(r.Len(), 10))
w.Header().Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", off, off+r.Len()-1, r.Size()))
w.Header().Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", start, start+r.Len()-1, r.Size()))
w.WriteHeader(http.StatusPartialContent)

_, err = io.Copy(w, r) // decrypts chunk by chunk, O(chunk size) memory
Expand All @@ -302,9 +299,9 @@ Nothing beyond the header is fetched until the first `Read`, so a caller backed
by a remote store can prefetch the whole span in a single range request:

```go
r, err := fee.DecryptRange(blob, size, unwrapper, off, length)
r, err := fee.DecryptRange(blob, size, unwrapper, start, end)
// ...
spanOff, spanLen := r.CiphertextSpan() // blob-absolute; one range request
spanStart, spanEnd := r.CiphertextSpan() // blob-absolute, inclusive; one range request
```

The span is chunk-aligned, so it over-fetches by at most the unused head of the
Expand All @@ -331,7 +328,7 @@ the upload is still streaming:
r, d, err := fee.Encrypt(plaintext, recipients)
```

`fee.DecryptRangeWithCEK(blob, blobSize, cek, off, length, &d)` then serves a
`fee.DecryptRangeWithCEK(blob, blobSize, cek, start, end, &d)` then serves a
range with no envelope round trip at all: the only bytes fetched are the
ciphertext chunks the range overlaps. `d.PlaintextSize(blobSize)` answers a
`HEAD` or resolves a suffix range from the stored record alone, reading
Expand Down
143 changes: 63 additions & 80 deletions aesstream/spanreader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@ var (
// distinct from ErrCorrupted (a chunk that failed authentication).
ErrCiphertextSize = fmt.Errorf("aesstream: ciphertext length is not a valid stream (the final chunk must be at least %d bytes, to hold the tag)", TagSize)

// ErrRange is returned when a requested plaintext range is invalid: a
// negative offset or length, or an offset past the end of the plaintext.
// A length that runs past the end is not an error — it is clamped to the
// bytes available from the offset.
// ErrRange is returned when a requested plaintext range is unsatisfiable:
// a negative start, an end before the start, or a start past the last
// plaintext byte — HTTP Range semantics, where there is no such thing as
// an empty range. An end past the last byte is not an error; it is
// clamped, exactly as HTTP clamps "bytes=a-b" to the resource length.
ErrRange = fmt.Errorf("aesstream: requested range is out of bounds")

// ErrShortSpan is returned when the supplied ciphertext span ends before
Expand Down Expand Up @@ -126,13 +127,16 @@ func ChunkCount(ciphertextLen int64, chunkSize int) (int64, error) {
}

// CiphertextRange returns the single contiguous ciphertext byte range
// [start, start+n) that must be read to serve the plaintext range
// [off, off+length) of a stream whose complete ciphertext is
// [ctStart, ctEnd] (inclusive) that must be read to serve the plaintext
// range [start, end] (inclusive) of a stream whose complete ciphertext is
// ciphertextSize bytes long, along with plainLen, the plaintext length of
// the range after clamping. A zero chunkSize selects DefaultChunkSize; any
// other value must be in [MinChunkSize, MaxChunkSize] (else ErrChunkSize),
// the same rule as Config.ChunkSize.
//
// start/end follow HTTP Range semantics (see ErrRange): both inclusive, end
// clamped to the last plaintext byte, no empty ranges.
//
// Because the chunks overlapping any plaintext range are adjacent in the
// ciphertext, the bytes needed are always one contiguous span — so a caller
// fetching from a remote store (e.g. a range request to a blob) can pull
Expand All @@ -141,24 +145,23 @@ func ChunkCount(ciphertextLen int64, chunkSize int) (int64, error) {
// itself (its first byte is 0); a caller whose blob prefixes the stream
// with an envelope header must add that header length.
//
// plainLen is min(length, plaintext size − off): exactly the byte count the
// span decrypts to (SpanReader.Len, the length of OpenSpan's result). It is
// available before any ciphertext is fetched, so an HTTP consumer can size
// its response (Content-Length, the end of a Content-Range) from it rather
// than re-deriving the clamping rule.
// plainLen is the byte count the span decrypts to after end is clamped
// (SpanReader.Len, the length of OpenSpan's result). It is available before
// any ciphertext is fetched, so an HTTP consumer can size its response
// (Content-Length, the end of a Content-Range) from it rather than
// re-deriving the clamping rule.
//
// For a random-access source already in hand (a local file or in-memory
// buffer) rather than a one-shot fetch, wrap it as the span with
// io.NewSectionReader(src, start, n) and pass that to NewSpanReader.
// io.NewSectionReader(src, ctStart, ctEnd-ctStart+1) and pass that to
// NewSpanReader.
//
// The span is chunk-aligned, so it may include a little more than the
// requested bytes: at most the unused head of the first overlapping chunk
// and tail of the last (under 2*chunkSize total), since GCM authenticates a
// whole chunk at a time. A length past the end of the plaintext clamps; an
// empty span (n == 0) means no ciphertext need be fetched. off/length and
// ciphertextSize are validated exactly as in NewSpanReader, returning
// ErrRange, ErrCiphertextSize or ErrChunkSize.
func CiphertextRange(ciphertextSize int64, chunkSize int, off, length int64) (start, n, plainLen int64, err error) {
// whole chunk at a time. start/end and ciphertextSize are validated exactly
// as in NewSpanReader, returning ErrRange, ErrCiphertextSize or ErrChunkSize.
func CiphertextRange(ciphertextSize int64, chunkSize int, start, end int64) (ctStart, ctEnd, plainLen int64, err error) {
chunkSize, err = resolveChunkSize(chunkSize)
if err != nil {
return 0, 0, 0, err
Expand All @@ -167,27 +170,23 @@ func CiphertextRange(ciphertextSize int64, chunkSize int, off, length int64) (st
if err != nil {
return 0, 0, 0, err
}
if off < 0 || length < 0 || off > plaintextLen {
if start < 0 || end < start || start > plaintextLen-1 {
return 0, 0, 0, ErrRange
}
effLen := length
if avail := plaintextLen - off; effLen > avail {
effLen = avail
}
if effLen == 0 {
return 0, 0, 0, nil
if end > plaintextLen-1 {
end = plaintextLen - 1
}

enc := int64(chunkSize) + TagSize
firstChunk := off / int64(chunkSize)
lastChunk := (off + effLen - 1) / int64(chunkSize)
start = firstChunk * enc
end := (lastChunk + 1) * enc
firstChunk := start / int64(chunkSize)
lastChunk := end / int64(chunkSize)
ctStart = firstChunk * enc
ctEnd = (lastChunk+1)*enc - 1
if lastChunk == numChunks-1 {
// The final chunk may be short; clamp to the true stream end.
end = ciphertextSize
ctEnd = ciphertextSize - 1
}
return start, end - start, effLen, nil
return ctStart, ctEnd, end - start + 1, nil
}

// SpanReader decrypts an arbitrary plaintext byte range of a chunked
Expand All @@ -201,13 +200,13 @@ func CiphertextRange(ciphertextSize int64, chunkSize int, off, length int64) (st
// The span must begin at the offset CiphertextRange reported (the boundary
// of the first overlapping chunk) and contain the whole chunks overlapping
// the range, in order. SpanReader recomputes that geometry from the same
// (ciphertextSize, off, length) and reads exactly the chunks it needs, so:
// (ciphertextSize, start, end) and reads exactly the chunks it needs, so:
//
// - the total ciphertext length pins each chunk's index, final-chunk flag
// and length up front — no last-chunk look-ahead or retry, unlike the
// whole-stream Reader; and
// - the head of the first chunk (before off) and the tail of the last
// (after the range end) are trimmed, so the output is exactly the range.
// - the head of the first chunk (before start) and the tail of the last
// (after end) are trimmed, so the output is exactly the range.
//
// A SpanReader implements io.Reader. Any non-EOF error means the plaintext
// is incomplete and must be discarded: a tampered/reordered/misframed chunk
Expand Down Expand Up @@ -249,18 +248,19 @@ type SpanReader struct {
}

// NewSpanReader returns a SpanReader that yields the plaintext bytes
// [off, off+length) of the stream whose complete ciphertext is
// [start, end] (inclusive) of the stream whose complete ciphertext is
// ciphertextSize bytes long, reading the ciphertext from span. span must be
// positioned at the start of the contiguous range CiphertextRange reports
// for the same (ciphertextSize, chunkSize, off, length) — typically the
// for the same (ciphertextSize, chunkSize, start, end) — typically the
// body of a single range request for exactly that span. cfg must match the
// Writer that produced the stream, including the chunk size.
//
// length is clamped to the bytes available from off; off may equal the
// plaintext length (an empty read) but a larger off, or a negative off or
// length, returns ErrRange. ciphertextSize must be a structurally valid
// stream length (else ErrCiphertextSize).
func NewSpanReader(span io.Reader, cfg Config, ciphertextSize, off, length int64) (*SpanReader, error) {
// start/end follow HTTP Range semantics (see ErrRange): an end past the
// last plaintext byte clamps; a negative start, an end before the start, or
// a start past the last byte returns ErrRange — there is no empty range.
// ciphertextSize must be a structurally valid stream length (else
// ErrCiphertextSize).
func NewSpanReader(span io.Reader, cfg Config, ciphertextSize, start, end int64) (*SpanReader, error) {
if err := cfg.validate(); err != nil {
return nil, err
}
Expand All @@ -270,30 +270,13 @@ func NewSpanReader(span io.Reader, cfg Config, ciphertextSize, off, length int64
if err != nil {
return nil, err
}
if off < 0 || length < 0 || off > plaintextLen {
if start < 0 || end < start || start > plaintextLen-1 {
return nil, ErrRange
}

// Clamp the length to what's actually available from off.
effLen := length
if avail := plaintextLen - off; effLen > avail {
effLen = avail
}

if effLen == 0 {
r := &SpanReader{
src: span,
chunkSize: chunkSize,
encChunk: int64(chunkSize) + TagSize,
numChunks: numChunks,
lastCipherLen: lastCipherLen,
total: 0,
remaining: 0,
err: io.EOF,
}
copy(r.base[:], cfg.BaseNonce)
return r, nil
if end > plaintextLen-1 {
end = plaintextLen - 1
}
total := end - start + 1

aead, err := newGCM(cfg.Key)
if err != nil {
Expand All @@ -308,15 +291,15 @@ func NewSpanReader(span io.Reader, cfg Config, ciphertextSize, off, length int64
encChunk: int64(chunkSize) + TagSize,
numChunks: numChunks,
lastCipherLen: lastCipherLen,
total: effLen,
remaining: effLen,
total: total,
remaining: total,
inBuf: make([]byte, chunkSize+TagSize),
outBuf: make([]byte, 0, chunkSize),
}
copy(r.base[:], cfg.BaseNonce)

r.nextChunk = off / int64(chunkSize)
r.skipFirst = int(off - r.nextChunk*int64(chunkSize))
r.nextChunk = start / int64(chunkSize)
r.skipFirst = int(start - r.nextChunk*int64(chunkSize))
r.onFirst = true
return r, nil
}
Expand Down Expand Up @@ -352,10 +335,10 @@ func (r *SpanReader) Read(p []byte) (int, error) {
r.unread = r.unread[n:]
// Unlike Reader.Read, no (n == 0 && r.err != nil) guard is needed here:
// nextSpanChunk always yields at least one byte. It only runs while
// remaining > 0, which implies off < plaintextLen, so the first chunk
// holds more plaintext than skipFirst trims, and every later chunk in
// the range is non-empty by the stream geometry. (The zero-length range
// never gets here — the constructor pre-sets io.EOF.)
// remaining > 0, which implies start <= the last plaintext byte, so the
// first chunk holds more plaintext than skipFirst trims, and every later
// chunk in the range is non-empty by the stream geometry. (An empty
// range never gets here — the constructor rejects it with ErrRange.)
return n, nil
}

Expand Down Expand Up @@ -416,18 +399,18 @@ func (r *SpanReader) fail(err error) error {
return err
}

// OpenSpan decrypts the plaintext byte range [off, off+length) from a single
// contiguous ciphertext span and returns exactly those bytes. span carries
// the ciphertext range CiphertextRange reports for the same arguments (see
// NewSpanReader); OpenSpan is the one-shot convenience over SpanReader for
// callers that want the range in one buffer.
// OpenSpan decrypts the plaintext byte range [start, end] (inclusive) from a
// single contiguous ciphertext span and returns exactly those bytes. span
// carries the ciphertext range CiphertextRange reports for the same
// arguments (see NewSpanReader); OpenSpan is the one-shot convenience over
// SpanReader for callers that want the range in one buffer.
//
// length is clamped to the bytes available from off, and like
// SpanReader.Read it reports tampering or reordering (ErrCorrupted) or a
// short span (ErrShortSpan) as a non-nil error, in which case the returned
// bytes must be discarded.
func OpenSpan(cfg Config, span io.Reader, ciphertextSize, off, length int64) ([]byte, error) {
r, err := NewSpanReader(span, cfg, ciphertextSize, off, length)
// start/end follow HTTP Range semantics (an over-long end clamps; see
// ErrRange), and like SpanReader.Read it reports tampering or reordering
// (ErrCorrupted) or a short span (ErrShortSpan) as a non-nil error, in which
// case the returned bytes must be discarded.
func OpenSpan(cfg Config, span io.Reader, ciphertextSize, start, end int64) ([]byte, error) {
r, err := NewSpanReader(span, cfg, ciphertextSize, start, end)
if err != nil {
return nil, err
}
Expand Down
Loading
Loading