fix(security): harden untrusted deserialization paths

Author: chaokunyang

.agents/languages/dart.md

@@ -95,9 +95,9 @@ Load this file when changing `dart/`.
   formulas.
   Treat the option as an approximate collection/map/array/struct/object gate, not an exact heap
   cap. Leaf values skipped by graph budgeting remain gated by unread input bytes.
-- Do not add parallel header-low/header-high slot caches or multi-slot recent caches in TypeMeta hot paths to chase benchmark gaps. Header-cache hits must use the concrete checked cache owner directly; if a hit hint is needed, cache one TypeInfo/TypeMeta object and compare the validated header identity on that object, not separate low/high header fields or benchmark-pattern state.
-- If Dart TypeMeta cache ownership changes, keep the invariant in a source comment near the hit path: a checked metadata-cache hit skips the body and must not grow low-bit sentinels, accepted-header fields, parallel header slots, or benchmark-pattern state.
-- Dart expected-type TypeDef reads should compare the expected `TypeInfo` object's cached local TypeDef header before consulting the parsed-metadata map. A match is a direct local-schema hit: skip the remote body, add the expected type to the per-read shared type table, and do not publish to `ParsedTypeMetaCache`, record a remote schema version, or parse/hash the body.
+- Do not add parallel header-low/header-high slot caches or multi-slot recent caches in TypeMeta hot paths to chase benchmark gaps. Header-cache hits must use the concrete checked cache owner directly; if a hit hint is needed, cache one TypeInfo/TypeMeta object and compare the protocol-defined top 52 header bits on that object, not separate low/high header fields or benchmark-pattern state.
+- The top 52 TypeDef/TypeMeta header bits are the schema identity. The full low 12 bits belong only to the current frame and must not participate in hit selection. On a hit, decode the current body size from its low eight bits and any extended-size varuint, prove those bytes readable, and skip exactly that body. Do not validate reserved/compress flags, compare cached or local low bits, parse or rehash the body, repeat schema or policy validation, or grow low-bit sentinels, accepted-header fields, parallel header slots, or benchmark-pattern state. The cold miss path owns low-flag validation.
+- Dart expected-type TypeDef reads should compare only the top 52 bits of the expected `TypeInfo` object's cached local TypeDef header before consulting the parsed-metadata map. A match is a direct local-schema hit: use the current frame's size encoding only for bounds and skip, add the expected type to the per-read shared type table, and do not validate its low flags, publish to `ParsedTypeMetaCache`, record a remote schema version, or parse/hash the body.
 - Dart local TypeDef construction is registration-owned: record registrations
   and finalize their dependent TypeDefs and struct serializers before the first
   root read or write. The first `serialize`, `serializeTo`, `serializeBuiltin`,

.agents/languages/rust.md

@@ -7,6 +7,7 @@ Load this file when changing `rust/` or Rust xlang behavior.
 - Run all cargo commands from within `rust/`.
 - Changes under `rust/` must pass `clippy` and tests.
 - Rust code must compile without compiler or Clippy warnings. Treat warnings as blockers and keep `cargo clippy --all-targets --all-features -- -D warnings` passing.
+- `check_string_read(false)` is a trusted-input-only performance option. It deliberately skips UTF-8 validation, so callers that enable it own the guarantee that serialized string bytes are valid UTF-8. Keep the default checked mode for untrusted input, and do not treat misuse of the unchecked option as a default-path deserialization vulnerability.
 - Use `RUST_BACKTRACE=1 FORY_PANIC_ON_ERROR=1` when debugging failing Rust tests.
 - Add `-- --nocapture` when you need test output during debugging.
 - Do not set `FORY_PANIC_ON_ERROR=1` when running the full Rust test suite, because some tests assert on error contents.

.agents/repo-reference.md

@@ -58,21 +58,21 @@ Apache Fory is a multi-language serialization framework with multiple wire forma
   versions; compatible named enum/ext/union metadata normally has one version but still counts
   against remote metadata total limits when it is sent as shared metadata. Pure id-based enum, ext,
   and typed-union values use type id plus user type id and must not be moved onto this metadata
-  cache path. Exact-local metadata bypass is not struct-only: after the existing type and
-  deserialization-policy checks for the selected local type, compare the original received encoded
-  bytes with the local encoded metadata bytes, and allow exact matches for struct and named
-  enum/ext/union metadata without consuming remote schema-version limits. Derive the local
-  exact-match candidate inside the metadata owner from the decoded `userTypeId` or
-  `(namespace, typeName)` identity; do not add caller-threaded expected-type parameters only for
-  this check in any runtime. Java and Python may lazy-build local metadata bytes only after this
-  identity lookup selects a local class and the existing class, registration, and deserialization
-  policy checks have run. When a typed read path already has a declared local type and validates the
-  decoded metadata identity against it, use that same local type for exact-local byte comparison and
-  cache publish; dynamic or Any paths resolve the local type from decoded metadata. A runtime may
-  also skip a received metadata body when the current declared type already owns an identical local
-  metadata header; that is a local-schema hit, not a remote cache publish, and it must not consume
-  schema-version limits. Other header-only skips are allowed only after the same owning remote
-  metadata cache has validated a previous body for that header.
+  cache path. The protocol-defined 52-bit TypeDef/TypeMeta header hash is the unique schema
+  identity. When the selected local type already owns the received header, that is a local-schema
+  hit: skip the body and use the local metadata without body comparison, cache publication, or
+  schema-version counting. A checked remote-cache hit likewise skips the body without rehashing,
+  byte comparison, repeated validation, or policy work. The low 12 bits describe only the current
+  frame; a hit uses its current size for bounds and skip without validating reserved or compression
+  flags. A cache miss is the only path that parses and validates a body. After that first
+  validation, a runtime may compare the received 52-bit hash with lazily built local metadata when
+  no local header was available before the parse; hash equality selects the local owner and may
+  bypass remote schema-version counting without a byte or field comparison.
+  Derive a miss-only local candidate inside the metadata owner from the decoded identity, after
+  existing class, registration, and policy checks. A statically declared reader may pass its
+  concrete expected owner so reference and cache hits can route by owner identity before publish;
+  do not thread expected-type parameters solely to repeat miss-time metadata validation. Do not add
+  parallel accepted-header state or retain metadata bytes to revalidate either path.
 - Remote metadata body and struct field-count limits are also cold-path resource controls.
   `maxTypeMetaBytes` limits one received TypeDef or TypeMeta body excluding the 8-byte header and
   extended-size varint; `maxTypeFields` limits one received struct metadata body's field count

AGENTS.md

@@ -33,6 +33,8 @@ This is the entry point for AI guidance in Apache Fory. Read this file first, th
 - Do not change an existing `RefReader`/`RefWriter` architecture or API to support compatible skip. Compatible skip must not add alternate reference slots or tables, alternate reference lookup or publication methods, or forwarding APIs in read/write contexts, builders, serializers, or generated-code plumbing. Keep ordinary reference publication and lookup unchanged and resolve the case in the existing compatible generated owner. For an authorized removed-field read of an unregistered Struct, the empty object created by the skip reader is that path's final owner: publish that same object for `RefValue`, consume the Struct fields, and let later `RefFlag` values resolve to it. This preserves reference numbering and identity without registering the Struct; an independent dynamic root still requires normal registration. Do not add parallel reference state, a sentinel, a rejection, or a common-path branch for this case.
 - Respect ownership. Keep logic, state, and helpers in their natural owner, and do not move serializer-local, context-local, runtime-type-local, or protocol-local problems into global utilities.
 - Check the spec before implementation. For wire behavior and xlang mapping, use the specs as the source of truth and never copy one runtime's bug into another runtime just to make tests pass.
+- `foryc` is a build-time compiler for trusted schema inputs and is never invoked by runtime serialization or deserialization. Schema provenance, package/namespace options, output-path options, and generated-source review belong to the application or build owner. Do not classify hostile-schema source injection or path traversal as a Fory runtime security vulnerability; `foryc` does not promise to sandbox untrusted schemas.
+- Row format accepts only trusted input and is outside Fory's untrusted binary-deserialization security boundary. Rust `check_string_read(false)` is likewise an explicit trusted-input mode that disables UTF-8 validation; its caller owns the validity guarantee. Classify issues in those paths as correctness, soundness, or hardening bugs when applicable, not as attacker-controlled deserialization vulnerabilities under the default security model.
 - Do not make assumptions about runtime behavior, ownership, registration, metadata construction, protocol semantics, or test coverage. Read the current code, owning docs/specs, and relevant tests before making a design judgment or implementation decision. If the evidence is incomplete, inspect more or state the uncertainty explicitly instead of filling gaps from memory or analogy with another runtime.
 - For untrusted deserialization, read `docs/security/deserialization.md` before changing allocation, stream filling, skip, reference, metadata, or policy validation behavior. Variable-length deserialization must not allocate or reserve backing/output capacity from attacker-declared lengths or counts before the byte owner has proven proportional readable bytes with `checkReadableBytes` or the runtime equivalent. Root graph memory reservation is accounting only and may happen before that byte check, but it must not replace the byte check.
 - Malformed input must surface as a controlled root-operation error and still run
@@ -139,7 +141,8 @@ This is the entry point for AI guidance in Apache Fory. Read this file first, th
   or map entry may instead advance because of ref, null, or type envelopes; name those derived
   facts `fieldReadAlwaysAdvances`, `elementReadAlwaysAdvances`, or `entryReadAlwaysAdvances` rather
   than conflating them with `readData`.
-- For remote TypeDef/TypeMeta reads, the checked metadata cache is the only owner of remote "already validated" state. Cache hit means the header was previously parsed, body/hash-validated, policy-checked, and published by that cache, so the hot path must skip the body and use cached metadata without extra validation, hashing, limit checks, exact-local checks, allocation, or policy work. The protocol-defined 52-bit TypeDef/TypeMeta header hash is the unique schema identity, so a known expected local header/hash match is a local-schema hit and must not recompare field arrays or metadata bodies. It may skip the body and use the local TypeInfo/TypeMeta without schema-version counting or cache publish. Cache miss is the only path that parses and validates non-local metadata, enforces limits, and publishes remote metadata to the cache. Do not add nullable accepted-header fields, sentinel headers, per-TypeInfo markers, pending metadata state, parallel header-low/header-high slots, or parallel acceptance state for this decision. If a runtime needs a metadata hit hint, cache the concrete checked metadata owner object, such as the TypeInfo, TypeDef, or TypeMeta used by that runtime, and compare its validated header identity directly.
+- For remote TypeDef/TypeMeta reads, the checked metadata cache is the only owner of remote "already validated" state. Cache hit means the header was previously parsed, body/hash-validated, policy-checked, and published by that cache, so the hot path must skip the body and use cached metadata without extra validation, hashing, limit checks, exact-local checks, allocation, or policy work. The protocol-defined 52-bit TypeDef/TypeMeta header hash is the unique schema identity, so a known expected local header/hash match is a local-schema hit and must not recompare field arrays or metadata bodies. The low 12 header bits belong only to the current frame; on a hit, use its current size and optional extension for bounds and skip, but do not validate its reserved or compression flags. A local hit uses the local TypeInfo/TypeMeta without schema-version counting or cache publish. Cache miss is the only path that parses and validates non-local metadata, including low flags, and enforces limits. If the local header becomes available only after that first parse, compare its 52-bit hash with the validated received hash; equality selects the local owner without a second byte or field comparison. Only a non-local miss publishes remote metadata to the cache. Do not add nullable accepted-header fields, sentinel headers, per-TypeInfo markers, pending metadata state, parallel header-low/header-high slots, or parallel acceptance state for this decision. If a runtime needs a metadata hit hint, cache the concrete checked metadata owner object, such as the TypeInfo, TypeDef, or TypeMeta used by that runtime, and compare its validated header identity directly.
+- Checked MetaString caches follow the same rule: validate and publish only on cache miss; on cache hit, skip the encoded body and use the cached value without rehashing, comparing body bytes, or repeating validation. The protocol-defined wire hash alone is the MetaString cache identity; the current frame length is used only for bounds checking and advancing the reader, and must not participate in hit selection. Do not add hit-time byte or length comparison or parallel acceptance state for MetaString caches.
 - When a user corrects a non-obvious invariant, encode it in the nearest source comment before continuing, and also update `AGENTS.md`, `.agents/**`, docs, or specs when the rule is reusable beyond one file. Do not rely only on chat history, task notes, commit messages, or benchmark logs for corrections that protect security, protocol behavior, ownership, naming, or hot-path performance.
 - Reject semantic hacks. Do not bypass broken semantics by deleting cases, simplifying callers, adding coercion hooks, or using workaround fallbacks; fix the underlying bug and prove it with focused tests.
 - Protect hot paths. Avoid per-call allocations, callback objects, result tuples or records, unnecessary runtime branches, and wrapper-class substitutions in hot codec/runtime paths; prefer conditional imports and allocation-free concrete implementations where they fit the language.

cpp/fory/serialization/collection_serializer.h

@@ -100,11 +100,22 @@ template <typename T> inline constexpr bool need_type_for_collection_elem() {
 template <typename T>
 FORY_ALWAYS_INLINE const TypeInfo *
 read_collection_element_type_info(ReadContext &ctx) {
-  const TypeInfo *type_info = ctx.read_any_type_info(ctx.error());
+  using ElemType = nullable_element_t<T>;
+  const TypeInfo *expected_type_info = nullptr;
+  if constexpr (is_user_type(Serializer<ElemType>::type_id)) {
+    auto expected_result =
+        ctx.type_resolver().template get_type_info<ElemType>();
+    if (FORY_PREDICT_FALSE(!expected_result.ok())) {
+      ctx.set_error(std::move(expected_result).error());
+      return nullptr;
+    }
+    expected_type_info = expected_result.value();
+  }
+  const TypeInfo *type_info =
+      ctx.read_any_type_info_owner(ctx.error(), expected_type_info);
   if (FORY_PREDICT_FALSE(ctx.has_error())) {
     return nullptr;
   }
-  using ElemType = nullable_element_t<T>;
   constexpr uint32_t expected =
       static_cast<uint32_t>(Serializer<ElemType>::type_id);
   if (FORY_PREDICT_FALSE(!type_id_matches(type_info->type_id, expected))) {
@@ -796,7 +807,11 @@ inline Container read_collection_data_slow(ReadContext &ctx, uint32_t length) {
   const TypeInfo *elem_type_info = nullptr;
   Harness::ReadAsFn elem_reader = nullptr;
   if (is_same_type && !is_decl_type) {
-    elem_type_info = ctx.read_any_type_info(ctx.error());
+    if constexpr (elem_is_polymorphic) {
+      elem_type_info = ctx.read_any_type_info(ctx.error());
+    } else {
+      elem_type_info = read_collection_element_type_info<T>(ctx);
+    }
     if (FORY_PREDICT_FALSE(ctx.has_error())) {
       return result;
     }