[PROPOSAL] Khronos Avatar Extensions - Phase 1

extensions/2.0/Khronos/KHR_avatar/README.md

@@ -0,0 +1,151 @@
+# KHR_avatar
+
+## Contributors
+
+- Ken Jakubzak, Meta
+- Hideaki Eguchi / VirtualCast, Inc.
+- K. S. Ernest (iFire) Lee, Individual Contributor / https://github.com/fire
+- Shinnosuke Iwaki / VirtualCast, Inc.
+- 0b5vr / pixiv Inc.
+- Leonard Daly, Individual Contributor
+
+## Status
+
+**Draft** – This extension is not yet ratified by the Khronos Group and is subject to change.
+
+## Dependencies
+
+Written against the glTF 2.0 specification.
+
+Dependencies: `KHR_xmp_json_ld`
+This extension also leverages the `KHR_xmp_json_ld` pattern for attaching extensible metadata as JSON-LD blocks within glTF assets. For background on this approach, see:  
+[KHR_xmp_json_ld](https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_xmp_json_ld)
+
+## Overview
+
+The `KHR_avatar` extension designates a glTF asset as representing an avatar. This top-level marker enables tools and runtimes to interpret the asset as containing avatar-specific content such as rigging, blendshapes, animation retargeting, or metadata.
+
+This extension does not define avatar features directly but acts as a root declaration that avatar-related extensions may be present, and that consumers should treat the asset using avatar-specific logic and pipelines. It's part of the wider set of KHR avatar extensions that are meant to be building blocks to represent a contract stating functionality and data requirements between a given model and an endpoint.
+
+The extension supports referencing the source `scene` that represents the avatar and optionally includes structured metadata through the `KHR_xmp_json_ld` mechanism.
+
+## Extension Schema
+
+```json
+{
+  "extensions": {
+    "KHR_avatar": {
+      "sceneIndex": 0
+    }
+  }
+}
+```
+
+### Properties
+
+| Property     | Type    | Description                                                                 |
+|--------------|---------|-----------------------------------------------------------------------------|
+| `sceneIndex` | integer | Index of the glTF `scene` representing the avatar. Used to distinguish the avatar root when multiple scenes exist. |
+
+## Metadata Attachment: JSON_XMP_LD
+
+Avatar metadata should be expressed using the `KHR_xmp_json_ld` format, a structured mechanism for attaching JSON-LD metadata blocks to glTF files. In the context of `KHR_avatar`, this allows consistent expression of avatar provenance, licensing, creator, versioning, and intended use, among others.
+
+The `KHR_xmp_json_ld` block is placed at the root level of the glTF asset as part of the defined extension usage. Metadata keys and structures are defined in the shared Khronos Avatar Metadata schema (TBD).
+
+| DC/XMP_JSON_LD Property | Why                                                                          | Required |
+|-------------------------|------------------------------------------------------------------------------|----------|
+| dc:title                |                                                                              | Yes      |
+| dc:creator              |                                                                              | Yes      |
+| dc:license              |                                                                              | No       |
+| dc:rights               |                                                                              | No       |
+| dc:created              | Date on which the asset was created                                          | No       |
+| dc:publisher            | Identifies the entity responsible for making the resource available; important for understanding the source and authority of the content. | No       |
+| dc:description          | Context and a summary of the content                                         | No       |
+| dc:subject              | Can potentially be used for content tagging/association                      | No       |
+| dc:source               | Important for tracing the provenance and ensuring proper attribution.        | Yes      |
+| khr:version             |                                                                              | No       |
+| khr:thumbnailImage      |                                                                              | No       |
+
+## Example
+
+```json
+{
+  "asset": {
+    "version": "2.0"
+  },
+  "scene": 0,
+  "scenes": [
+    {
+      "nodes": [0]
+    }
+  ],
+  "nodes": [
+    {
+      "name": "AvatarRoot"
+    }
+  ],
+  "extensionsUsed": [
+    "KHR_avatar",
+    "KHR_xmp_json_ld"
+  ],
+  "extensions": {
+    "KHR_avatar": {
+      "sceneIndex": 0
+    },
+
+    "KHR_xmp_json_ld": {
+    "packets": [
+      {
+  "@context": {
+    "dc": "http://purl.org/dc/elements/1.1/",
+    "vrm": "https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_vrm-1.0/meta.md"
+  },
+  "dc:title": "Example Model",
+  "dc:creator": {
+    "@list": [
+      "Author1",
+      "AuthorEmail1@email.com",
+      "Author2",
+      "AuthorEmail2@email.com"
+    ]
+  },
+  "dc:license": {
+    "@list": [
+      "https://vrm.dev/licenses/1.0/",
+      "https://example.com/third-party-license"
+    ]
+  },
+  "dc:created": "2023-05-05",
+  "dc:rights": "Copyright information about the model",
+  "dc:publisher": "Imaginary Corporation A, LLC",
+  "dc:description": "A sentence, or paragraph describing the avatar at hand",
+  "dc:subject": {
+    "@list": [
+      "Example trait",
+      "Another example trait"
+    ]
+  },
+  "dc:source": "imaginaryCompany.com/avatarl",
+  "khr:version": "1.0",
+  "khr:thumbnailImage": 0
+}
+    ]
+    }
+  }
+}
+```
+
+## Implementation Notes
+
+- `sceneIndex` is required and indicates what scene a particular avatar belongs to
+- Consumers should use this marker as a signal to search for additional avatar-related extensions, including skeletal, expression, and other khronos avatar extensions.
+- Support for `JSON_XMP_LD` is encouraged to ensure interoperable metadata across tools and runtimes.
+
+## Known Implementations
+
+
+## License
+
+This extension specification is licensed under the Khronos Group Extension License.  
+See: https://www.khronos.org/registry/gltf/license.html

extensions/2.0/Khronos/KHR_avatar_expression_joint/README.md

@@ -0,0 +1,101 @@
+# KHR_avatar_expression_joint
+
+## Contributors
+
+- Ken Jakubzak, Meta
+- Hideaki Eguchi / VirtualCast, Inc.
+- K. S. Ernest (iFire) Lee, Individual Contributor / https://github.com/fire
+- Shinnosuke Iwaki / VirtualCast, Inc.
+- 0b5vr / pixiv Inc.
+- Leonard Daly, Individual Contributor
+
+## Status
+
+**Draft** – This extension is not yet ratified by the Khronos Group and is subject to change.
+
+## Dependencies
+
+Written against the glTF 2.0 specification.    
+Dependent on: `KHR_avatar`
+Typically used in conjunction with: `KHR_avatar_expression_mapping`
+
+## Overview
+
+The `KHR_avatar_expression_joint` extension provides a semantic mapping between facial expressions and joint transformations in the glTF node hierarchy. It enables tools and runtimes to associate expressions like `blink`, `smile`, or `jawOpen` with specific nodes whose transforms are animated using standard glTF animation channels.
+
+This extension is purely descriptive: it does not define or store animation data itself.
+
+## Expression Vocabulary
+
+Expression types include:
+
+- **Emotions**: `happy`, `angry`, `surprised`, etc.  
+- **Visemes**: `aa`, `ee`, `th`, `oo`, etc.  
+- **Modifiers**: `left`, `right`, `upper`, `lower`, etc.  
+- **Gestures and Actions**: `blink`, `smile`, `jawOpen`, etc.
+
+Optionally, these may be aligned with industry standards, such as [Facial Action Coding System (FACS)](https://en.wikipedia.org/wiki/Facial_Action_Coding_System). 
+
+## Extension Schema
+
+```json
+{
+ "extensions": {
+    "KHR_avatar_expression_mapping": {
+      "expressions": [
+        {
+          "expression": "smile",
+          "animation": 0,
+          "channels": [0,1,2]
+        },
+        {
+          "expression": "frown",
+          "animation": 1,
+          "channels": [0,1]
+        }
+      ]
+    }
+  }
+}
+```
+
+### Properties
+
+| Property     | Type    | Description                                                                 |
+|--------------|---------|-----------------------------------------------------------------------------|
+| `expressions`| array   | Array of mappings between animation/channels and expression labels.               |
+| `animation`  | integer | Index into the glTF `animations[]` array representing an expression animation.     |
+| `expression` | string  | Expression name this joint contributes to.                                  |
+| `channels`   | array   | array representing channels that must correspond to either `"rotation"`, `"translation"`, or `"scale"`; indicates transform types. |
+
+## Animation Integration
+
+- Expression timing, blending, and control must use glTF `animations` channels.
+- Animations targeting expression-driven `rotation`, `translation`, or `scale` must conform to glTF 2.0's animation model.
+- This ensures consistency, ease of implementation, and interoperability across runtimes.
+
+Each animation channel used to drive an expression should operate within a **normalized 0-to-1 range**, where:
+- `0.0` indicates the expression is fully inactive.
+- `1.0` indicates the expression is fully active.
+
+The transformation values themselves (e.g., degree of rotation or distance of translation) should scale proportionally with the normalized input range.
+
+This approach simplifies avatar implementation by centralizing expression playback in the glTF animation system and unifying runtime logic for blending and prioritization.
+
+### Recommended Interpolation for Binary Expressions
+
+For expressions that represent binary or toggle states (such as `blinkLeft`, `blinkRight`, or `jawOpen`), the use of glTF animation channels with `"interpolation": "STEP"` is strongly recommended.
+
+STEP interpolation ensures that an expression toggles cleanly between fully off (`0.0`) and fully on (`1.0`) states, providing crisp visual transitions and avoiding interpolation artifacts that could occur with `LINEAR` interpolation in binary scenarios.
+
+
+## Implementation Notes
+
+- Multiple joints may be assigned to the same expression.
+- Expression states should be normalized to the [0.0–1.0] range for consistent runtime interpretation.
+- This extension does not conflict with standard rigging or skinning systems.
+
+## License
+
+This extension is licensed under the Khronos Group Extension License.  
+See: https://www.khronos.org/registry/gltf/license.html

extensions/2.0/Khronos/KHR_avatar_expression_mapping/README.md

@@ -0,0 +1,104 @@
+# KHR_avatar_expression_mapping
+
+## Contributors
+
+- Ken Jakubzak, Meta
+- Hideaki Eguchi / VirtualCast, Inc.
+- K. S. Ernest (iFire) Lee, Individual Contributor / https://github.com/fire
+- Shinnosuke Iwaki / VirtualCast, Inc.
+- 0b5vr / pixiv Inc.
+- Leonard Daly, Individual Contributor
+
+## Status
+
+**Draft** – This extension is not yet ratified by the Khronos Group and is subject to change.
+
+## Dependencies
+
+Written against the glTF 2.0 specification.    
+Dependent on: `KHR_avatar`
+Can be used alongside: `KHR_avatar_expression_morphtargets` or other expression sources
+
+## Overview
+
+
+## Reference Expression Vocabulary
+Expression names may be grouped into categories including:
+
+- **Emotions** (e.g., `happy`, `angry`, `surprised`)
+- **Visemes** (e.g., `aa`, `oo`, `th`)
+- **Modifiers** (e.g., `left`, `right`, `upper`, `lower`)
+- **Gestures and Actions** (e.g., `blink`, `smile`, `jawOpen`)
+
+Implementers are encouraged to use this vocabulary directly or map custom expressions to it using `KHR_avatar_expression_mapping`.
+
+The `KHR_avatar_expression_mapping` extension provides a general-purpose mechanism for mapping expression names used in an avatar’s mesh to a known expression vocabulary or rig specification. This allows different authoring pipelines or runtimes to translate between heterogeneous expression sets.
+
+## Extension Schema
+
+```json
+{
+  "extensions": {
+    "KHR_avatar_expression_mapping": {
+      "mappings": {
+        "smileLeft": [
+          { "target": "Smile", "weight": 0.8 },
+          { "target": "LeftCheekRaise", "weight": 0.2 }
+        ],
+        "jawOpen": [
+          { "target": "MouthOpen", "weight": 1.0 }
+        ]
+      }
+    }
+  }
+}
+```
+
+### Properties
+
+| Property     | Type    | Description                                                                 |
+|--------------|---------|-----------------------------------------------------------------------------|
+| `mappings`   | object  | Dictionary mapping expression names to reference vocabulary terms.          |
+| `target`     | string  | Name of the expression in the target vocabulary.                            |
+| `weight`     | number  | Influence of this target. Must sum to 1.0 per expression key.               |
+
+
+### Mapping Types
+
+This extension supports both one-to-one and one-to-many mappings:
+
+- **One-to-one**: An expression maps directly to a single reference vocabulary term with weight 1.0.
+- **One-to-many**: An expression is composed from multiple reference terms, blended using assigned weights.
+
+This allows developers to bridge between custom expression sets and shared vocabularies.
+
+## Implementation Notes
+
+- This extension is typically used at the top level of the glTF file.
+- Expression names should match those used in `KHR_avatar_expression_morphtargets`, animation tracks, or tracking pipelines.
+- Tools can interpret this mapping to apply automatic translation between expression sets.
+
+## Example
+
+```json
+{
+  "extensionsUsed": [
+    "KHR_avatar_expression_mapping"
+  ],
+  "extensions": {
+    "KHR_avatar_expression_mapping": {
+      "mappings": {
+        "smileLeft": [
+          { "target": "Smile", "weight": 0.8 },
+          { "target": "LeftCheekRaise", "weight": 0.2 }
+        ]
+      }
+    }
+  }
+}
+```
+
+## License
+
+This extension specification is licensed under the Khronos Group Extension License.  
+See: https://www.khronos.org/registry/gltf/license.html