[PROPOSAL] Khronos Avatar Extensions - Phase 1

extensions/2.0/Khronos/KHR_character/README.md

@@ -0,0 +1,153 @@
+# KHR_character
+
+## Contributors
+
+- Ken Jakubzak, Meta
+- Hideaki Eguchi / VirtualCast, Inc.
+- K. S. Ernest (iFire) Lee, Independent Contributor / https://github.com/fire
+- Shinnosuke Iwaki / VirtualCast, Inc.
+- 0b5vr / pixiv Inc.
+- Leonard Daly, Independent Contributor
+- Nick Burkard, Meta
+- Sarah Cooney, Microsoft XGTG
+- Aaron Franke, Independent Contributor
+
+## Status
+
+**Draft** – This extension is not yet ratified by the Khronos Group and is subject to change.
+
+## Dependencies
+
+Written against the glTF 2.0 specification.
+
+Requires the extensions: `KHR_xmp_json_ld`
+
+This extension also leverages the `KHR_xmp_json_ld` pattern for attaching extensible metadata as JSON-LD blocks within glTF assets. For background on this approach, see:
+[KHR_xmp_json_ld](https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_xmp_json_ld)
+
+## Overview
+
+The `KHR_character` extension designates a glTF asset as representing an character. This top-level marker enables tools and runtimes to interpret the asset as containing character-specific content such as rigging, blendshapes, animation retargeting, or metadata.
+
+This extension does not define character features directly but acts as a root declaration that character-related extensions may be present, and that consumers should treat the asset using character-specific logic and pipelines. It's part of the wider set of KHR character extensions that are meant to be building blocks to represent a contract stating functionality and data requirements between a given model and an endpoint.
+
+The extension supports referencing the root `node` that represents the character and optionally includes structured metadata through the `KHR_xmp_json_ld` mechanism.
+
+## Extension Schema
+
+```json
+{
+  "extensions": {
+    "KHR_character": {
+      "rootNode": 0
+    }
+  }
+}
+```
+
+### Properties
+
+| Property   | Type    | Description                                                                                                                                                                                  |
+| ---------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `rootNode` | integer | Index of the glTF `node` representing the root of the character hierarchy. This node should be a common ancestor of all nodes containing character-related data, such as meshes, skins, and animations. |
+
+## Metadata Attachment: KHR_xmp_json_ld
+
+character metadata should be expressed using the `KHR_xmp_json_ld` format, a structured mechanism for attaching JSON-LD metadata blocks to glTF files. In the context of `KHR_character`, this allows consistent expression of character provenance, licensing, creator, versioning, and intended use, among others.
+
+The `KHR_xmp_json_ld` block is placed at the root level of the glTF asset as part of the defined extension usage. Metadata keys and structures are defined in the shared Khronos character Metadata schema (TBD).
+
+| DC/XMP_JSON_LD Property | Why                                                                                                                                       | Required |
+| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -------- |
+| dc:title                |                                                                                                                                           | Yes      |
+| dc:creator              |                                                                                                                                           | Yes      |
+| dc:license              |                                                                                                                                           | No       |
+| dc:rights               |                                                                                                                                           | No       |
+| dc:created              | Date on which the asset was created                                                                                                       | No       |
+| dc:publisher            | Identifies the entity responsible for making the resource available; important for understanding the source and authority of the content. | No       |
+| dc:description          | Context and a summary of the content                                                                                                      | No       |
+| dc:subject              | Can potentially be used for content tagging/association                                                                                   | No       |
+| dc:source               | Important for tracing the provenance and ensuring proper attribution.                                                                     | Yes      |
+| khr:version             |                                                                                                                                           | No       |
+| khr:thumbnailImage      |                                                                                                                                           | No       |
+
+## Example
+
+```json
+{
+  "asset": {
+    "version": "2.0",
+    "extensions": {
+      "KHR_xmp_json_ld": {
+        "packet": 0
+      }
+    }
+  },
+  "scene": 0,
+  "scenes": [
+    {
+      "nodes": [0]
+    }
+  ],
+  "nodes": [
+    {
+      "name": "characterRoot"
+    }
+  ],
+  "extensionsUsed": ["KHR_character", "KHR_xmp_json_ld"],
+  "extensions": {
+    "KHR_character": {
+      "rootNode": 0
+    },
+
+    "KHR_xmp_json_ld": {
+      "packets": [
+        {
+          "@context": {
+            "dc": "http://purl.org/dc/elements/1.1/",
+            "vrm": "https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_vrm-1.0/meta.md"
+          },
+          "dc:title": "Example Model",
+          "dc:creator": {
+            "@list": [
+              "Author1",
+              "AuthorEmail1@email.com",
+              "Author2",
+              "AuthorEmail2@email.com"
+            ]
+          },
+          "dc:license": {
+            "@list": [
+              "https://vrm.dev/licenses/1.0/",
+              "https://example.com/third-party-license"
+            ]
+          },
+          "dc:created": "2023-05-05",
+          "dc:rights": "Copyright information about the model",
+          "dc:publisher": "Imaginary Corporation A, LLC",
+          "dc:description": "A sentence, or paragraph describing the character at hand",
+          "dc:subject": {
+            "@list": ["Example trait", "Another example trait"]
+          },
+          "dc:source": "imaginaryCompany.com/characterl",
+          "khr:version": "1.0",
+          "khr:thumbnailImage": 0
+        }
+      ]
+    }
+  }
+}
+```
+
+## Implementation Notes
+
+- `rootNode` is required, representing the index of the glTF `node` that serves as the root of the character hierarchy. This node should be a common ancestor of all nodes containing character-related data, such as meshes, skins, and animations.
+- Consumers should use this marker as a signal to search for additional character-related extensions, including skeletal, expression, and other khronos character extensions.
+- Support for `KHR_xmp_json_ld` is encouraged to ensure interoperable metadata across tools and runtimes.
+
+## Known Implementations
+
+## License
+
+This extension specification is licensed under the Khronos Group Extension License.
+See: https://www.khronos.org/registry/gltf/license.html

extensions/2.0/Khronos/KHR_character/schema/glTF.KHR_character.schema.json

@@ -0,0 +1,18 @@
+{
+    "$schema": "http://json-schema.org/draft-04/schema",
+    "title": "KHR_character glTF Document Extension",
+    "type": "object",
+    "description": "glTF extension for character-specific metadata and properties.",
+    "allOf": [ { "$ref": "glTFProperty.schema.json" } ],
+    "properties": {
+        "rootNode": {
+            "allOf": [{ "$ref": "glTFid.schema.json" }],
+            "description": "Index of the glTF node representing the root of the character hierarchy. This node should be a common ancestor of all nodes containing character-related data, such as meshes, skins, and animations."
+        },
+        "extensions": { },
+        "extras": { }
+    },
+    "required": [
+        "rootNode"
+    ]
+}

extensions/2.0/Khronos/KHR_character_expression/README.md

@@ -0,0 +1,144 @@
+# KHR_character_expression
+
+## Contributors
+
+- Ken Jakubzak, Meta
+- Hideaki Eguchi / VirtualCast, Inc.
+- K. S. Ernest (iFire) Lee, Independent Contributor / https://github.com/fire
+- Shinnosuke Iwaki / VirtualCast, Inc.
+- 0b5vr / pixiv Inc.
+- Leonard Daly, Independent Contributor
+- Nick Burkard, Meta
+- Sarah Cooney, Microsoft XGTG
+- Aaron Franke, Independent Contributor
+
+## Status
+
+**Draft** – This extension is not yet ratified by the Khronos Group and is subject to change.
+
+## Dependencies
+
+Written against the glTF 2.0 specification.
+Requires the extension(s): `KHR_character`
+
+## Overview
+
+The `KHR_character_expression` extension provides a common interface for facial expression animations. It enables tools and runtimes to associate expressions like `blink`, `smile`, or `jawOpen` with specific animations in the glTF model's animations field.
+
+When used in conjunction with the other expression extensions, enables a data contract with endpoints allowing them to understand just what kind of data is present and being powered.
+
+This extension is purely descriptive: it does not define or store animation data itself.
+
+## Reference Expression Categories/Vocabularies
+
+Expressions in this context describe face-localized animations used to drive small and/or larger movements across the face and/or down-chain meshes needed for reasonable conveyance of emotion/intent.
+
+For examples of relevant types of expressions, you can reference concepts such as:
+
+- **Emotions** (Emotion-derived facial movements such as what [VRM defines as presets](https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_vrm-1.0/expressions.md), e.g. `happy`, `angry`, `surprised`)
+- **Visemes** (A visual representations of mouth movements for parts of speech, e.g. `aa`, `oo`, `th`)
+- **FACS** ([Facial Action Coding System (FACS)](https://en.wikipedia.org/wiki/Facial_Action_Coding_System) which is a system intended to describe visually distinguishable facial movements (and is often split further based on left/right), e.g. `brow lowerer`, `chin raiser`, `lid droop`)
+- **Gestures and Actions** (Larger descriptors that describe general facial actionse (but not emotion), e.g. `blink`, `smile`, `jawOpen`)
+
+Optionally, these expressions may be aligned with industry standards (or an endpoint/experiences expected expressions set).
+
+## Extension Schema
+
+```json
+{
+  "extensions": {
+    "KHR_character_expression": {
+      "expressions": [
+        {
+          "expression": "smile",
+          "animation": 0
+        },
+        {
+          "expression": "frown",
+          "animation": 1
+        }
+      ]
+    }
+  }
+}
+```
+
+### Properties
+
+| Property      | Type    | Description                                                                    |
+| ------------- | ------- | ------------------------------------------------------------------------------ |
+| `expressions` | array   | Array of mappings between animation/channels and expression labels.            |
+| `animation`   | integer | Index into the glTF `animations[]` array representing an expression animation. |
+| `expression`  | string  | Expression name this joint contributes to.                                     |
+
+## Animation Integration
+
+- Expression timing, blending, and control must use glTF `animations` channels.
+- This ensures consistency, ease of implementation, and interoperability across runtimes.
+
+Each animation channel used to drive an expression should operate within a **normalized 0-to-1 range**, where:
+
+- `0.0` indicates the expression is fully inactive.
+- `1.0` indicates the expression is fully active.
+
+The transformation values themselves (e.g., degree of rotation or distance of translation) should scale proportionally with the normalized input range.
+
+This approach simplifies character implementation by centralizing expression playback in the glTF animation system and unifying runtime logic for blending and prioritization.
+
+### Recommended Interpolation for Binary Expressions
+
+For expressions that represent binary or toggle states (such as `blinkLeft`, `blinkRight`, or `jawOpen`), the use of glTF animation channels with `"interpolation": "STEP"` is strongly recommended.
+
+STEP interpolation ensures that an expression toggles cleanly between fully off (`0.0`) and fully on (`1.0`) states, providing crisp visual transitions and avoiding interpolation artifacts that could occur with `LINEAR` interpolation in binary scenarios.
+
+## Extension Example w/ typed extensions
+
+```json
+{
+  "extensions": {
+    "KHR_character_expression": {
+      "expressions": [
+        {
+          "expression": "smile",
+          "animation": 0,
+          "extensions": {
+            "KHR_avatar_expression_joint": {
+              "channels": [0, 1]
+            },
+            "KHR_avatar_expression_texture": {
+              "channels": [2]
+            },
+            "KHR_avatar_expression_morphtarget": {
+              "channels": [4, 5]
+            }
+          }
+        },
+        {
+          "expression": "frown",
+          "animation": 1,
+          "extensions": {
+            "KHR_avatar_expression_joint": {
+              "channels": [0]
+            },
+            "KHR_avatar_expression_texture": {
+              "channels": [1, 2]
+            },
+            "KHR_avatar_expression_morphtarget": {
+              "channels": [3]
+            }
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+## Implementation Notes
+
+- Expression states should be normalized to the [0.0–1.0] range for consistent runtime interpretation.
+
+## License
+
+This extension is licensed under the Khronos Group Extension License.
+See: https://www.khronos.org/registry/gltf/license.html

extensions/2.0/Khronos/KHR_character_expression/schema/glTF.KHR_character_expression.schema.json

@@ -0,0 +1,40 @@
+{
+    "$schema": "http://json-schema.org/draft-04/schema",
+    "title": "KHR_character_expression glTF Document Extension",
+    "type": "object",
+    "description": "glTF extension that provides a common interface for facial expression animations.",
+    "allOf": [ { "$ref": "glTFProperty.schema.json" } ],
+    "properties": {
+        "expressions": {
+            "type": "array",
+            "description": "Array of mappings between animation/channels and expression labels.",
+            "items": {
+                "type": "object",
+                "properties": {
+                    "expression": {
+                        "type": "string",
+                        "description": "Expression name this animation contributes to."
+                    },
+                    "animation": {
+                        "allOf": [
+                            {
+                                "$ref": "glTFid.schema.json"
+                            }
+                        ],
+                        "description": "Index into the glTF animations array representing an expression animation."
+                    },
+                    "extensions": { },
+                    "extras": { }
+                },
+                "required": ["expression", "animation"],
+                "additionalProperties": false
+            },
+            "minItems": 1
+        },
+        "extensions": { },
+        "extras": { }
+    },
+    "required": [
+        "expressions"
+    ]
+}