From aeb80a93c49e9000eb2fa790760fa67d57ff1895 Mon Sep 17 00:00:00 2001 From: Lean Mendoza Date: Mon, 28 Oct 2024 20:17:32 -0300 Subject: [PATCH 1/5] wip --- content/ADR-253-extension-gltf-container.md | 40 +++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 content/ADR-253-extension-gltf-container.md diff --git a/content/ADR-253-extension-gltf-container.md b/content/ADR-253-extension-gltf-container.md new file mode 100644 index 00000000..c0ed4cd4 --- /dev/null +++ b/content/ADR-253-extension-gltf-container.md @@ -0,0 +1,40 @@ +--- +layout: adr +adr: 250 +title: Extension GLTF functionality in runtime 7 +date: 2024-10-26 +status: Draft +type: Standards Track +spdx-license: CC0-1.0 +authors: +- robftm +- leanmendoza +--- + +# Abstract +This document sets the extension of GLTF functionality, the adopted standard in Decentraland to load 3D scenes, by allowing the access to the internal structure of GLTF/GLB models. This capability enables various functionalities such as the use of pointer events for specific colliders, modification of internal nodes and materials, and fetching animation lists. By exposing these internal components, developers can create more dynamic and interactive experiences within the engine. + +# Context and Motivation +The existing pipeline for using 3D assets in interactive environments lacks the flexibility needed to dynamically control or modify specific elements within a GLTF model. Currently, artists and developers face limitations such as: +- Difficulty creating interactable components within a single GLTF asset, needing separate exports for each interactive model part. +- Inability to directly reference or manipulate internal nodes, meshes, and materials within the GLTF, which complicates asset integration and limits creative potential. +- Challenges in synchronizing code-driven animations, events, and visual modifications with specific parts of a model, impeding advanced customizations. + +This feature is motivated by the need to improve both basic and advanced use cases for creators working with GLTF models: +- For **entry-level creators**: This feature enables direct access to model elements like colliders and transforms without separate exports, supporting quicker, easier setup for interactable components. +- For **advanced developers**: It provides powerful tools to transform internal nodes, repurpose model elements (e.g., using a mesh or material as a resource), and create code-driven animations or effects that modify specific model components dynamically. + +By extending access to these internal resources, this solution provides a deeper level of control, supporting a flexible workflow that integrates more seamlessly with artist pipelines and accelerates creative experimentation. + +# Proposed Solution + +The proposed solution expands various GLTF components, enabling detailed, interactive control over model elements. Key updates include: + +- Modify `GltfLoadingState`: New properties (animationNames, materialNames, meshNames, nodePaths) make it easier to access GLTF’s internal components. These properties let developers quickly reference animation clips, material names, mesh names, and node paths, simplifying animations, interactions, or modifications. +- Modify `GltfContainer`: add the property `internalFeedback` to directly pipe the internal info in the `GltfLoadingState` +- Modify `Material` Component: A `gltf` material reference added to the Material component allows for dynamic use of GLTF materials within a scene. Material properties default to GLTF specifications unless changed, simplifying setup and customization without needing manual adjustments. +- Modify `MeshRenderer` and `MeshCollider`: a `gltf` mesh reference added to these components to support mesh reuse across different entities, enabling interactive development with a single model. +- Add `GltfNode` and `GltfNodeState` Components: The new `GltfNode` and `GltfNodeState` components **let developers map GLTF nodes to scene entities**, allowing independent modification of a node’s Transform, Material, and MeshRenderer. These components stay in sync with GLTF animations, supporting dynamic transformations, event-based interactions, and resource re-use. + +## Usage + From 8de9cb00baaa448797f0817e6bba7624b36552e2 Mon Sep 17 00:00:00 2001 From: Lean Mendoza Date: Fri, 1 Nov 2024 07:05:18 -0300 Subject: [PATCH 2/5] polish --- content/ADR-253-extension-gltf-container.md | 78 +++++++++++++++++++-- 1 file changed, 74 insertions(+), 4 deletions(-) diff --git a/content/ADR-253-extension-gltf-container.md b/content/ADR-253-extension-gltf-container.md index c0ed4cd4..d66e3b85 100644 --- a/content/ADR-253-extension-gltf-container.md +++ b/content/ADR-253-extension-gltf-container.md @@ -21,8 +21,8 @@ The existing pipeline for using 3D assets in interactive environments lacks the - Challenges in synchronizing code-driven animations, events, and visual modifications with specific parts of a model, impeding advanced customizations. This feature is motivated by the need to improve both basic and advanced use cases for creators working with GLTF models: -- For **entry-level creators**: This feature enables direct access to model elements like colliders and transforms without separate exports, supporting quicker, easier setup for interactable components. -- For **advanced developers**: It provides powerful tools to transform internal nodes, repurpose model elements (e.g., using a mesh or material as a resource), and create code-driven animations or effects that modify specific model components dynamically. +- For **entry-level creators**: This feature enables direct access to model elements like colliders and transforms without separate exports, supporting quicker, easier setup for interactable components. For instance, you would be able to export a entire scene, prefix all the nodes you find relevant, and then fetch them in a list to manipulate in the scene-code. +- For **advanced developers**: It provides powerful tools to transform internal nodes, repurpose model elements (e.g., using a mesh or material as a resource), and create code-driven animations or effects that modify specific model components dynamically. For example, you could attach other models to internal part, like a sword to a hand in a NPC model, or you can use a GLTF to instance several meshes with pointing the same resource. By extending access to these internal resources, this solution provides a deeper level of control, supporting a flexible workflow that integrates more seamlessly with artist pipelines and accelerates creative experimentation. @@ -32,9 +32,79 @@ The proposed solution expands various GLTF components, enabling detailed, intera - Modify `GltfLoadingState`: New properties (animationNames, materialNames, meshNames, nodePaths) make it easier to access GLTF’s internal components. These properties let developers quickly reference animation clips, material names, mesh names, and node paths, simplifying animations, interactions, or modifications. - Modify `GltfContainer`: add the property `internalFeedback` to directly pipe the internal info in the `GltfLoadingState` -- Modify `Material` Component: A `gltf` material reference added to the Material component allows for dynamic use of GLTF materials within a scene. Material properties default to GLTF specifications unless changed, simplifying setup and customization without needing manual adjustments. +- Modify `Material` Component: A `gltf` material reference added to the `Material` component allows for dynamic use of GLTF materials within a scene. `Material` properties default to GLTF specifications unless changed, simplifying setup and customization without needing manual adjustments. - Modify `MeshRenderer` and `MeshCollider`: a `gltf` mesh reference added to these components to support mesh reuse across different entities, enabling interactive development with a single model. -- Add `GltfNode` and `GltfNodeState` Components: The new `GltfNode` and `GltfNodeState` components **let developers map GLTF nodes to scene entities**, allowing independent modification of a node’s Transform, Material, and MeshRenderer. These components stay in sync with GLTF animations, supporting dynamic transformations, event-based interactions, and resource re-use. +- Add `GltfNode` and `GltfNodeState` Components: The new `GltfNode` and `GltfNodeState` components **let developers map GLTF nodes to scene entities**, allowing independent modification of a node’s `Transform`, Material, and MeshRenderer. These components stay in sync with GLTF animations, supporting dynamic transformations, event-based interactions, and resource re-use. ## Usage +This specification is designed not only to provide functionality for creators but also to enable the development of advanced tools built upon it. + +The primary and key elements for usage are as follows: + +### Fetching the Structure +When assigning a `GltfContainer` to an entity, and upon successful loading, the `GltfLoadingState` is populated with the complete GLTF scene data. This feedback can be disabled by setting `internalFeedback=false` within the `GltfContainer` (default is `true`). + +#### Generating an Accurate Animation List +As a result of fetching the structure, one common challenge in utilizing animations is ensuring the accuracy of animation names. By retrieving the animation list directly from the GLTF, developers can either copy the string names or map the states through the array. + +For instance, hardcoding each GLTF animation name is impractical when developing dynamic smart items within environments like the In-World-Builder. Obtaining the animation list at runtime is crucial to streamline the direct animation selection pipeline. + +### Accessing a Node from the Internal Tree +Once a GLTF is loaded and instantiated within the world, referencing specific nodes of the GLTF scene becomes possible. The availability of the node list enables efficient identification without the need to reference the entire tree structure. + +To reference a node, add a `GltfNode` component as the parent of the `GltfContainer` and await the `GltfNodeState`. Upon reaching the `READY` state, certain components are added to the entity as follows: +- **Transform**: While set for parenting, this is populated with the relative transform from the `GltfContainer`. +- **MeshRenderer**: For visible meshes, this component is added with the `gltf` option. +- **MeshRenderer**: For invisible meshes, this component is added with the `gltf` option. +- **Material**: When a mesh is affected by a `Material`, this component is added with the `gltf` option. + +While all components are writable, specific considerations apply: +- **Transform**: When an animation influences this node, the transform is disregarded. +- **Material**: If the `gltf` field is set in `Material`, default values fall back to the GLTF material properties. For example, properties such as `albedoColor` or `diffuseColor` default to *WHITE* but defer to the GLTF-defined settings if no custom values are provided. + +### Using GLTF as a Mesh/Material Resource +This specification further allows the use of resources from a GLTF by setting the `gltf` field within `Material`, `MeshRenderer`, or `MeshCollider`. It is recommended to first load the `GltfContainer` to retrieve available meshes, materials, and textures, ensuring that these resources are loaded into memory before use. + +## Specification + +1. **GltfContainer Initialization and Loading State** + - A `GltfContainer` component **MUST** be assigned to an entity to initialize and populate `PBGltfContainerLoadingState` with detailed GLTF structure data. + - `PBGltfContainerLoadingState` **MUST** expose the following properties: + - `node_paths`: All node paths in the GLTF. + - `mesh_names`: All meshes within the GLTF, with unnamed meshes auto-assigned names. + - `material_names`: All materials in the GLTF, with unnamed materials auto-assigned names. + - `skin_names`: All mesh skins in the GLTF, with unnamed skins auto-assigned names. + - `animation_names`: All animations in the GLTF, with unnamed animations auto-assigned names. + - Unnamed assets **SHOULD** be automatically assigned unique names to avoid conflicts, and creators **SHOULD** explicitly name all assets to ensure clarity. + +2. **Adding and Using GltfNode Components** + - A `GltfNode` component **MUST** be added to an entity that is either a direct child of a `GltfContainer` entity or another entity with a `GltfNode`. + - The `path` in `PBGltfNode` **MUST** match the path of one of the `node_paths` available within `PBGltfContainerLoadingState`. + - Upon the renderer attaching `PBGltfNodeState`, the entity **MUST** progress through the following states: + - **`GNSV_PENDING`**: Node is loading. + - **`GNSV_FAILED`**: Node loading failed, with an optional `error` string describing the failure. + - **`GNSV_READY`**: Node is ready and its `Transform`, `MeshRenderer`, `MeshCollider`, and `Material` components **MUST** be populated according to the GLTF data. + - Once `GNSV_READY`, the following behaviors **SHALL** apply: + - `Transform`: **MUST** match the position of the GLTF node relative to its parent. + - `MeshRenderer`: **SHOULD** be added with a `GltfMesh` type if the GLTF node has a mesh. + - `MeshCollider`: **SHOULD** be added with a `GltfMesh` type if the GLTF node has a collider. + - `Material`: **MAY** be added if the node has an associated material. + +3. **Component Modification and Animation Synchronization** + - Modifying a `GltfNode` component’s `Transform`, `MeshRenderer`, `MeshCollider`, or `Material` **MAY** dynamically alter the node as follows: + - **Transform**: **MUST** update based on animation priority, with any ongoing animations taking precedence over manual modifications. + - **Visibility**: **MAY** be toggled to show or hide the node and its children within the GLTF hierarchy. + - **MeshRenderer and MeshCollider**: **MAY** be modified to update or remove meshes and colliders. + - **Material**: **MAY** be modified to adjust material properties, with default values based on GLTF-defined settings. Unsupported GLTF features will **NOT** be exposed directly. + - Attempting to modify the scene hierarchy by moving the entity out of the GLTF structure **MUST** set the state to `GNSV_FAILED`, severing the link. + +4. **Resource Usage** + - To use GLTF resources (meshes, materials) in `Material`, `MeshRenderer`, or `MeshCollider` components, resources **MUST** be preloaded, with loading confirmed in `PBGltfContainerLoadingState`. + - Structural modifications to the GLTF are **NOT RECOMMENDED** as they **SHALL NOT** reflect in the GLTF hierarchy. Removing `GltfNode` **WILL** reset the node to its original GLTF configuration, and retained components may duplicate if not removed. + +This specification provides structured access to GLTF data for interactive scene creation and manipulation, enforcing a robust, standardized protocol for GLTF integration within the engine. + +## RFC 2119 and RFC 8174 + +> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 and RFC 8174. From 4b88ca59f2a3fcadc0840ea11b78406298667ee3 Mon Sep 17 00:00:00 2001 From: Lean Mendoza Date: Fri, 1 Nov 2024 07:07:44 -0300 Subject: [PATCH 3/5] set adr num --- content/ADR-253-extension-gltf-container.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ADR-253-extension-gltf-container.md b/content/ADR-253-extension-gltf-container.md index d66e3b85..af091ddb 100644 --- a/content/ADR-253-extension-gltf-container.md +++ b/content/ADR-253-extension-gltf-container.md @@ -1,6 +1,6 @@ --- layout: adr -adr: 250 +adr: 253 title: Extension GLTF functionality in runtime 7 date: 2024-10-26 status: Draft From 6f8d6149186bca42efe61c7368c05b2f32e56b87 Mon Sep 17 00:00:00 2001 From: Lean Mendoza Date: Fri, 1 Nov 2024 07:47:25 -0300 Subject: [PATCH 4/5] polish --- content/ADR-253-extension-gltf-container.md | 127 ++++++++++++++++++-- 1 file changed, 117 insertions(+), 10 deletions(-) diff --git a/content/ADR-253-extension-gltf-container.md b/content/ADR-253-extension-gltf-container.md index af091ddb..9dfd8370 100644 --- a/content/ADR-253-extension-gltf-container.md +++ b/content/ADR-253-extension-gltf-container.md @@ -12,7 +12,7 @@ authors: --- # Abstract -This document sets the extension of GLTF functionality, the adopted standard in Decentraland to load 3D scenes, by allowing the access to the internal structure of GLTF/GLB models. This capability enables various functionalities such as the use of pointer events for specific colliders, modification of internal nodes and materials, and fetching animation lists. By exposing these internal components, developers can create more dynamic and interactive experiences within the engine. +This document sets the extension of [GLTF functionality](https://adr.decentraland.org/adr/ADR-215), the adopted standard in Decentraland to load 3D scenes, by allowing the access to the internal structure of GLTF/GLB models. This capability enables various functionalities such as the use of pointer events for specific colliders, modification of internal nodes and materials, and fetching animation lists. By exposing these internal components, developers can create more dynamic and interactive experiences within the engine. # Context and Motivation The existing pipeline for using 3D assets in interactive environments lacks the flexibility needed to dynamically control or modify specific elements within a GLTF model. Currently, artists and developers face limitations such as: @@ -36,24 +36,24 @@ The proposed solution expands various GLTF components, enabling detailed, intera - Modify `MeshRenderer` and `MeshCollider`: a `gltf` mesh reference added to these components to support mesh reuse across different entities, enabling interactive development with a single model. - Add `GltfNode` and `GltfNodeState` Components: The new `GltfNode` and `GltfNodeState` components **let developers map GLTF nodes to scene entities**, allowing independent modification of a node’s `Transform`, Material, and MeshRenderer. These components stay in sync with GLTF animations, supporting dynamic transformations, event-based interactions, and resource re-use. -## Usage +# Usage This specification is designed not only to provide functionality for creators but also to enable the development of advanced tools built upon it. The primary and key elements for usage are as follows: -### Fetching the Structure +## Fetching the Structure When assigning a `GltfContainer` to an entity, and upon successful loading, the `GltfLoadingState` is populated with the complete GLTF scene data. This feedback can be disabled by setting `internalFeedback=false` within the `GltfContainer` (default is `true`). -#### Generating an Accurate Animation List +### Generating an Accurate Animation List As a result of fetching the structure, one common challenge in utilizing animations is ensuring the accuracy of animation names. By retrieving the animation list directly from the GLTF, developers can either copy the string names or map the states through the array. For instance, hardcoding each GLTF animation name is impractical when developing dynamic smart items within environments like the In-World-Builder. Obtaining the animation list at runtime is crucial to streamline the direct animation selection pipeline. -### Accessing a Node from the Internal Tree +## Accessing a Node from the Internal Tree Once a GLTF is loaded and instantiated within the world, referencing specific nodes of the GLTF scene becomes possible. The availability of the node list enables efficient identification without the need to reference the entire tree structure. -To reference a node, add a `GltfNode` component as the parent of the `GltfContainer` and await the `GltfNodeState`. Upon reaching the `READY` state, certain components are added to the entity as follows: +To reference a node, add a `GltfNode` component as the **child** of the `GltfContainer` and await the `GltfNodeState`. Upon reaching the `READY` state, certain components are added to the entity as follows: - **Transform**: While set for parenting, this is populated with the relative transform from the `GltfContainer`. - **MeshRenderer**: For visible meshes, this component is added with the `gltf` option. - **MeshRenderer**: For invisible meshes, this component is added with the `gltf` option. @@ -63,10 +63,11 @@ While all components are writable, specific considerations apply: - **Transform**: When an animation influences this node, the transform is disregarded. - **Material**: If the `gltf` field is set in `Material`, default values fall back to the GLTF material properties. For example, properties such as `albedoColor` or `diffuseColor` default to *WHITE* but defer to the GLTF-defined settings if no custom values are provided. -### Using GLTF as a Mesh/Material Resource +## Using GLTF as a Mesh/Material Resource This specification further allows the use of resources from a GLTF by setting the `gltf` field within `Material`, `MeshRenderer`, or `MeshCollider`. It is recommended to first load the `GltfContainer` to retrieve available meshes, materials, and textures, ensuring that these resources are loaded into memory before use. -## Specification +# Specification +## Implementation details 1. **GltfContainer Initialization and Loading State** - A `GltfContainer` component **MUST** be assigned to an entity to initialize and populate `PBGltfContainerLoadingState` with detailed GLTF structure data. @@ -80,6 +81,7 @@ This specification further allows the use of resources from a GLTF by setting th 2. **Adding and Using GltfNode Components** - A `GltfNode` component **MUST** be added to an entity that is either a direct child of a `GltfContainer` entity or another entity with a `GltfNode`. + - More than one `GltfNode` for the same path in the same `GltfContainer` is not allowed. - The `path` in `PBGltfNode` **MUST** match the path of one of the `node_paths` available within `PBGltfContainerLoadingState`. - Upon the renderer attaching `PBGltfNodeState`, the entity **MUST** progress through the following states: - **`GNSV_PENDING`**: Node is loading. @@ -96,15 +98,120 @@ This specification further allows the use of resources from a GLTF by setting th - **Transform**: **MUST** update based on animation priority, with any ongoing animations taking precedence over manual modifications. - **Visibility**: **MAY** be toggled to show or hide the node and its children within the GLTF hierarchy. - **MeshRenderer and MeshCollider**: **MAY** be modified to update or remove meshes and colliders. - - **Material**: **MAY** be modified to adjust material properties, with default values based on GLTF-defined settings. Unsupported GLTF features will **NOT** be exposed directly. + - **Material**: **MAY** be modified to adjust material properties, with default values based on GLTF-defined settings. Unsupported GLTF features by the SDK will **NOT** be exposed directly. - Attempting to modify the scene hierarchy by moving the entity out of the GLTF structure **MUST** set the state to `GNSV_FAILED`, severing the link. 4. **Resource Usage** - - To use GLTF resources (meshes, materials) in `Material`, `MeshRenderer`, or `MeshCollider` components, resources **MUST** be preloaded, with loading confirmed in `PBGltfContainerLoadingState`. - Structural modifications to the GLTF are **NOT RECOMMENDED** as they **SHALL NOT** reflect in the GLTF hierarchy. Removing `GltfNode` **WILL** reset the node to its original GLTF configuration, and retained components may duplicate if not removed. This specification provides structured access to GLTF data for interactive scene creation and manipulation, enforcing a robust, standardized protocol for GLTF integration within the engine. + +## Protobuf components +Here is the specific part of modification to directly do in the Protocol. + +### Modify +It only shows the part is modified. + +```protobuf +option (common.ecs_component_id) = 1017; +message PBMaterial { + .... + message GltfMaterial { + string gltf_src = 1; + string name = 2; + } + + optional GltfMaterial gltf = 3; +} +``` + +```protobuf +option (common.ecs_component_id) = 1019; +message PBMeshCollider { + ... + message GltfMesh { + string gltf_src = 1; + string name = 2; + } + + oneof mesh { + ... + GltfMesh gltf = 6; + } +} +``` + +```protobuf +option (common.ecs_component_id) = 1018; +message PBMeshRenderer { + ... + message GltfMesh { + string gltf_src = 1; + string name = 2; + } + + oneof mesh { + ... + GltfMesh gltf = 6; + } +} +``` + +```protobuf +option (common.ecs_component_id) = 1049; +import "decentraland/sdk/components/common/loading_state.proto"; +message PBGltfContainerLoadingState { + ... + repeated string node_paths = 2; + repeated string mesh_names = 3; + repeated string material_names = 4; + repeated string skin_names = 5; + repeated string animation_names = 6; +} +``` +```protobuf +option (common.ecs_component_id) = 1049; +import "decentraland/sdk/components/common/loading_state.proto"; +message PBGltfContainerLoadingState { + ... + repeated string node_paths = 2; + repeated string mesh_names = 3; + repeated string material_names = 4; + repeated string skin_names = 5; + repeated string animation_names = 6; +} +``` + +```protobuf +option (common.ecs_component_id) = 1041; +message PBGltfContainer { + ... + optional bool internal_feedback = 6; +} +``` + +### New +```protobuf +option (common.ecs_component_id) = 1200; +message PBGltfNode { + string path = 1; +} +``` + +```protobuf +option (common.ecs_component_id) = 1201; +message PBGltfNodeState { + GltfNodeStateValue state = 1; + optional string error = 2; +} +enum GltfNodeStateValue { + GNSV_PENDING = 0; + GNSV_FAILED = 1; + GNSV_READY = 2; +} +``` + ## RFC 2119 and RFC 8174 > The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 and RFC 8174. From 60842dffd4d3911bc3fa2479b3794a2d2c04bdbf Mon Sep 17 00:00:00 2001 From: Lean Mendoza Date: Fri, 1 Nov 2024 08:02:59 -0300 Subject: [PATCH 5/5] polish and set final title --- ....md => ADR-253-exposing-gltf-internals.md} | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) rename content/{ADR-253-extension-gltf-container.md => ADR-253-exposing-gltf-internals.md} (85%) diff --git a/content/ADR-253-extension-gltf-container.md b/content/ADR-253-exposing-gltf-internals.md similarity index 85% rename from content/ADR-253-extension-gltf-container.md rename to content/ADR-253-exposing-gltf-internals.md index 9dfd8370..7b268ffc 100644 --- a/content/ADR-253-extension-gltf-container.md +++ b/content/ADR-253-exposing-gltf-internals.md @@ -1,7 +1,7 @@ --- layout: adr adr: 253 -title: Extension GLTF functionality in runtime 7 +title: Exposing GLTF internals date: 2024-10-26 status: Draft type: Standards Track @@ -12,19 +12,19 @@ authors: --- # Abstract -This document sets the extension of [GLTF functionality](https://adr.decentraland.org/adr/ADR-215), the adopted standard in Decentraland to load 3D scenes, by allowing the access to the internal structure of GLTF/GLB models. This capability enables various functionalities such as the use of pointer events for specific colliders, modification of internal nodes and materials, and fetching animation lists. By exposing these internal components, developers can create more dynamic and interactive experiences within the engine. +This document sets the extension of [GLTF functionality](https://adr.decentraland.org/adr/ADR-215), the adopted standard in Decentraland to load 3D scenes, by allowing access to the internal structure of GLTF/GLB models. This capability enables various functionalities, such as using pointer events for specific colliders, modifying internal nodes and materials, and fetching animation lists. By exposing these internal components, developers can create more dynamic and interactive experiences within the engine. # Context and Motivation The existing pipeline for using 3D assets in interactive environments lacks the flexibility needed to dynamically control or modify specific elements within a GLTF model. Currently, artists and developers face limitations such as: - Difficulty creating interactable components within a single GLTF asset, needing separate exports for each interactive model part. -- Inability to directly reference or manipulate internal nodes, meshes, and materials within the GLTF, which complicates asset integration and limits creative potential. +- The inability to directly reference or manipulate internal nodes, meshes, and materials within the GLTF complicates asset integration and limits creative potential. - Challenges in synchronizing code-driven animations, events, and visual modifications with specific parts of a model, impeding advanced customizations. This feature is motivated by the need to improve both basic and advanced use cases for creators working with GLTF models: -- For **entry-level creators**: This feature enables direct access to model elements like colliders and transforms without separate exports, supporting quicker, easier setup for interactable components. For instance, you would be able to export a entire scene, prefix all the nodes you find relevant, and then fetch them in a list to manipulate in the scene-code. -- For **advanced developers**: It provides powerful tools to transform internal nodes, repurpose model elements (e.g., using a mesh or material as a resource), and create code-driven animations or effects that modify specific model components dynamically. For example, you could attach other models to internal part, like a sword to a hand in a NPC model, or you can use a GLTF to instance several meshes with pointing the same resource. +- For **entry-level creators**: This feature enables direct access to model elements like colliders and transforms without separate exports, supporting quicker, easier setup for interactable components. For instance, you would be able to export an entire scene, prefix all the relevant nodes, and then fetch them in a list to manipulate in the scene code. +- For **advanced developers**: It provides powerful tools to transform internal nodes, repurpose model elements (e.g., using a mesh or material as a resource), and create code-driven animations or effects that modify specific model components dynamically. For example, you could attach other models to the internal parts, like a sword to a hand in an NPC model, or you can use a GLTF to instance several meshes pointing to the same resource. -By extending access to these internal resources, this solution provides a deeper level of control, supporting a flexible workflow that integrates more seamlessly with artist pipelines and accelerates creative experimentation. +This solution provides a deeper level of control by extending access to these internal resources. It supports a flexible workflow that integrates more seamlessly with artist pipelines and accelerates creative experimentation. # Proposed Solution @@ -46,9 +46,9 @@ The primary and key elements for usage are as follows: When assigning a `GltfContainer` to an entity, and upon successful loading, the `GltfLoadingState` is populated with the complete GLTF scene data. This feedback can be disabled by setting `internalFeedback=false` within the `GltfContainer` (default is `true`). ### Generating an Accurate Animation List -As a result of fetching the structure, one common challenge in utilizing animations is ensuring the accuracy of animation names. By retrieving the animation list directly from the GLTF, developers can either copy the string names or map the states through the array. +One common challenge in utilizing animations is ensuring the accuracy of animation names. As a result of fetching the structure, the animation list can directly be taken from the GLTF. Developers can either copy the string names or map the states through the array. -For instance, hardcoding each GLTF animation name is impractical when developing dynamic smart items within environments like the In-World-Builder. Obtaining the animation list at runtime is crucial to streamline the direct animation selection pipeline. +For instance, hardcoding each GLTF animation name is impractical when developing dynamic smart items within environments like the In-World Builder. Obtaining the animation list at runtime is crucial to streamlining the direct animation selection pipeline. ## Accessing a Node from the Internal Tree Once a GLTF is loaded and instantiated within the world, referencing specific nodes of the GLTF scene becomes possible. The availability of the node list enables efficient identification without the need to reference the entire tree structure. @@ -64,7 +64,7 @@ While all components are writable, specific considerations apply: - **Material**: If the `gltf` field is set in `Material`, default values fall back to the GLTF material properties. For example, properties such as `albedoColor` or `diffuseColor` default to *WHITE* but defer to the GLTF-defined settings if no custom values are provided. ## Using GLTF as a Mesh/Material Resource -This specification further allows the use of resources from a GLTF by setting the `gltf` field within `Material`, `MeshRenderer`, or `MeshCollider`. It is recommended to first load the `GltfContainer` to retrieve available meshes, materials, and textures, ensuring that these resources are loaded into memory before use. +This specification further allows the use of resources from a GLTF by setting the `gltf` field within `Material`, `MeshRenderer`, or `MeshCollider`. It is recommended to first load the `GltfContainer` and retrieve available meshes, materials, and textures, ensuring that these resources are loaded into memory before being used. # Specification ## Implementation details @@ -108,7 +108,7 @@ This specification provides structured access to GLTF data for interactive scene ## Protobuf components -Here is the specific part of modification to directly do in the Protocol. +Here is the specific part of the modification to directly do in the Protocol. ### Modify It only shows the part is modified.