Update rabbit in a hat node

[alicia-koh]

• Add a new node WhiteRabbitNode which returns scan metadata (csv or postgresql) as a result
• Remove hardcoding in Rabbit in a Hat node and use scan metadata to retrieve source data for mapping

Merge Checklist

Please cross check this list if additions / modifications needs to be done on top of your core changes and tick them off. Reviewer can as well glance through and help the developer if something is missed out.

• Automated Tests (Jasmine integration tests, Unit tests, and/or Performance tests)
• Updated Manual tests / Demo Config
• Documentation (Application guide, Admin guide, Markdown, Readme and/or Wiki)
• Verified that local development environment is working with latest changes (integrated with latest develop branch)
• following best practices in code review doc

[Copilot]

Pull request overview

This PR updates the dataflow UI plugin to support a new “White Rabbit” node and reworks CSV loading + data mapping to consume scan metadata (instead of hard-coded source assumptions), aligning the backend node execution with the “rabbit in a hat” workflow.

Changes:

• Updated NodeType enum (adds WHITERABBIT, renames the data-mapping node type value).
• Added a shared load_csv_from_storage helper and refactored CSV node loading to use it.
• Introduced WhiteRabbitNode and modified DataMappingNode to use upstream scan metadata for sourcing tables (CSV/DB).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
plugins/flows/data_transformation/dataflow_ui_plugin/types.py	Adds WHITERABBIT node type and changes DATAMAPPING node type value; updates DB table source value.
plugins/flows/data_transformation/dataflow_ui_plugin/nodeutils/csvutils.py	Adds load_csv_from_storage helper using SupabaseStorageAPI.download_file_to_path.
plugins/flows/data_transformation/dataflow_ui_plugin/nodes.py	Refactors CSV loading, adds WhiteRabbitNode, and reworks DataMappingNode to use scan metadata + new query generation structure.
plugins/flows/data_transformation/dataflow_ui_plugin/flow.py	Treats WHITERABBIT as a no-input node during execution; minor log message update.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

plugins/flows/data_transformation/dataflow_ui_plugin/nodes.py:832

• DataMappingNode.test now requires an _input argument, but execute_node_task() calls _node.test(task_run_context) when test mode is enabled. This will raise a TypeError for DataMappingNode in test mode. Either keep the old signature for test(), or update execute_node_task() to pass the incoming input to test() (mirroring how task() is invoked).

    def test(self, _input: dict[str, Result], task_run_context) -> Result:
        return self.task(_input, task_run_context)


    def task(self, _input: dict[str, Result], task_run_context) -> Result:
        try:

plugins/flows/data_transformation/dataflow_ui_plugin/nodes.py:836

• DataMappingNode.task assumes there is at least one incoming node and does next(iter(_input.values())) to fetch scan metadata. If the node is executed without an incoming WhiteRabbitNode (or with an empty input dict), this will raise StopIteration and fail the flow without a helpful error. Add an explicit check for required input and return a clear Result(error=True, ...) when missing.

            table_mapping = self.etl_mapping.table.edges
            target_table_list = set(t.targetHandle for t in table_mapping)
            scan_metadata = next(iter(_input.values())).result