Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC: Add support for DML with properties in engine + Add support for branching in Iceberg #23513

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Sep 20, 2024

Description

The 1st commit allows properties after the target table name in DML:

    | INSERT INTO qualifiedName (WITH properties)?
       columnAliases? rootQuery                                        #insertInto
    | DELETE FROM qualifiedName (WITH properties)?
         (WHERE booleanExpression)?                                    #delete
    | UPDATE qualifiedName (WITH properties)?
        SET updateAssignment (',' updateAssignment)*
        (WHERE where=booleanExpression)?                               #update
    | MERGE INTO qualifiedName (AS? identifier)? (WITH properties)?
        USING relation ON expression mergeCase+                        #merge

Avoided session property approach because users may forget the current branch and modify unexpected branch.

The 2nd commit introduces target_branch property to get a branch name in DML and adds the following procedures in Iceberg connector:

  • ALTER TABLE table EXECUTE create_branch(name)
    • creating a new branch
  • ALTER TABLE table EXECUTE drop_branch(name)
    • removing a branch by name
  • ALTER TABLE table EXECUTE fast_forward(from, to)
    • performs a fast-forward of from up to the to snapshot if from is an ancestor of to.

TestIcebergBranching uses those new syntax and procedures.

Relates to #12844

References

ToDo

  • Deprecated old methods in SPI
  • Investigate the expected behavior when modifying branch with schema evolution in Iceberg
  • Fix handling of table statistics

Release notes

# SPI
* TBD. ({issue}`issuenumber`)

# Iceberg
* Add support for `create_branch`, `drop_branch` and `fast_forward` table procedures. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Sep 20, 2024
@github-actions github-actions bot added the iceberg Iceberg connector label Sep 20, 2024
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes branch 4 times, most recently from 7797fcf to eebadc4 Compare September 20, 2024 09:02
@ebyhr ebyhr removed the request for review from pajaks September 20, 2024 09:24
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes branch from eebadc4 to f33b041 Compare September 20, 2024 22:38
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes branch from f33b041 to c7945f9 Compare September 26, 2024 11:47
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes branch 4 times, most recently from abef40e to 10a80b9 Compare September 30, 2024 09:25
@ebyhr ebyhr force-pushed the ebi/iceberg-tag-branch-writes branch from 10a80b9 to 921a012 Compare October 4, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

2 participants