Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[KYUUBI #6691] A new Spark SQL command to merge small files #6695

Closed
wants to merge 35 commits into from
Closed

[KYUUBI #6691] A new Spark SQL command to merge small files #6695

wants to merge 35 commits into from

Commits on Sep 11, 2024

  1. 1. involve a compact table command to merge small files 

    2. parser tests pass
    @gabrywu
    gabrywu committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    532369d View commit details
    Browse the repository at this point in the history

  2. parser tests pass

    @gabrywu
    gabrywu committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    d2cbcc2 View commit details
    Browse the repository at this point in the history

  3. reformat all codes

    @gabrywu
    gabrywu committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1a92be0 View commit details
    Browse the repository at this point in the history

  4. SparkPlan resolved successfully

    @gabrywu
    gabrywu committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    906b47e View commit details
    Browse the repository at this point in the history

  5. reformat

    @gabrywu
    gabrywu committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    4e9d8ca View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. adding unit test to recover command

    @gabrywu
    gabrywu committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    350326e View commit details
    Browse the repository at this point in the history

  2. compact table execution tests pass

    @gabrywu
    gabrywu committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    0d30887 View commit details
    Browse the repository at this point in the history

  3. remove unnecessary comments

    @gabrywu
    gabrywu committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    a52d501 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. recover compact table command tests pass

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    7b10692 View commit details
    Browse the repository at this point in the history

  2. more unit tests

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    0ef55ff View commit details
    Browse the repository at this point in the history

  3. fix scala style issue

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    f04170b View commit details
    Browse the repository at this point in the history

  4. reduce message count

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    02f6303 View commit details
    Browse the repository at this point in the history

  5. involve createToScalaConverter

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    cc0ecce View commit details
    Browse the repository at this point in the history

  6. remove unused import

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    32276b5 View commit details
    Browse the repository at this point in the history

  7. remove unused import

    @gabrywu
    gabrywu committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    c48b167 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2024

  1. remove unnecessary comment & reformat

    @gabrywu
    gabrywu committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    b4fd2ad View commit details
    Browse the repository at this point in the history

  2. involve SPECULATION_ENABLED_SYNONYM

    @gabrywu
    gabrywu committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    79e4e94 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2024

  1. involve createRandomTable

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    257dfd6 View commit details
    Browse the repository at this point in the history

  2. try to catch unknown Row

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    66faca4 View commit details
    Browse the repository at this point in the history

  3. use Seq instead of WrappedArray

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    fd5a3c4 View commit details
    Browse the repository at this point in the history

  4. compile on scala-2.13 successfully

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    3775c95 View commit details
    Browse the repository at this point in the history

  5. remove unused import

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    dccc23a View commit details
    Browse the repository at this point in the history

  6. ByteUnit.MiB.toBytes to fix default target size

    @gabrywu
    gabrywu committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    62c1044 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. rename compact-table.md to docs 

    more unit test
    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    42d1fc0 View commit details
    Browse the repository at this point in the history

  2. remove unused comments

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    232407a View commit details
    Browse the repository at this point in the history

  3. support orc

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    5a34b97 View commit details
    Browse the repository at this point in the history

  4. involve toJavaList to compile on scala 2.13

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    c042dfa View commit details
    Browse the repository at this point in the history

  5. add bzip2 unit tests

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    d35f7d4 View commit details
    Browse the repository at this point in the history

  6. spotless:apply

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    8b637de View commit details
    Browse the repository at this point in the history

  7. fix getCodecFromFilePath for orc

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    7fbc805 View commit details
    Browse the repository at this point in the history

  8. reformat

    @gabrywu
    gabrywu committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    59f0b99 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. support more codec

    @gabrywu
    gabrywu committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    ab9b674 View commit details
    Browse the repository at this point in the history

  2. remove unused util class, close opened stream in finally block

    @gabrywu
    gabrywu committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    22f0b79 View commit details
    Browse the repository at this point in the history

  3. rollback regardless of the success or failure of the command

    @gabrywu
    gabrywu committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    8cc2390 View commit details
    Browse the repository at this point in the history

  4. reformat

    @gabrywu
    gabrywu committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    fd39f66 View commit details
    Browse the repository at this point in the history