[Feature]: Support Iceberg optimization tool

[nqvuong1998]

Description

Currently, the Nessie catalog has a GC tool to clean up orphaned files. Additionally, Nessie can support an optimization tool to compact, sort, and expire snapshots for Iceberg tables.

Expected Use Cases

GC and optimization tools are useful for keeping Iceberg tables optimized.

Requested Changes in public API

No response

[nqvuong1998]

cc @snazy @ajantha-bhat

[ajantha-bhat]

@nqvuong1998: Nessie GC handles expire snapshots and orphan files together. So, we don't need expire snapshots implementation again. But due to Nessie's feature of catalog level tags, we cannot update the table metadata on the tags after running expire snapshots functionality. Hence, Nessie GC is not updating the table metadata files.

Other operations like compactions (including sort), can work with Engine's existing implementation.
By default it runs spark procedures (say compaction) at table at branch. But we can specify the branch info also.
Testcase:

nessie/integrations/spark-extensions-basetests/src/main/java/org/projectnessie/spark/extensions/AbstractNessieSparkSqlExtensionTest.java

Line 716 in 4d3d53e

void testCompaction(String branchName, String tableName)