BigQuery: METASTORE format not supported.

[vejeta]

Recently, while using the library, a new error have appeared

j.l.IllegalArgumentException: Format METASTORE is not supported
	at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)

That error happens when trying to access a bigquery table that doesn't belong to one of the expected formats

TableDefinition.java#L174)
      default:        throw new IllegalArgumentException("Format " + tablePb.getType() + " is not supported");

Is this something that it is expected to be supported soon?

Environment details

1. Specify the API at the beginning of the title: BigQuery
2. OS type and version: Irrelevant
3. Java version: All java versions
4. version(s): all versions

Steps to reproduce

1. Run the following code to get the list of bigquery tables and their metadata.

Code example

    private val client: BigQuery = BigQueryOptions.newBuilder.setProjectId(projectId).build().getService
    
    client.listDatasets(projectId).iterateAll().asScala.map(_.getDatasetId)

Stack trace

j.l.IllegalArgumentException: Format METASTORE is not supported
	at c.g.c.b.TableDefinition.fromPb(TableDefinition.java:174)
	at c.g.c.b.TableInfo$BuilderImpl.<init>(TableInfo.java:253)
	at c.g.c.bigquery.Table.fromPb(Table.java:690)
	at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:966)
	at c.g.c.b.BigQueryImpl$22.apply(BigQueryImpl.java:963)
	at c.g.c.c.Iterators$6.transform(Iterators.java:829)
	at c.g.c.c.TransformedIterator.next(TransformedIterator.java:52)
	at c.g.c.PageImpl$PageIterator.computeNext(PageImpl.java:72)
	at c.g.c.c.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
	at c.g.c.c.AbstractIterator.hasNext(AbstractIterator.java:140)
	at s.c.c.JavaCollectionWrappers$JIteratorWrapper.hasNext(JavaCollectionWrappers.scala:46)
	at s.c.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:99)
	at s.c.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
	at s.c.c.JavaCollectionWrappers$JIterableWrapper.map(JavaCollectionWrappers.scala:83)

External references such as API reference guides

This seems to be related to the new Iceberg formats
https://medium.com/@cesar.cordoba/iceberg-format-a-short-introduction-6cca39e48ac
https://cloud.google.com/bigquery/docs/iceberg-tables

Any additional information below

Checking the information from the command line over that dataset

 bq ls --format=json --project_id="project_id" "dataset"

We can verify several tables have the "METASTORE" format.

{"kind":"bigquery#table","id":"project_id:dataset.mytable","tableReference":{"projectId":"project_id","datasetId":"dataset","tableId":"mytable"},"type":"METASTORE","creationTime":"1711215011000"},

Checking that table:

bq show --format=json "project_id:dataset.mytable"

"type":"METASTORE","externalDataConfiguration":{"sourceUris":["s3://edited.metadata.json"],"sourceFormat":"ICEBERG"