Partitioning value in create_table() properties

[vmsaipreeth]

Hi Team,

We are using Trino as our backend. We were playing around with creating tables with partitions.

code used for partitioning:

// assume ‘connection’ has connection established via ibis.trino.connect()
// df_ibis is a ibis dataframe
connection.create_table(‘test_partition’, obj = df_ibis, overwrite = True, properties = {“partitioning”: “ARRAY[‘column_name’]”} )

With the above code, SQL that is getting passed to Trino backend is CREATE TABLE test_partition WITH (partitioning = “ARRAY[‘column_name’]”) AS … and throwing an ProgrammingError due to “”.

As of today, SQL accepted by Trino( for creation of tables using partitions) are:
CREATE TABLE test_partition WITH (partitioning = ARRAY[‘column_name’]) AS …

Or

CREATE TABLE test_partition WITH (partitioning = array(‘column_name’)) AS …

Are there any suggestions or workarounds to deal with this issue? TIA!

[cpcloud]

What connector are you using?

If you're using the Hive connector, the property you're looking for is partitioned_by.

[cpcloud]

If the connector you're working with does actually have a partitioning property, you should be able to pass that in as a Python list:

connection.create_table(
    "test_partition",
    obj=df_ibis,
	overwrite=True,
	properties={"partitioning": ["column_name"]},
)

[vmsaipreeth]

Yes. Connector has partitioning property. Passing as a list worked. 👍🏼
Thanks for quick response. Really appreciate it.

[vmsaipreeth]

I understand this discussion is closed. However, I have question related to this topic.

Now that we have a partitioned table created, just checking if there is a way to overwrite a particular partition in the partitioned table?

[gforsyth]

HI @vmsaipreeth -- I don't think we have a way of expressing that in Ibis atm, but I could be wrong. How would you do this in Trino SQL?