Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Parquet udfs take in PgRelation #209

Merged
merged 10 commits into from
Feb 8, 2025
Merged

fix: Parquet udfs take in PgRelation #209

merged 10 commits into from
Feb 8, 2025

Conversation

rebasedming
Copy link
Contributor

@rebasedming rebasedming commented Feb 7, 2025

Ticket(s) Closed

What

Instead of taking in the file strings, parquet_schema and parquet_describe take in the foreign table relation, which allows us to load in the user mapping options.

BEFORE:

SELECT * FROM parquet_describe('/path/to/file.parquet');

AFTER:

SELECT * FROM parquet_describe('TABLE_NAME');

Why

Fixes a bug where user mapping options were not getting loaded in to new sessions.

How

Tests

Copy link

github-actions bot commented Feb 7, 2025

A schema difference was detected.

A suggested "upgrade.sql" script entry might be:

DROP FUNCTION IF EXISTS parquet_describe(files text);
CREATE OR REPLACE FUNCTION parquet_describe(relation regclass) RETURNS TABLE(column_name text, column_type text, "null" text, key text, "default" text, extra text) AS 'MODULE_PATHNAME', 'parquet_describe_wrapper' LANGUAGE c STRICT;
DROP FUNCTION IF EXISTS parquet_schema(files text);
CREATE OR REPLACE FUNCTION parquet_schema(relation regclass) RETURNS TABLE(file_name text, name text, type text, type_length text, repetition_type text, num_children pg_catalog.int8, converted_type text, scale pg_catalog.int8, "precision" pg_catalog.int8, field_id pg_catalog.int8, logical_type text) AS 'MODULE_PATHNAME', 'parquet_schema_wrapper' LANGUAGE c STRICT;
The full diff between both schemas is:
217c217
< -- src/api/parquet.rs:48
---
> -- src/api/parquet.rs:52
220c220
< 	"files" TEXT /* &str */
---
> 	"relation" regclass /* pgrx::rel::PgRelation */
270c270
< -- src/api/parquet.rs:66
---
> -- src/api/parquet.rs:73
273c273
< 	"files" TEXT /* &str */
---
> 	"relation" regclass /* pgrx::rel::PgRelation */

@philippemnoel philippemnoel merged commit 020bf09 into dev Feb 8, 2025
11 checks passed
@philippemnoel philippemnoel deleted the fix/parquet-udfs branch February 8, 2025 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

s3_endpoint setting is getting lost for parquet_describe
2 participants