Facing issues in parquet file reads from GCS using velox Backend.

[pratham76]

I am trying to read a parquet file from GCS using service credentials file as per this doc and have added following configurations.

        "spark.hadoop.fs.gs.auth.type": "SERVICE_ACCOUNT_JSON_KEYFILE",
        "spark.hadoop.fs.gs.auth.service.account.json.keyfile": "/mnts/applications/key.json",

though Java based spark works using these credentials. Gluten hangs with the following statement.
W20241005 09:53:12.233268 188 GCSFileSystem.cpp:303] Config::gcsCredentials is empty

Please help if there is any workaround for this?

[majetideepak]

This was recently fixed in Velox. See facebookincubator/velox#11086
Can you try with the latest Gluten version?

[majetideepak]

We need two configs for GCS in Velox.

hive.gcs.json-key-file-path=/mnts/applications/key.json
hive.gcs.endpoint=storage.googleapis.com

[majetideepak]

@pratham76 I checked the code and you will need this PR #7421
Can you please check with this patch?
I see that the default gcs endpoint should work as expected.

[pratham76]

sure, will verify on a build with this change.