Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for pyiceberg using s3 signer #9318

Closed
guitcastro opened this issue Aug 9, 2024 · 1 comment · Fixed by #9868
Closed

Support for pyiceberg using s3 signer #9318

guitcastro opened this issue Aug 9, 2024 · 1 comment · Fixed by #9868

Comments

@guitcastro
Copy link

Issue description

It's not possible to use Nessie rest s3 signer with pyiceberg. This is on their side, and I working to fix it. I am opening an issue here just to keep track.

@guitcastro guitcastro changed the title Support for pyiceberg Support for pyiceberg using s3 signer Aug 9, 2024
@hienduyph
Copy link

Currently, there's a workaround to make pyiceberg works:

    if properties.get("s3.remote-signing-enabled", None):
        properties["s3.signer"] = "S3V4RestSigner"
    if signer := properties.get("s3.signer"):

And make sure you're using FsSpec

catalog = load_catalog(
    "nessie",
    **{
        ...
        "py-io-impl": "pyiceberg.io.fsspec.FsspecFileIO",
    },
)

snazy added a commit to snazy/nessie that referenced this issue Nov 4, 2024
* Add sending `py-io-impl=pyiceberg.io.fsspec.FsspecFileIO` a config-default.
* Add sending `s3.signer=S3V4RestSigner` when S3 signing is being used.

Fixes projectnessie#9318
snazy added a commit that referenced this issue Nov 4, 2024
* Add sending `py-io-impl=pyiceberg.io.fsspec.FsspecFileIO` a config-default.
* Add sending `s3.signer=S3V4RestSigner` when S3 signing is being used.

Fixes #9318
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants