Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs.cache seems to be enabled by default #23517

Closed
metalshanked opened this issue Sep 20, 2024 · 4 comments
Closed

fs.cache seems to be enabled by default #23517

metalshanked opened this issue Sep 20, 2024 · 4 comments

Comments

@metalshanked
Copy link

Upgraded from 454 to 458 and see below errors on many iceberg queries.
The doc says that fs.cache is disabled by default and I dont have any configs set for enabling cache yet the stacktrace seems to suggest that caching is being used
Failed to open file: io.trino.filesystem.cache.CacheInputFile@3849a554

Can you please advise?
Thanks!

{"type":"org.apache.iceberg.exceptions.RuntimeIOException","message":"Failed to open file: io.trino.filesystem.cache.CacheInputFile@3849a554","cause":{"type":"org.apache.avro.InvalidAvroMagicException","message":"Not an Avro data file","suppressed":[],"stack":["org.apache.avro.file.DataFileReader.openReader(DataFileReader.java:79)","org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:102)","org.apache.iceberg.avro.AvroIterable.iterator(AvroIterable.java:77)","org.apache.iceberg.io.CloseableIterable$7$1.<init>(CloseableIterable.java:188)","org.apache.iceberg.io.CloseableIter
@wendigo
Copy link
Contributor

wendigo commented Sep 21, 2024

Cc @raunaqmorarka

@raunaqmorarka
Copy link
Member

Filesystem cache is different, this is an in-memory cache of iceberg metadata that is enabled by default. It can be disabled using iceberg.metadata-cache.enabled=false.
Can you show the full error stacktrace ?

@metalshanked
Copy link
Author

Thanks @raunaqmorarka. seems like a standard issue of a corrupted avro on my side.
Thanks for clarifying that file cache here is a metadata cache not the main cache and so it is not any bug on the Trino side.

@raunaqmorarka
Copy link
Member

Thanks for the confirmation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants