Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analysis_timeseries.py shelves don't follow best practice #23

Open
ledm opened this issue Aug 3, 2022 · 1 comment
Open

Analysis_timeseries.py shelves don't follow best practice #23

ledm opened this issue Aug 3, 2022 · 1 comment

Comments

@ledm
Copy link
Collaborator

ledm commented Aug 3, 2022

The Analysis_timeseries.py shelves files don't follow best practice. As it stands, they currently fully re-written regularly. It would be better to use the shelve sync function to update on disk.

It's not a huge issue as there is not a lot of data to write most of the time, but it could become more significant on days where jasmin is slow, or when we're looking ar very large shelve files (hundreds or thousands of years).

It would also be nice to hear your thoughts on shelve files in general, @valeriupredoi. They're so easy to use, but have some risks. We could reduce some of their volatility by for instance requesting a specific pickle protocol to reduce uncertainty.

@valeriupredoi
Copy link
Owner

this is a bit Chinese for me atm - can we talk with voice about it maybe? 🇨🇳

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants