Pickle format upgrade (2 -> 4) in FilesDB

[ermo]

Summary

The goal of this PR is to ensure that the pickle format variable for the filesdb shelve is now
set to upstream pickle.DEFAULT_PROTOCOL, such that no refactoring is required for
debugging output.

This makes it upstream's responsibility to ensure pickle format compatibility.

Test Plan

• Benchmarked this branch against joey's lmdb branch with no issue.
• Joey said he saw a small speedup going from pickle protocol 2 (py2 compatible) to pickle protocol 4 (py3.12 pickle.DEFAULT_PROTOCOL value):

[EbonJaeger]

So, after looking more into this and thinking about it for a while, I'm really not sure what we actually gain from this. It doesn't change the default behavior. That variable already wasn't used for anything else like logging.

[ermo]

So, after looking more into this and thinking about it for a while, I'm really not sure what we actually gain from this. It doesn't change the default behavior. That variable already wasn't used for anything else like logging.

The gain is the explicitness about what pickle protocol is used in lazydb relative to the filesdb implementation.

My proposal is to make this the lzma_mt "baseline" that gets used by eopkg and ypkg until the rest of the optimisations/simplifications (py-lmdb, fetcher, multiprocessing) are landed.

The development + deployment plan I am proposing is here: https://gist.github.com/ermo/6da8178a1d678b5c5b31c1a5b86829bb

[EbonJaeger]

The gain is the explicitness about what pickle protocol is used relative to the filesdb implementation.

It really doesn't, though. It's still just going to use the default version. The change literally does nothing vs not having it.

[ermo]

The gain is the explicitness about what pickle protocol is used relative to the filesdb implementation.

It really doesn't, though. It's still just going to use the default version. The change literally does nothing vs not having it.

The commits touching the lazydb stuff have been dropped. The lazydb stuff was only about making the version explicit.

In the present PR, pickle.DEFAULT_PROTOCOL in filesdb (not lazydb) resolves to 4 (py3.12 pickle.DEFAULT_PROTOCOL value), which is an upgrade over the previous 2 (which was the py2-compatible format).

Note also that, in Joey's benchmarks, 4 was slightly quicker than 2:

If you search for shelve.open in the full filesdb.py file, you get the following hits that reference the FILESDB_PICKLE_PROTOCOL_VERSION variable:

• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L234
• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L238
• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L315
• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L317
• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L389
• https://github.com/ermo/eopkg/blob/e2c3c0ccd20dd6244c90c074c420457728c413d7/pisi/db/filesdb.py#L391

... and because it is managed by a context manager, that's the version that gets written as well.