Improve example of pickle "Handling Stateful Objects"

[Serge3leo]

Documentation

In the example section "pickle — Python object serialization" / "Pickling Class Instances" / "Handling Stateful Objects", the TextReader class code is correct only if it will never be used as one of the base classes of multiple inheritance. In addition, this limitation is not described anywhere, which can cause errors during Cut-and-Paste.

Correct code for the general case:

class TextReader:
    ...
    def __getstate__(self):
        ...
        state = super().__getstate__().copy()
        # Remove the unpicklable entries.
        del state['file']
        return state

    def __setstate__(self, state):
        # Restore instance attributes (i.e., filename and lineno).
        getattr(super(), '__setstate__', self.__dict__.update)(state)
        ...

[picnixz]

I'm not really sure we need to make the class entirely usable as a parent class. As a standalone class, it's fine and it works (or am I wrong?). Users that inherit from this class shoud then make the appropriate changes.

The purpose of this tutorial is to illustrate how to override __setstate__ and __getstate__, and to keep it simple.

[Serge3leo]

The example is currently incorrect. The TextReader class cannot be used in general (will cause erroneous behavior of other classes). For example, it cannot be used like this:

class Foo(TextReader, Boo):
    ...

P.S.

It's good when the example is simple. It's bad when the example violates the MRO call chain conventions of __getstate__() and __setstate__().

[picnixz]

The example is currently incorrect

Strictly speaking, it is correct. Nothing says that it can be used as a base class and we can assume that inheriting from it would require additional work. Now, using

getattr(super(), '__setstate__', self.__dict__.update)(state)

has some merits so I can understand why adding this one. However, what if Boo defines a __reduce__ interface and returns specific callable to be used instead of __setstate__? Same question with __getstate__: what if Boo defines a __reduce__ method but not a __getstate__ one? I haven't tested nor implemented anything so I may be wrong, but I think we need to tweak the mixins a bit more (but agreed, this is a corner case).

---

@serhiy-storchaka Do you want to update the docs with a more generic __getstate__ and __setstate__? I'm ok with keeping them as is.

[Serge3leo]

As a standalone class, it's fine and it works

Then it's worth writing clearly: "The example standalone TextReader class below opens a text file, and ..."

The purpose of this tutorial is to illustrate how to override setstate and getstate, and to keep it simple.

Calling super().__getstate__() and super.__setstate__(), which are further down by MRO, is also part of the "overriding" illustrated. In the general case, it is worth illustrating an explicit general mechanism, rather than implicit optimizations for particular cases.

However, what if Boo defines a __reduce__ interface and returns specific callable to be used instead of __setstate__? Same question with __getstate__: what if Boo defines a __reduce__ method but not a __getstate__ one?

If at this moment, the control in __getstate__()/__setstate__() and further along the MRO have __reduce__(), then __getstate__()/__setstate__() were called from __reduce__() the current or higher level. I assume that the upstream __reduce__() will call the downstream __reduce__().

Anyway, this is an area of documentation and examples for __reduce__(). And it says:

Although powerful, implementing __reduce__() directly in your classes is error prone. For this reason, class designers should use the high-level interface (i.e., __getnewargs_ex__(), __getstate__() and __setstate__()) whenever possible. We will show, however, cases where using __reduce__() is the only option or leads to more efficient pickling or both.