Mocking __new__ breaks subsequent object creation

[ymyke]

Potential pytest bug (see also Comments section).

Problem description

My goal is to patch a class' __new__ method in order to control the exact object it creates in a test case. See the code below.

Each of the two tests test_1 and test_2 work as expected when run individually. When they are both run consecutively, however, test_2 fails with:

TypeError: object.__new__() takes exactly one argument (the type to instantiate)

Code

import pytest


# The class to test:
class A:
    def __init__(self, x):
        self.x = x


def test_1(mocker):
    """First test patches the new method."""
    # Create object:
    a = A(x=3)
    # Make sure new method always returns this object:
    mocker.patch.object(A, "__new__", return_value=a)
    # Switch off the initializer (so it doesn't overwrite values in the pre-made object):
    mocker.patch.object(A, "__init__", return_value=None)
    # Do the test:
    res = A(x=1)
    assert res.x == 3


def test_2():
    """Second test uses the class as intended."""
    A(2)  # <- Fails (when run right after test_1)

Comments

I'm not sure if this is really a pytest issue, since the problem also occurs even if I store away the original __new__ method pointer and restore it at the end of test_1. Either way, I'd appreciate any help in finding out what is going on.

Environment

• pytest 5.3.5
• pytest-mock 3.2.0
• pytest-socket 0.3.5
• pytest-sugar 0.9.4
• Python 3.7.5
• Windows 10

[RonnyPfannschmidt]

At first glance it looks like misuse of mocking, nothing pytest can do here

[ymyke]

Why is that a misuse? Any idea how I can achieve my goal in "legal" ways?

[ymyke]

BTW, it works if I change the class definition to this:

class A(object):
    def __new__(cls, *_, **__):
        return super(A, cls).__new__(cls)

    def __init__(self, x):
        self.x = x

[mleiter06]

I'm finding this two years later. But why...

[ViaFlorian]

This took hours to find!!!! Thanks for the solution :-) <3

[RonnyPfannschmidt]

To figure the correct way I'd have to experiment, i would suggest to Patch __call__ however

[crispyam]

How would that help

[nicoddemus]

Weird, I can't reproduce the problem either on master or 5.3.4.

I would try @RonnyPfannschmidt's suggestion though.

[dlahyani]

@ymyke ~4 years later...
Had a very similar issue, dug a bit into the Python docs and implementation and ran into this:

/* You may wonder why object.__new__() only complains about arguments
   when object.__init__() is not overridden, and vice versa.

   Consider the use cases:

   1. When neither is overridden, we want to hear complaints about
      excess (i.e., any) arguments, since their presence could
      indicate there's a bug.

   2. When defining an Immutable type, we are likely to override only
      __new__(), since __init__() is called too late to initialize an
      Immutable object.  Since __new__() defines the signature for the
      type, it would be a pain to have to override __init__() just to
      stop it from complaining about excess arguments.

   3. When defining a Mutable type, we are likely to override only
      __init__().  So here the converse reasoning applies: we don't
      want to have to override __new__() just to stop it from
      complaining.

   4. When __init__() is overridden, and the subclass __init__() calls
      object.__init__(), the latter should complain about excess
      arguments; ditto for __new__().

   Use cases 2 and 3 make it unattractive to unconditionally check for
   excess arguments.  The best solution that addresses all four use
   cases is as follows: __init__() complains about excess arguments
   unless __new__() is overridden and __init__() is not overridden
   (IOW, if __init__() is overridden or __new__() is not overridden);
   symmetrically, __new__() complains about excess arguments unless
   __init__() is overridden and __new__() is not overridden
   (IOW, if __new__() is overridden or __init__() is not overridden).

   However, for backwards compatibility, this breaks too much code.
   Therefore, in 2.6, we'll *warn* about excess arguments when both
   methods are overridden; for all other cases we'll use the above
   rules.

*/

Source: https://github.com/python/cpython/blob/55596ae0446e40f47e2a28b8897fe9530c32a19a/Objects/typeobject.c#L6230C1-L6269C1

Also, see the implementations of object_new and object_init following this comment.
Basically, what they do there is checking if the types' __new__ method is not identical to the default new_object of PyObjectType (which is the implementation of type.__new__), and if it's not identical they disallow "excess args".

I suspect that once the __new__ method is overridden by the patcher, it will not go back to be identical to PyObjectType s new_object even when it is being restored (since it was assigned a value).