patching a class yields "AttributeError: Mock object has no attribute" when accessing instance attributes

Question

The Problem
Using mock.patch with autospec=True to patch a class is not preserving attributes of instances of that class.

The Details
I am trying to test a class Bar that instantiates an instance of class Foo as a Bar object attribute called foo. The Bar method under test is called bar; it calls method foo of the Foo instance belonging to Bar. In testing this, I am mocking Foo, as I only want to test that Bar is accessing the correct Foo member:

import unittest
from mock import patch
class Foo(object):
 def __init__(self):
 self.foo = 'foo'
class Bar(object):
 def __init__(self):
 self.foo = Foo()
 def bar(self):
 return self.foo.foo
class TestBar(unittest.TestCase):
 @patch('foo.Foo', autospec=True)
 def test_patched(self, mock_Foo):
 Bar().bar()
 def test_unpatched(self):
 assert Bar().bar() == 'foo'

The classes and methods work just fine (test_unpatched passes), but when I try to Foo in a test case (tested using both nosetests and pytest) using autospec=True, I encounter "AttributeError: Mock object has no attribute 'foo'"

19:39 $ nosetests -sv foo.py
test_patched (foo.TestBar) ... ERROR
test_unpatched (foo.TestBar) ... ok
======================================================================
ERROR: test_patched (foo.TestBar)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1201, in patched
 return func(*args, **keywargs)
 File "/home/vagrant/dev/constellation/test/foo.py", line 19, in test_patched
 Bar().bar()
 File "/home/vagrant/dev/constellation/test/foo.py", line 14, in bar
 return self.foo.foo
 File "/usr/local/lib/python2.7/dist-packages/mock.py", line 658, in __getattr__
 raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'foo'

Indeed, when I print out mock_Foo.return_value.__dict__, I can see that foo is not in the list of children or methods:

{'_mock_call_args': None,
 '_mock_call_args_list': [],
 '_mock_call_count': 0,
 '_mock_called': False,
 '_mock_children': {},
 '_mock_delegate': None,
 '_mock_methods': ['__class__',
 '__delattr__',
 '__dict__',
 '__doc__',
 '__format__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__module__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__'],
 '_mock_mock_calls': [],
 '_mock_name': '()',
 '_mock_new_name': '()',
 '_mock_new_parent': <MagicMock name='Foo' spec='Foo' id='38485392'>,
 '_mock_parent': <MagicMock name='Foo' spec='Foo' id='38485392'>,
 '_mock_wraps': None,
 '_spec_class': <class 'foo.Foo'>,
 '_spec_set': None,
 'method_calls': []}

My understanding of autospec is that, if True, the patch specs should apply recursively. Since foo is indeed an attribute of Foo instances, should it not be patched? If not, how do I get the Foo mock to preserve the attributes of Foo instances?

NOTE:
This is a trivial example that shows the basic problem. In reality, I am mocking a third party module.Class -- consul.Consul -- whose client I instantiate in a Consul wrapper class that I have. As I don't maintain the consul module, I can't modify the source to suit my tests (I wouldn't really want to do that anyway). For what it's worth, consul.Consul() returns a consul client, which has an attribute kv -- an instance of consul.Consul.KV. kv has a method get, which I am wrapping in an instance method get_key in my Consul class. After patching consul.Consul, the call to get fails because of AttributeError: Mock object has no attribute kv.

Resources Already Checked:

http://mock.readthedocs.org/en/latest/helpers.html#autospeccing http://mock.readthedocs.org/en/latest/patch.html

• Martijn Pieters: That would require the mock to create an instance of the class. That's never a good idea, because that would require it to execute the code you were trying to replace with a mock in the first place.
• Greg Hilston: Unrelated side comment but this question is asked in a really beautiful way. I love how the OP describes their situation, what they expect, shows and example and then explains their real world situation followed by references they've look at. Kudos

Answer 1

No, autospeccing cannot mock out attributes set in the __init__ method of the original class (or in any other method). It can only mock out static attributes, everything that can be found on the class.

Otherwise, the mock would have to create an instance of the class you tried to replace with a mock in the first place, which is not a good idea (think classes that create a lot of real resources when instantiated).

The recursive nature of an auto-specced mock is then limited to those static attributes; if foo is a class attribute, accessing Foo().foo will return an auto-specced mock for that attribute. If you have a class Spam whose eggs attribute is an object of type Ham, then the mock of Spam.eggs will be an auto-specced mock of the Ham class.

The documentation you read explicitly covers this:

A more serious problem is that it is common for instance attributes to be created in the __init__ method and not to exist on the class at all. autospec can’t know about any dynamically created attributes and restricts the api to visible attributes.

You should just set the missing attributes yourself:

@patch('foo.Foo', autospec=Foo)
def test_patched(self, mock_Foo):
 mock_Foo.return_value.foo = 'foo'
 Bar().bar()

or create a subclass of your Foo class for testing purposes that adds the attribute as a class attribute:

class TestFoo(foo.Foo):
 foo = 'foo' # class attribute
@patch('foo.Foo', autospec=TestFoo)
def test_patched(self, mock_Foo):
 Bar().bar()

Answer 2

There is only a create kwarg in patch that, when set to True, will create the attribute if it doesn't exist already.

If you pass in create=True, and the attribute doesn’t exist, patch will create the attribute for you when the patched function is called, and delete it again after the patched function has exited.

https://docs.python.org/3/library/unittest.mock.html#patch