Mocking with Python!


img/mock.png


By Juan Norris

img/santex_logo.png

Presenter Notes

Overview

  • Introduction
  • Basics: Mock, MagicMock, patch
  • Don't mock yourself
  • More helpers
  • Thoughts/Questions

Presenter Notes

Introduction

What is it?

Mock objects are simulated objects that mimic the behavior of real objects in a controlled way.

Python mock

  • Is a library for testing in Python.
  • It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.
  • Provides a core Mock class removing the need to create a host of stubs throughout your test suite.
  • Is part of the standard library, available as unittest.mock in Python >= 3.3
  • For Python < 3.3, it is available at PyPI: $ pip install mock

Presenter Notes

Motivation

Why should I use it?

In a unit test, mock objects can simulate the behavior of complex, real objects and are therefore useful when a real object is impractical or impossible to incorporate into a unit test. If an actual object has any of the following characteristics, it may be useful to use a mock object in its place:

  • supplies non-deterministic results (e.g., the current time or the current temperature)
  • it has states that are difficult to create or reproduce (e.g., a network error)
  • it is slow (e.g., a complete database initialization, filesystem and network access)
  • has some undesired side-effects in the context of your unit-tests
  • it does not yet exist or may change behavior
  • it would have to include information and methods exclusively for testing purposes (and not for its actual task)

For example, an alarm clock program which causes a bell to ring at a certain time might get the current time from the outside world. To test this, the test must wait until the alarm time to know whether it has rung the bell correctly. If a mock object is used in place of the real object, it can be programmed to provide the bell-ringing time (whether it is actually that time or not) so that the alarm clock program can be tested in isolation.

Presenter Notes

mock.Mock

  • Is a flexible mock object.
  • Intended to replace the use of stubs and test doubles throughout your code.
  • Are callable and create attributes as new mocks when you access them.
  • Records how you use them, allowing you to make assertions about what your code has done to them.

Setting attributes

1
2
3
4
5
>>> import mock
>>> mock_obj = mock.Mock(x=3)
>>> mock_obj.y = 2
>>> mock_obj.x, mock_obj.y
(3, 2)

Presenter Notes

Return values and exceptions

Setting a return value

1
2
3
>>> mock_obj.foo.return_value = 'val'
>>> mock_obj.foo('bar')
'val'

Raising exceptions

1
2
3
4
5
>>> mock_obj.open_file.side_effect = [IOError('File not found')]
>>> mock_obj.open_file()
Traceback (most recent call last):
...
IOError: File not found

Presenter Notes

More side effects

Multiple calls

1
2
3
4
5
>>> mock_obj.file_exists.side_effect = [True, False]
>>> mock_obj.file_exists('filename')
True
>>> mock_obj.file_exists('filename')
False

Dynamic results

1
2
3
4
5
6
7
8
9
>>> vals = {'file1': True, 'file2': False}
>>> def side_effect(arg):
...     return vals[arg]
...
>>> mock_obj.file_exists.side_effect = side_effect
>>> mock_obj.file_exists('file1')
True
>>> mock_obj.file_exists('file2')
False

Presenter Notes

Tracking calls

All calls to a mock object are recorded and can be seen by specific attributes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
>>> mock_obj.file_exists.called
True
>>> mock_obj.file_exists.call_count
4
>>> mock_obj.file_exists.call_args
call('file2')
>>> mock_obj.file_exists.call_args_list
[call('filename'), call('filename'), call('file1'), call('file2')]
>>> mock_obj.file_exists.method_calls
[]
>>> mock_obj.file_exists.mock_calls
[call('filename'), call('filename'), call('file1'), call('file2')]
>>> mock_obj.method_calls
[call.foo('bar'),
 call.open_file(),
 call.file_exists('filename'),
 call.file_exists('filename'),
 call.file_exists('file1'),
 call.file_exists('file2')]

Presenter Notes

Resetting calls

Reset all call attributes

1
2
3
4
5
>>> mock_obj.reset_mock()
>>> mock_obj.file_exists.call_count
0
>>> mock_obj.file_exists.call_args_list
[]

Presenter Notes

Assertions

A mock object has methods to make assertions about how it has been used.

1
2
3
4
5
6
7
8
>>> mock_obj.my_method.return_value = 'my_value'
>>> mock_obj.my_method('foo', x=1)
'my_value'
>>> mock_obj.my_method.assert_called_with('bar')
Traceback (most recent call last):
...
AssertionError: Expected call: my_method('bar')
Actual call: my_method('foo', x=1)
1
2
3
4
5
6
7
8
>>> mock_obj.my_method.assert_called_once_with('foo', x=1)
>>> mock_obj.foo('bar')
'val'
>>> mock_obj.assert_has_calls(
...     [mock.call.my_method('foo', x=1), mock.call.foo('bar')],
...     any_order=False  # By default
... )
>>> mock_obj.foo.assert_any_call('bar')

Presenter Notes

Mock attributes

Mocks can also be called with arbitrary keyword arguments. These will be used to set attributes on the mock after it is created.

**kwargs

1
2
3
4
>>> attrs = {'get_name.return_value': "Bruce Wayne", 'code_name': 'Batman'}
>>> m = mock.Mock(**attrs)
>>> '{} is {}'.format(m.get_name(), m.code_name)
'Bruce Wayne is Batman'

configure_mock

1
2
3
4
>>> m = mock.Mock()
>>> m.configure_mock(**attrs)
>>> '{} is {}'.format(m.get_name(), m.code_name)
'Bruce Wayne is Batman'

Presenter Notes

Mock attributes

Name

If the mock has a name then it will be used in the repr of the mock. This can be useful for debugging. The name is propagated to child mocks.

If you are mocking a class that has a name attribute, then you'll need to set that attribute manually, because name is a keyword argument for Mock.

1
2
3
4
5
6
>>> m = mock.Mock(name='PersonClass')
>>> m.name == 'PersonClass'
False
>>> m.name = 'Bruce'
>>> m.name
'Bruce'

Presenter Notes

mock.MagicMock

A subclass of Mock with default implementations of most of the magic methods.

By default many of the protocol methods are required to return objects of a specific type. These methods are preconfigured with a default return value:

__lt__: NotImplemented
__gt__: NotImplemented
__le__: NotImplemented
__ge__: NotImplemented
__int__: 1
__contains__: False
__len__: 1
__iter__: iter([])
__exit__: False
__complex__: 1j
__float__: 1.0
__bool__: True
__index__: 1
__hash__: default hash for the mock
__str__: default str for the mock
__sizeof__: default sizeof for the mock

Presenter Notes

The magic

As MagicMock is more capable it makes sense to use it as default.

Fails

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
>>> mock_conn = mock.Mock()
>>> with mock_conn as conn:
...     conn.send('msg')
Traceback (most recent call last):
...
AttributeError: __exit__
>>> 1 in mock_conn
Traceback (most recent call last):
...
TypeError: argument of type 'Mock' is not iterable

Works

1
2
3
4
5
6
>>> mock_conn = mock.MagicMock()
>>> with mock_conn as conn:
...     conn.send('msg')
>>> conn.send.assert_called_once_with('msg')
>>> 1 in mock_conn
False

Presenter Notes

Patch decorators

  • Are used for patching objects only within the scope of the function they decorate.
  • Automatically handle the unpatching for you, even if exceptions are raised.
  • These functions can also be used in with statements, as class decorators or as normal functions.

Without patch

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
>>> class MyClass(object):
...     def file_exists(self, file_path):
...         return os.path.isfile(file_path)
...     def process_file(self, file_path):
...         if self.file_exists(file_path):
...             pass
...
>>> real = MyClass()
>>> real.file_exists = mock.MagicMock()
>>> real.process_file('my_file')
>>> real.file_exists.assert_called_once_with('my_file')

Presenter Notes

mock.patch

As function decorator

1
2
3
4
5
6
7
8
>>> @mock.patch('__main__.MyClass.file_exists')
... def test_process_file(mock_exists):
...     mock_exists.return_value = False
...     my_obj = MyClass()
...     my_obj.process_file('my_file')
...     my_obj.file_exists.assert_called_once_with('my_file')
...
>>> test_process_file()

As context manager

1
2
3
4
5
6
7
8
>>> from StringIO import StringIO
>>> def foo():
...     print 'Something'
...
>>> with mock.patch('sys.stdout', new_callable=StringIO) as mock_stdout:
...     foo()
...     assert mock_stdout.getvalue() == 'Something\\n'
...

Presenter Notes

More patchers

mock.patch.object

Patch the named member (attribute) on an object (target) with a mock object.

1
2
3
>>> @mock.patch.object(MyClass, 'file_exists')
... def test_process_file(mock_exists):
...

mock.patch.dict

Patch a dictionary, or dictionary like object, and restore the dictionary to its original state afterwards.

1
2
3
4
5
6
>>> import os
>>> with mock.patch.dict('os.environ', {'new_key': 'new_value'}):
...     print os.environ['new_key']
...
new_value
>>> assert 'new_key' not in os.environ

Presenter Notes

Handle patching yourself

start/stop

All the patchers have start and stop methods. These make it simpler to do patching in setUp methods or where you want to do multiple patches without nesting decorators or with statements.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
>>> import unittest
>>> class MyTest(unittest.TestCase):
...     exists_patcher = mock.patch.object(MyClass, 'file_exists')
...
...     def setUp(self):
...         self.mock_exists = self.exists_patcher.start()
...         self.mock_exists.return_value = False
...
...     def tearDown(self):
...         self.exists_patcher.stop()
...
...     def test_process_file(self):
...         my_obj = MyClass()
...         my_obj.process_file('my_file')
...         my_obj.file_exists.assert_called_once_with('my_file')
...
>>> MyTest('test_process_file').run()

avoid patch calls on each test

Presenter Notes

Handle patching yourself

It is also possible to stop all patches which have been started.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
>>> class MyTest(unittest.TestCase):
...     exists_patcher = mock.patch.object(MyClass, 'file_exists')
...     process_file_patcher = mock.patch.object(MyClass, 'process_file')
...
...     def setUp(self):
...         self.mock_exists = self.exists_patcher.start()
...         self.mock_process_file = self.process_file_patcher.start()
...
...     def tearDown(self):
...         mock.patch.stopall()

Presenter Notes

Where to patch

patch is straightforward to use. The key is to do the patching in the right namespace. The basic principle is that you patch where an object is looked up, which is not necessarily the same place as where it is defined.

1
2
3
# a.py
class MyClass(object):
    pass
1
2
3
4
5
# b.py
from a import MyClass

def my_func():
    my_obj = MyClass()
1
2
3
4
5
6
# test_b.py
import b

@mock.patch.object(b, 'MyClass')
def test_func(mock_my_class):
    b.my_func()

Presenter Notes

Where to patch

1
2
3
4
5
# b.py
import a

def my_func():
    my_obj = a.MyClass()
1
2
3
4
5
6
# test_b.py
import b

@mock.patch('a.MyClass')
def test_func(mock_my_class):
    b.my_func()

Presenter Notes

Don't mock yourself

mock suffers from two flaws.

Mocks everywhere

The first one is specific to the Mock api.

Because mocks auto-create attributes on demand, and allow you to call them with arbitrary arguments, if you misspell one of these assert methods, then tests can pass silently and incorrectly because of the typo.

1
2
3
4
5
6
7
>>> mock_obj = mock.Mock(name='Thing', return_value=None)
>>> mock_obj.x
<Mock name='Thing.x' id='24930448'>
>>> mock_obj(2)
>>> # Should fail
>>> mock_obj.assret_called_once_with(3)
<Mock name='Thing.assret_called_once_with()' id='25492560'>

You could make all your assertions fail in order to ensure they are correct.

Presenter Notes

Speccing

The second issue is more general to mocking.

If you refactor some of your code, rename members and so on, any tests for code that is still using the old api but uses mocks instead of the real objects will still pass. This means your tests can all pass even though your code is broken.

mock already provides a feature to help with this, called speccing. If you use a class or instance as the spec for a mock then you can only access attributes on the mock that exist on the real class.

1
2
3
4
5
6
7
>>> import collections
>>> mock_obj = Mock(spec=collections.Counter)
>>> mock_obj(2)
>>> mock_obj.assret_called_with(2)
Traceback (most recent call last):
 ...
AttributeError: Mock object has no attribute 'assret_called_with'

Presenter Notes

Autospeccing

  • Limits the API of mocks to the API of an original object (the spec)
  • Is recursive, so attributes have the same API as the attributes of the spec
  • Mocked functions/methods have the same call signature as the original so they raise a TypeError if they are called incorrectly
  • Also applies to instantiated classes, i.e. the return value of specced mocks

Presenter Notes

Autospeccing

Autospec can’t know about any dynamically created attributes and restricts the api to visible attributes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
>>> class Storage(object):
...     def __init__(self, base_dir):
...         self.base_dir = base_dir
...
>>> def my_func():
...     storage = Storage('base')
...     print storage.base_dir
...
>>> @mock.patch('__main__.Storage', autospec=True)
... def test_my_func(mock_storage):
...     storage_instance = mock_storage.return_value
...     storage_instance.configure_mock(base_dir='my_base_dir')
...     my_func()
'my_base_dir'

Presenter Notes

Autospeccing

If we now change our Storage implementation, the test will fail, instead of passing silently like when we don't use specs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
>>> class Storage(object):
...     def __init__(self, base_dir, subfolder):
...         self.base_dir = base_dir
...         self.subfolder_base = subfolder
...
>>> @mock.patch('__main__.Storage', autospec=True)
... def test_my_func(mock_storage):
...     storage_instance = mock_storage.return_value
...     storage_instance.configure_mock(base_dir='my_base_dir')
...     my_func()
Traceback (most recent call last)
...
TypeError: <lambda>() takes exactly 3 arguments (2 given)

Presenter Notes

More Helpers

mock.PropertyMock

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
>>> class Storage(object):
...     def __init__(self, base_dir):
...         self.base_dir = base_dir
...     @property
...     def subfolder_dir(self):
...         return os.path.join(self.base_dir, self.subfolder_base)
...     def get_file_path(self, filename):
...         return os.path.join(self.subfolder_dir, filename)
...
>>> class StorageTestCase(unittest.TestCase):
...
...     @mock.patch.object(Storage, 'subfolder_dir',
...                        new_callable=mock.PropertyMock)
...     def test_get_file_path(self, mock_subfolder):
...         mock_subfolder.return_value = 'base_dir/subfolder'
...         storage = api.Storage('base_dir')
...         file_path = storage.get_file_path('filename')
...         self.assertEqual(file_path, 'base_dir/subfolder/filename')
...

Presenter Notes

More Helpers

mock.mock_open

Mocks the use of open. It works for open called directly or used as a context manager.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
>>> def read():
...     with open('/some/path', 'r') as f:
...         return f.read()
>>> def write(msg):
...     with open('/some/path', 'w') as f:
...         f.write(msg)
...
>>> m = mock.mock_open(read_data='some_data')
>>> with mock.patch('__builtin__.open', m):
...     data = read()
...     write('something')
...
>>> f = m.return_value
>>> # Read checks
>>> m.assert_any_call('/some/path', 'r')
>>> assert data == 'some_data'
>>> f.read.assert_called_once_with()
>>> # Write checks
>>> m.assert_any_call('/some/path', 'w')
>>> f.write.assert_called_once_with('something')

Presenter Notes

Final thoughts





Mock is a powerful and useful testing library, but it has to be used carefully and consciously in order to create meaningful and maintainable tests.

Presenter Notes

Questions??

Presenter Notes