Pytest

Tags

library

Is the standard python testing library.

Note: Pytest is an external testing framework building on the builtin unittest library. Both are described in this file for ease of reference, despite actually being two separate libraries.

Writing Tests

Tests are functions that start with a test_ prefix.

Fixtures

Fixtures are objects or resources that need to be shared across multiple tests in a convenient way. In pytest a fixture is a function that each test-case can accept and whose value can be initialised for the entire test suite, or for each test case.

import pytest


# Note: if you omit name=... the fixture would have the same name
# as the function.
@pytest.fixture(name="container_box")
def container_box_fixture():
    return Box(height=102, width=96, length=240)


def test_container(container_box):
    # pytest automatically passes in container_box with the value of the
    # container_box_fixture() function.
    ...

Note: For convenience you may place fixtures in a conftest.py file. This file is automatically imported by pytest meaning all your test cases can seamlessly use the fixture.

Note: To avoid repeatedly defining a fixtures value when it doesn't change across tests, you can mark the fixture as class scopes, session scoped or module scoped instead of the default function scope.

Cleaning Up Fixture Resources

If your fixture has resources, such as an open file, that needs to be cleaned up after its used you can yield the value of the fixture instead of returning it and then place any cleanup logic afterwards.

@pytest.fixture
def config_file():
    file = open('config.json', 'r')
    yield file
    file.close()

Or better yet, you can use the context-manager pattern to ensure the resource is properly released even if your test throws an exception.

@pytest.fixture
def config_file():
    with open('config.json', 'r') as file:
	yield file

Fixture Dependencies

Any parameters defined by a fixture can be filled in by another fixture. This allows you to write highly modular test code.

import pytest

@pytest.fixture
def config_file():
    return 'config.json'

@pytest.fixture
def config(config_file):
    with open(config_file, 'r') as fd:
	return fd.read()

Marking Tests

The pytest.mark module contains a list of decorators you can apply to test cases to change their behaviour. This includes decorators to skip a test, skipif the test fails some predicate, or to mark a test as xfail meaning you expect the test to fail and it not failing is a failure.

Parameterised Tests

The module also exposes methods to parameterise test cases. This lets you conveniently pass multiple combinations of values without redefining the test logic.

def square(a):
    return a * a

@pytest.mark.parametrize("test_input,expected", [(1, 1), (2, 4), (3, 9), (4, 16)])
def test_square(test_input, expected):
    assert square(test_input) == expected

Note: If you apply parametrize multiple times pytest will use a combination of all the parameterised types.

@pytest.mark.parametrize("foo", [1, 2])
@pytest.mark.parametrize("foo", [3, 4])
def test_foo(foo, bar):
    # Will output:
    # 1, 3
    # 1, 4
    # 2, 3
    # 2, 4
    print(foo, bar)

Custom Marks

Trying to use a mark function that hasn't been defined will cause pytest to generate one for you. This allows you to apply custom markers to your own tests and later use these to, for example, only run tests with a given mark.

@pytest.mark.foobar
def test_foo():
    ...

Mocking

Is the act of generating objects whose members and return values are fixed and that track how the test may interact with those members during a tests execution.

In python mocks are provided by the unittest library. A unittest.Mock object is an object that can mock another object.

from unittest import mock

m1 = mock.Mock(an_attribute="foo")
m1.an_attribute  # => 'foo'

# If you attempt to access an undefined attribute then the mock will
# implicitly generate a new attribute and initialise it as a new mock
m1.something_else  # => unittest.Mock(...)
# To prevent this from automatically happening you must seal the mock
unittest.mock.seal(m1)
# m1.something_new  # => AttributeError

# Passing a return_value to a Mock makes it a function mock, that
# returns the given value when called.
m2 = mock.Mock(return_value="bar")
m2()  # => 'bar'
m2()  # => 'bar'
m2("foo")  # => 'bar'
# Function mocks keep track of how many times they've been called and
# any arguments that were passed to them.
m2.call_count  # => 3
m2.call_list  # => [call(), call(), call('foo')]

# They also have several built in assertion functions for basic use cases.
m2.assert_called_once()

# You can attach side affects to mocks.
mock.Mock(side_effect=[1, 2, 3])  # Will return one value at a time when called.
mock.Mock(side_effect=TypeError)  # Will raise an exception when called
mock.Mock(side_effect=lambda x: x * 2)  # Will run side effect function when called

# Mocks don't implement magic methods such as __len__ by default. To use
# these use a MagicMock.
len(mock.MagicMock())  # => 0
# With a type argument to the MagicMock only magic methods defined by that type
# are implemented in the MagicMock.
-mock.MagicMock(int)

Patching Libraries with Mocks

You can automatically patch out calls to an external module using unittest.mock.patch.

import os
from unittest import mock
from unittest.mock import patch


def test_makesdir():
    with patch.object(os.path, "mkdir", autospec=True) as mkdir_mock:
	...  # Run test which eventually calls os.path.mkdir()
	mkdir_mock.assert_called_once()

Warn: This only works if at the caller site the caller access the method through the module, not by importing or accessing it directly (example: from os.path import mkdir; mkdir('foo')).

Note: This logic may be better served in a fixture instead of a test-case.

Note: patch.object for simple use cases can be used as a decorator.