Writing unit tests

Last updated on 2026-03-13 | Edit this page

Overview

Questions

What is a unit test?
How do I write and run unit tests?
How can I avoid test duplication and ensure isolation?
How can I run tests automatically and measure their coverage?

Objectives

Implement effective unit tests using pytest
Execute tests in Visual Studio Code
Explain the issues relating to non-deterministic code
Implement fixtures and test parametrisation using pytest decorators
Configure git hooks and pytest-coverage
Apply best-practices when setting up a new Python project
Recognise analogous tools for other programming languages
Apply testing to Jupyter notebooks

Introduction

Unit testing validates, in isolation, functionally independent components of a program.

In this lesson we’ll demonstrate how to write and execute unit tests for a simple scientific code.

This involves making some technical decisions…

Test frameworks

We’ll use pytest as our test framework. It’s powerful but also user friendly.

For comparison: testing using assert statements:

PYTHON

from temperature import to_fahrenheit

assert to_fahrenheit(30) == 86

Testing using the built-in unittest library:

PYTHON

from temperature import to_fahrenheit
import unittest

class TestTemperature(unittest.TestCase):
    def test_to_farenheit(self):
        self.assertEqual(to_fahrenheit(30), 86)

Testing using pytest:

PYTHON

from temperature import to_fahrenheit

def test_answer():
    assert to_fahrenheit(30) == 86

Why use a test framework?

Avoid reinventing the wheel - frameworks such as pytest provide lots of convenient features (some of which we’ll see shortly)
Standardisation leads to better tooling, easier onboarding etc

Projects that use pytest:

Callout

Learning by example

Reading the test suites of mature projects is a good way to learn about testing methodologies and frameworks

Callout

Testing in other languages

R: testthat
C++: GoogleTest, Boost Test and Catch 2. See our C++ testing course using Google Test.
Fortran: Test-Drive and pFUnit. See the Fibonacci series example coded in Fortran.
See the Software Sustainability Institute’s Build and Test Examples for many more.

Code editors

We’ve chosen Visual Studio Code as our editor. It’s free, open source, cross-platform and has excellent Python (and pytest) support. It also has built-in Git integration, can be used to edit files on remote systems (e.g. HPC), and handles Jupyter notebooks (plus many more formats).

Callout

Demonstration of pytest + VS Code + coverage

Test discovery, status indicators and ability to run tests inline
Code navigation (“Go to Definition”)
The Test perspective and Test Output
Maximising coverage (assert recursive_fibonacci(7) == 13)
Test-driven development: adding and fixing a new test (test_negative_number)

A tour of pytest

Checking for exceptions

If a function invocation is expected to throw an exception it can be wrapped with a pytest raises block:

PYTHON

def test_non_numeric_input():
    with pytest.raises(TypeError):
        recursive_fibonacci("foo")

Parametrisation

Similar test invocations can be grouped together to avoid repetition. Note how the parameters are named, and “injected” by pytest into the test function at runtime:

PYTHON

@pytest.mark.parametrize("number,expected", [(0, 0), (1, 1), (2, 1), (3, 2)])
def test_initial_numbers(number, expected):
    assert recursive_fibonacci(number) == expected

This corresponds to running the same test with different parameters, and is our first example of a pytest decorator (@pytest). Decorators are a special syntax used in Python to modify the behaviour of the function, without modifying the code of the function itself.

Skipping tests and ignoring failures

Sometimes it is useful to skip tests (conditionally or otherwise), or ignore failures (for example if you’re in the middle of refactoring some code).

This can be achieved using other @pytest.mark annotations e.g.

PYTHON

@pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows")
def test_linux_only_features():
    ...

@pytest.mark.xfail
def test_unimplemented_code():
    ...

Callout

Refactoring

Code refactoring is “the process of restructuring existing computer code without changing its external behavior” and is often required to make code more amenable to testing.

Fixtures

If multiple tests require access to the same data, or a resource that is expensive to initialise, then it can be provided via a fixture. These can be cached by modifying the scope of the fixture. See this example from Devito:

PYTHON

@pytest.fixture
def grid(shape=(11, 11)):
    return Grid(shape=shape)

def test_forward(grid):
    grid.data[0, :] = 1.
    ...

def test_backward(grid):
    grid.data[-1, :] = 7.
    ...

This corresponds to providing the same parameters to different tests.

Tolerances

It’s common for scientific codes to perform estimation by simulation or other means. pytest can check for approximate equality:

PYTHON

def test_approximate_pi():
    assert 22/7 == pytest.approx(math.pi, abs=1e-2)

Callout

Random numbers

If your simulation or approximation technique depends on random numbers then consistently seeding your generator can help with testing. See random.seed() for an example or the pytest-randomly plugin for a more comprehensive solution.

doctest

pytest has automatic integration with the Python’s standard doctest module when invoked with the --doctest-modules argument. This is a nice way to provide examples of how to use a library, via interactive examples in docstrings:

PYTHON

def recursive_fibonacci(n):
    """Return the nth number of the fibonacci sequence

    >>> recursive_fibonacci(7)
    13
    """
    return n if n <=1 else recursive_fibonacci(n - 1) + recursive_fibonacci(n - 2)

Hands-on unit testing (10 min)

Discussion

Setting up our editor

If you haven’t already, see the setup guide for instructions on how to install Visual Studio Code and conda.
Download and extract this zip file. If using an ICT managed PC please be sure to do this in your user area on the C: drive i.e. C:\Users\your_username
- Note that files placed here are not persistent so you must remember to take a copy before logging out
In Visual Studio Code go to File > Open Folder… and find the files you just extracted.
If you see an alert “This workspace has extension recommendations.” click Install All and then switch back to the Explorer perspective by clicking the top icon on the left-hand toolbar
Open Anaconda Prompt (Windows), or a terminal (Mac or Linux) and run:

BASH

conda env create --file "path_to_environment.yml"

The path_to_environment.yml can be obtained by right-clicking the file name in the left pane of Visual Studio Code and choosing “Copy Path”. Be sure to include the quotation marks. Right click on the command line interface to paste.

Important: After the environment has been created go to View > Command Palette in VS Code, start typing “Python: Select interpreter” and hit enter. From the list select your newly created environment “diffusion”

Running the tests

Open test_diffusion.py
You should now be able to click on Run Test above the test_heat() function and see a warning symbol appear, indicating that the test is currently failing. You may have to wait a moment for Run Test to appear.
Switch to the Test perspective by clicking on the flask icon on the left-hand toolbar. From here you can Run All Tests, and Show Test Output to view the coverage report (see Lesson 1 for more on coverage)
Important: If you aren’t able to run the test then please ask a demonstrator for help. It is essential for the next exercise.

Callout

Diffusion exercise in Codespaces

If you are having trouble setting up your system with conda and vscode, or running through this exercise locally in your computer, you can run it in Codespaces.

Check the information at the end of the setup on how to run Codespaces.
Apply it to this exercise repository in GitHub.

Advanced topics

More pytest plugins

pytest-benchmark provides a fixture that can transparently measure and track performance while running your tests:

PYTHON

def test_fibonacci(benchmark):
    result = benchmark(fibonacci, 7)
    assert result == 13

Callout

pytest-benchmark example

Demonstration of performance regression via recursive and formulaic approaches to Fibonacci calculation (output)

pytest-notebook can check for regressions in your Jupyter notebooks (see also Jupyter CI)

Hypothesis provides property-based testing, which is useful for verifying edge cases):

PYTHON

from fibonacci import recursive_fibonacci
from hypothesis import given, strategies

@given(strategies.integers())
def test_recursive_fibonacci(n):
    phi = (5 ** 0.5 + 1) / 2
    assert recursive_fibonacci(n) == int((phi ** n - -phi ** -n) / 5 ** 0.5)

Further resources

Key Points

Testing is not only standard practice in mainstream software engineering, it also provides distinct benefits for any non-trivial research software
pytest is a powerful testing framework, with more functionality than Python’s built-in features while still catering for simple use cases
Testing with VS Code is straightforward and encourages good habits (writing tests as you code, and simplifying test-driven development)
It is important to have a set-up you can use for every project - so that it becomes as routine in your workflow as version control itself
pytest has a myriad of extensions that are worthy of mention such as Jupyter, benchmark, Hypothesis etc