User Guide

This guide will lead you to your first CSV-file parametrized pytest test. It starts with designing your test, preparing your data, writing the test method and finally execute your new test.

The Scenario

Let’s say, you have to test this method:

from functools import reduce
from typing import List, Tuple, Union

def get_smallest_possible_container(
    number_of_items: int,
    dimensions_of_item: Tuple[int, int, int],
    available_container_sizes: Union[List[int], Tuple[int, ...]] = (1000, 2500, 7500),
) -> int:
    volume = reduce(lambda x, y: x * y, [*dimensions_of_item, number_of_items])
    possible_containers = list(filter(lambda s: s >= volume, available_container_sizes))
    if len(possible_containers) == 0:
        raise ValueError("No container available") from None
    return min(possible_containers)

Parts of the code are from a more complex example written for a German blog post. The example code is part of the source code and can be found unter tests/test_blog_example.py. It is documented as test_blog_example.

Prepare your data

Your test data resides in an CSV file. CSV files can have different formats, when it comes to:

• Field separators and delimiters

• Quoting

• Line Termination

The class pytest_csv_params.dialect.CsvParamsDefaultDialect defines a default CSV format that should fit most requirements:

import csv

class CsvParamsDefaultDialect(csv.Dialect):  # pylint: disable=too-few-public-methods
    delimiter = ","
    doublequote = True
    lineterminator = "\r\n"
    quotechar = '"'
    quoting = csv.QUOTE_ALL
    strict = True
    skipinitialspace = True

You can derive your own CSV format class from there (or from csv.Dialect), if your files look any other.

Your test data for the method above could look like this:

"Test-ID",           "Number of items", "Dimensions of item", "Expected Container Size", "Expect Exception?", "Expected Message"
"Small Container 1", "15",              "1 x 2 x 3",          "1000",                    "N",                 ""
"Small Container 2", "125",             "2 x 2 x 2",          "1000",                    "N",                 ""
"Small Container 3", "16",              "3 x 4 x 5",          "1000",                    "N",                 ""
"Medium Container",  "17",              "3 x 4 x 5",          "2500",                    "N",                 ""
"Large Container 1", "2",               "15 x 12 x 10",       "7500",                    "N",                 ""
"Large Container 2", "1",               "16 x 20 x 20",       "7500",                    "N",                 ""
"Not fitting 1",     "2",               "16 x 20 x 18",       "0",                       "Y",                 "No container available"
"Not fitting 2",     "7501",            "1 x 1 x 1",          "0",                       "Y",                 "No container available"

• We have a header line in the first line, that names the single columns

• The column names are not good for argument names

• The value in the dimensions column needs to be transformed in order to get tested

• There is a column that tells if an exception is to be expected, and the last two lines expect one

Design and write the test

The test must call the get_smallest_possible_container method with the right parameters. The CSV file has all information, but maybe not in the right format. We take care of that in a second.

The test may expect an exception, that should also be considered.

The parameters of the test method should reflect the input parameters for the method under test, and the expectations.

So let’s build it:

import pytest

def test_get_smallest_possible_container(
    number_of_items: int,
    dimensions_of_item: Tuple[int, int, int],
    expected_container_size: int,
    expect_exception: bool,
    expected_message: str,
) -> None:
    if expect_exception:
        with pytest.raises(ValueError) as expected_exception:
            get_smallest_possible_container(number_of_items, dimensions_of_item)
        assert expected_exception.value.args[0] == expected_message
    else:
        container_size = get_smallest_possible_container(number_of_items, dimensions_of_item)
        assert container_size == expected_container_size

• The test could now get all parameters needed to execute the get_smallest_container_method, as well as for the expectations

• Based on the expectation for an exception, the test goes in two different directions

Now it’s time for getting stuff from the CSV file.

Add the parameters from the CSV file

Here comes the csv_params() decorator. But one step after the other.

import pytest
from pytest_csv_params.decorator import csv_params

@csv_params(
    data_file=join(dirname(__file__), "assets", "doc-example.csv"),
    id_col="Test-ID",
    header_renames={
        "Number of items": "number_of_items",
        "Dimensions of item": "dimensions_of_item",
        "Expected Container Size": "expected_container_size",
        "Expect Exception?": "expect_exception",
        "Expected Message": "expected_message",
    },
    data_casts={
        "number_of_items": int,
        "dimensions_of_item": get_dimensions,
        "expected_container_size": int,
        "expect_exception": lambda x: x == "Y",
    },
)
def test_get_smallest_possible_container(
    number_of_items: int,
    dimensions_of_item: Tuple[int, int, int],
    expected_container_size: int,
    expect_exception: bool,
    expected_message: str,
) -> None:

• With the parameter data_file you point to your CSV file

• With the parameter id_col you name the column of the CSV file that contains the test case ID; the test case ID is shown in the execution logs

• With the header_renames dictionary you define how a column is represented as argument name for your test method; the highlighted example transforms “Number of items” to number_of_items

• The data_casts dictionary you define how data needs to be transformed to be usable for the test; you can use lambdas or method pointers; all values from the CSV arrive as str

All possible parameters are explained under Configuration, or more technically, in the source documentation of pytest_csv_params.decorator.csv_params().

The data_casts method get_dimensions looks like the following:

def get_dimensions(dimensions_str: str) -> Tuple[int, int, int]:
    dimensions_tuple = tuple(map(lambda x: int(x.strip()), dimensions_str.split("x")))
    if len(dimensions_tuple) != 3:
        raise ValueError("Dimensions invalid") from None
    return dimensions_tuple  # type: ignore

The method is called during the test collection phase. If the ValueError raises, the run would end in an error.

Execute the test

There is nothing special to do now. Just run your tests as always. Your run should look like this:

tests/test.py::test_get_smallest_possible_container[Small Container 1] PASSED            [ 12%] 
tests/test.py::test_get_smallest_possible_container[Small Container 2] PASSED            [ 25%] 
tests/test.py::test_get_smallest_possible_container[Small Container 3] PASSED            [ 37%] 
tests/test.py::test_get_smallest_possible_container[Medium Container] PASSED             [ 50%] 
tests/test.py::test_get_smallest_possible_container[Large Container 1] PASSED            [ 62%]
tests/test.py::test_get_smallest_possible_container[Large Container 2] PASSED            [ 75%] 
tests/test.py::test_get_smallest_possible_container[Not fitting 1] PASSED                [ 87%] 
tests/test.py::test_get_smallest_possible_container[Not fitting 2] PASSED                [100%] 

Analyse test failures

• Is it a failure for all test data elements or just for a few?

• When only some tests fail, the Test ID should tell you where to look at