Pytest Markers

Artemis is a plugin for testing floes using Pytest, and it includes several markers which provide utility for testing against Orion.

@package

The marker package is used to specify that a set of tests require a set of packages to be uploaded prior to them running. You can also dynamically modify the requirements that the packages specified require. Note that this decorator can only be used to decorate classes that inherit from FloeTestCase.

from artemis.test import FloeTestCase
from artemis.decorators import package


@package(
    "/path/to/package",
    "/another/a/package",
    manifest={
        "build_requirements": {
            "cpu": 2,
            "gpu": 1,
        }
    }
)
class PackageTestSet(FloeTestCase):

    def test_example(self):
        # Retrieve the packages registered with test case
        # Packages are FloePackages from Orion Client
        packages = self.get_packages()
        print(packages)

@orion_xfail

The marker orion_xfail is used to indicate that the test should pass locally but is expected to fail when run against Orion. This is useful for writing tests that should eventually work in Orion, but are not quite there yet.

from artemis.test import FloeTestCase
from artemis.wrappers import using_orion
from artemis.decorators import orion_xfail


class ExampleTestCase(FloeTestCase):

    @orion_xfail(reason="Not quite there yet")
    def test_example(self):
        if using_orion():
            raise AssertionError("This doesn't work yet")

@on_demand

The marker on_demand is useful for tests that don’t need to be run every testing cycle, and should be skipped most of the time. A common example would be tests that take a particularly long time to complete. Running these tests requires either to use -k <expr>, -m on_demand or --run-all to trigger them.

from time import sleep

from artemis.test import FloeTestCase
from artemis.decorators import on_demand


class ExampleTestCase(FloeTestCase):

    @on_demand(reason="This takes forever")
    def test_long_test(self):
        sleep(1000)