Postings by Petr Splichal

Flexible Metadata Format: Define Essential Attributes


During the last weeks we've been experimenting with the Flexible
Metadata Format proof of concept on some real-life components:
There's a small SELinux example showing how FMF could be used for
filtering relevant tests [1], Jakub Krysl successfully used FMF
for generating different device setups for his VDO testing [2] and
Jan Scotka is now integrating FMF with the Meta Test Family.

As the next step we would like to start discussions about the test
metadata content, that is to define essential attributes which
should be stored close to the test code, directly in git repo.

Flexible Metadata Format


In order to keep test execution efficient when number of test
cases grows, it is crucial to maintain corresponding metadata,
which define some aspects of how the test coverage is executed.
For example limiting environment combinations where the test is
relevant or selecting a subset of important test cases for quick
verification of essential features when testing a security update.

Within the BaseOS QE team we were thinking (for a long time) about
an efficient metadata solution which would cover our use cases and
would be open source.

New "tests" namespace to share test code


While working on adding CI tests [0] using the Standard Test
Interface a need arose to have a shared git repository where tests
could be stored:

* A large number of test files makes a dist-git repository more
difficult to maintain

* Tests might follow a different branching pattern than the
dist-git repo, leading to code duplication

* Shared maintenance for tests sometimes benefits from different
access levels than the release dist-git repository

The plan is to create a new “tests” namespace in Fedora git/pagure
dedicated to storing the shared test code.