Fedora 31 System-Wide Change proposal: Gating Rawhide - Single package updates

<a href="" title=""></a>

== Summary ==
We want to gate packages on test results before they can land in
rawhide. This will reduce the amount of broken dependency,
uninstallable packages and broken composes leading to a more stable
rawhide as well as less work on the infrastructure and rel-eng teams
to keep composes working.

This project will be split in two phases, at first only single package
updates will be supported, in a second stage, we will add support for
multi-packages updates.

This proposal is about the phase 1 of this project.

== Owner ==
* Name: [[User:pingou|Pierre-Yves Chibon]]
* Email: pingou -

People who are/will be involved in this:
* Coordinator: [[User:pingou|Pierre-Yves Chibon]]
* Bodhi: [[User:bowlofeggs|Randy Barlow]] and [[User:abompard|Aurélien Bompard]]
* Fedora CI: [[User:dperpeet|Dominik Perpeet]]

== Detailed Description ==

Querying the Bodhi database, it was revealed that 95% of all our
updates involve a single package.

To be exhaustive, here are the exact number found by Randy:

Of all time:

Single build updates: 123,657 (94.1%)
Multi build updates: 7,766 ( 5.9%)

Total: 131,423

Per release:

Fedora 29:

Single: 4,675 (93.7%)
Multi: 316 ( 6.3%)

Fedora 28:

Single: 9,153 (94.5%)
Multi: 536 ( 5.5%)


Single: 12,664 (94.0%)
Multi: 814 ( 6.0%)

When considering gating rawhide package updates on test results, we
need to consider two workflows: single package updates, multi-package
updates. This proposal only considers the single package updates
workflow. [[Changes/GatingRawhideMultiPackagesUpdates|Another
proposal]] will be submitted in the future to support the multi
package updates workflow.

At first we want to build the system allowing to gating rawhide,
packagers will '''opt-in into gating'''
using greenwave's opt-in gating mechanism] as they want. We will also
offer a way to '''opt-out''' so packages depending on other packages
can still be built without issues.

We do not aim at making any tests mandatory as part of these
proposals. Once the two proposals have been deployed it will be up to
the community and likely FESCo to decide if they would like to enforce
some rules on all packages and which ones.

Note that this proposal completes previous initiative such as

=== Workflows ===

Single_package_GatingRawhide_bodhi.png|Single package update with gating

== Benefit to Fedora ==

* As more packagers opt-into gating and add tests to their packages,
rawhide becomes more stable
* More stable rawhide will lead to easier composes and a smoother
release process
* Ability to use chain-build in rawhide and stable releases

== Scope ==
The scope of this work is adding a mechanism to gate single package
updates before they enter the rawhide buildroot (and thus become
accessible to others) so broken packages are kept out and cannot
affect other packages or the compose until they are fixed by their

'''The CI system, the tests and the decision on which tests are used
to gate upon are out of scope for the present document.'''

* Proposal owners: pingou will be coordinating the work among the
different stack holder

* Other developers:
** Bodhi developers: Implement the needed changes
** infrastructure: deploy the new version of bodhi and the new koji plugin

* Release engineering: <a href="" title=""></a>

* Policies and guidelines: Once the entire system is in place, FESCo
may want to set a policy on distribution-wide gating (ie: tests that
would be enforced for all packages in the distributions). This is
however out of scope of this proposal.

* Trademark approval: N/A (not needed for this Change)
=== Application impacted ===
==== Bodhi ====

* Single package update
** We will need to write a message-bus listener that will
automatically create a bodhi update for each package built in the
rawhide-gated tag.
** We will need to write a message-bus listener to automatically push
bodhi updates created for rawhide that have passed CI (the decision
being announced by Greenwave).

* Bodhi will comment on the updates when greenwave's decision about an
update is known
** This will notify users about the test results and the corresponding
gating status.

==== Greenwave / WaiverDB ====
Nothing should change for these tools but we will have to confirm this
and test in staging that they behave as expected.

==== CI system ====
Nothing should change for the CI system but we will have to confirm
this and test in staging that they behave as expected.

== Upgrade/compatibility impact ==

== How To Test ==

# Add tests to a package you maintain in Fedora using the [[
CI/Quick_Start_Guide ]] documentation (see the full [[CI|CI
documentation ]] for more information).
# Update your package in rawhide
# Ensure that the bodhi update is automatically created
# Check that the tests have properly ran (see the Tests tab in the bodhi update)
# Ensure that the bodhi update went through to rawhide once the tests passed

== User Experience ==

=== Single package update ===

fedpkg clone rpms/foobar
cd foobar/
vim foobar.spec
git commit -am "Git commit message"
fedpkg build

If the CI tests pass, the package will land in rawhide, if they fail the user
will be able to go to bodhi to see what failed and why as well as waiving the
failed tests if needed.

=== Multi package update with single package update gating ===

fedpkg clone rpms/foobar
cd foobar/
vim foobar.spec
git commit -am "Update of foobar to update foo to 1.2"
fedpkg push
fedpkg build --target=rawhide-candidate
cd ..
fedpkg clone rpms/foobaz
cd foobaz
vim foobaz.spec
git commit -am "Update of foobaz to update foo to 1.2"
fedpkg push
fedpkg build --target=rawhide-candidate
cd ..
fedpkg clone rpms/foo
cd foo
vim foo.spec
git commit -am "Update foo to 1.2"
fedpkg push
fedpkg build --target=rawhide-candidate

Users are practically by-passing the gating process by specifying a
specific build target.

== Dependencies ==

* Bodhi
** bus listener to automatically create updates when a build happens in rawhide
** bus listener to automatically push updates who passed gating
according to greenwave
** comment on updates about gating status change/announce to notify
packagers about it

== Contingency Plan ==
We can keep doing rawhide as we are doing it today.

== Documentation ==
To be written/updated

List of documentation pages to update:

* <a href="" title=""></a>
* <a href="" title=""></a>
* <a href="" title=""></a>
* <a href="" title=""></a>