DevHeads.net

Delegating hinting for autopkg failures to core developers and MOTU

Please can we delegate the hinting for autopkg test failures to core developers
and MOTU?

When ignoring an autopkg test failure, you usually have a reason to do so. As a
core developer you already can work around autopkg test failures with uploading
a work-around in a new package version, both for main and universe packages.
The disadvantage is a longer turn-around, re-triggered autopkg tests,
introducing a delta which has to be maintained. MOTU could be limited to only
hint failures for universe packages.

The current limitation of autopkg related hints being done by the release team
seems to be a technical limitation, because other hints are done by the release
team only. Of course there should be informal restrictions for hinting during
archive freezes, release freezes.

Matthias

Comments

Re: Delegating hinting for autopkg failures to core developers a

By Steve Langasek at 04/10/2019 - 21:11

Short answer: not without some engineering work around britney; and I don't
think that work would be the best use of someone's time who was trying to
improve development velocity in -proposed.

The autopkgtest hints are done through hints files in a bzr branch which is
owned by the ubuntu-release team. The hints files don't just control
whether to ignore autopkgtest failures; they control how to override all
manner of checks in britney, they control the behavior during the freeze,
etc. It would be work to decouple auotpkgtest hint privileges from the
rest.

I would definitely be -1 on giving all Ubuntu Developers (or core-devs, or
MOTU) free reign on being able to commit the other sorts of hints.

And I think that the more important work is to fix the known bugs, that the
release team has agreed should be addressed, that cause us to have to
manually add hints to override autopkgtest failures when they are not
regressions vs the release pocket. The vast majority of time spent on
autopkgtest hints is on either proving that a failure is not a regression vs
release, or manually adding the hints for these non-regressions; and I think
it's much more valuable to reduce the amount of human effort that goes into
managing autopkgtest regressions than to increase the pool of humans that
have access to override britney.

Ubuntu developers *can* work around autopkgtest failures in package uploads
by disabling tests. Most developers *don't* do this, because they recognize
that disabling tests that are pointing out actual bugs in the software, in
order to let that software migrate to the release pocket, does not generally
serve the goal of releasing a high-quality OS.

Nevertheless, I see a lot more willingness on the part of developers to
ignore regressing autopkgtests in britney - even though this has a much
GREATER negative impact on the quality of a release. Unlike disabling tests
in the test suite, a hint to override autopkgtest failures cannot be
selective, so any future further regressions in the test suite would not be
caught, completely negating the value of those tests for CI.

I think it's correct, in terms of encouraging the behavior we want, that it
is harder to get an autopkgtest override in britney than it is to fix the
testsuite in the package.

For this reason, I'm also -1 on the idea of giving non-release-team members
direct access to override autopkgtest regressions as well.

Re: Delegating hinting for autopkg failures to core developers a

By Simon Quigley at 04/10/2019 - 17:31

Hello,

On 4/10/19 12:20 AM, Matthias Klose wrote:
I generally disagree with this. The Release Team is ultimately
responsible for ensuring the archive is releasable; if we let every
Ubuntu Developer hint packages, the Release Team would ultimately be
giving up enforcement of what is supposed to be a hard freeze at some
points in the cycle, and what are generally supposed to be clear release
goals.

If the role of the Release Team were to be dissolved and the power were
to be given to individual developers, I doubt we would be able to keep a
stable release pocket, which hurts both our users and our customers. If
I believe a package should be hinted, I always discuss it on
#ubuntu-release; sure, sometimes they don't have to follow up with me
about it and the Release Team just does it, but I'm not perfect, and
productive discussion out in the open about hinting a package not only
fosters understanding for prospective developers, but it allows for a
higher-quality release.

I am aware of a handful of Ubuntu Developers who regularly either come
to me personally or to #ubuntu-release because they don't quite
understand the ramifications behind hinting something or are lost in
proposed-migration. From a DMB perspective, while we are working towards
setting forth clearer expectations, we don't always thoroughly verify
that a candidate knows top-to-bottom how proposed-migration works. I
personally have faith that the vast majority of Ubuntu Developers know
general Ubuntu processes and packaging, but I don't have faith that a
majority can walk through p-m past simple cases.

Where I would tend to agree with your general point is, we should have a
clearer process on becoming a member of the Release Team (and a more
diverse set of people on the team). If an Ubuntu Core Developer very
evidently knows proposed-migration and the ramifications of hinting
packages, they should be empowered to make rational decisions rather
than having to nag the Release Team all the time. To be clear, I'm not
suggesting the Release Team begins adding people left and right, and the
candidates should be high quality, but the Release Team has 7 members at
the moment; one of which is a community member, the rest work for
Canonical and (last time I checked) either are members of Canonical
Foundations or once were, with the most recent two members being added
in 2016 and 2017 (the rest are in the 2006-2012 range).

Thoughts on this, especially from the Release Team, would be appreciated.

Thanks,

Re: Delegating hinting for autopkg failures to core developers a

By Christian Ehrhardt at 04/10/2019 - 01:50

On Wed, Apr 10, 2019 at 7:21 AM Matthias Klose < ... at ubuntu dot com> wrote:
Hi Matthias,
I like the idea, but to be specific I assume you only refer to the
hinting done in the current dev release.
Changes to hinting for released versions should stay with the SRU Team
only right?

/me has removed a lot of suggestions how to implement things I tried
to write as answer at first.
I think we need to focus on discussing what we'd want before we go
into how to achieve that.
Therefore lets see what the discussion brings, as all of this is a bit
mix-and-match depending on preferences.

Personally I'd appreciate something staged like:
A/B depending on how much we want to trust each other :-)
A)
- overrides to main stay with the release team
- overrides to universe can be committed by core-devs (keep some
formal checking)
B)
- overrides to main can be committed by core-devs
- overrides to universe can be committed by MOTUs

Re: Delegating hinting for autopkg failures to core developers a

By Matthias Klose at 04/10/2019 - 01:57

On 10.04.19 07:50, Christian Ehrhardt wrote:
yes, this should be about devel only.