This is the policy that I expect to be discussed during the Fesco
meeting tomorrow. This is entirely orthogonal to the ongoing discussions
regarding whether updates in stable releases should be expected to
provide features or purely bugfixes, and I don't see any conflict in
introducing it before those discussions have concluded.
Introduction
We assume the following axioms:
1) Updates to stable that result in any reduction of functionality to
the user are unacceptable.
2) It is impossible to ensure that functionality will not be reduced
without sufficient testing.
3) Sufficient testing of software inherently requires manual
intervention by more than one individual.
Proposal
The ability for maintainers to flag an update directly into the updates
repository will be disabled. Before being added to updates, the package
must receive a net karma of +3 in Bodhi.
It should be noted that this does not require that packages pass through
updates-testing. The package will appear in Bodhi as soon as the update
is available. If sufficient karma is added before a push occurs, and the
update is flagged for automatic pushing when the karma threshold is
reached, the update will be pushed directly to updates.
It is the expectation of Fesco that the majority of updates should
easily be able to garner the necessary karma in a minimal space of time.
Updates in response to functional regressions should be coordinated with
those who have observed these regressions in order to confirm that the
update fixes them correctly.
At present, this policy will not apply to updates that are flagged as
security updates.
The future
Defining the purpose of Fedora updates is outside the scope of Fesco.
However, we note that updates intended to add new functionality are more
likely to result in user-visible regressions, and updates that alter ABI
or API are likely to break local customisations even if all Fedora
packages are updated to match. We encourage the development of
mechanisms that ensure that users who wish to obtain the latest version
of software remain able to do so, while not tending to introduce
functional, UI or interface bugs for users who wish to be able to
maintain a stable platform.
Comments
Re: Proposed udpates policy change
By Krzysztof Halasa at 03/09/2010 - 16:36Matthew Garrett < ... at srcf dot ucam.org> writes:
True.
Definitely. IOW, the testing is never sufficient.
That means any and all updates are unacceptable.
Re: Proposed udpates policy change
By Al Dunsmuir at 03/09/2010 - 16:48Hello Krzysztof,
The whole point of an update may be the deliberate removal of
features/functionality. This includes removal of elements whose
upstream is dead, removal of conflicts between packages, conversion of
static/copied libraries to the system provided library, removal of
features which are irretrievably broken, and those elements which do
not fit within Fedora's licence/mission (mp3 support, or patented
material).
Any nontrivial piece of software contains bugs until it reaches
end-of-life. This is a simple fact of life.
You can't test quality into a product like Fedora. You can only
attempt to assist developers in discovering issues that have escaped
their unit tests, so that through iterations of design/code/test the
package becomes stable and feature complete. It takes many iterations,
across many releases for some packages.
An absolute rule containing "any" ignores reality.
The opposite of change is death. No updates as a hard and fast rule
would drive many users and packages away from Fedora.
Re: Proposed udpates policy change
By Bill Nottingham at 03/09/2010 - 15:10Matthew Garrett (<a href="mailto: ... at srcf dot ucam.org"> ... at srcf dot ucam.org</a>) said:
I agree with the sentiments here.
However, I do wonder about some of the concerns about this being
a requirement for all packages. So, here's a counter-proposal/expansion.
If need be, each of these policies could be considered separately, although
they do stack on each other.
....
Proposal
For a package to be pushed to the stable updates repository, it must
meet the following criteria.
1) All updates (even security) must pass AutoQA tests.
Rationale: If a package breaks dependencies, does not install, or
fails other obvious tests, it should not be pushed. Period. Obviously,
this proposal would not be enacted until AutoQA is live.
2) Updates that constitute a part of the 'important' package set (defined
below) must follow the rules as defined for critical path packages for
pending releases, meaning that they require positive karma from releng
and/or QA before they go stable. This also includes security updates for
these packages.
The 'important' package set is defined as the following:
- The current critical path package set
- All major desktop environments' core functionality (GNOME, KDE, XFCE,
LXDE)
- Package updating frameworks (gnome-packagekit, kpackagekit)
- Major desktop productivity apps. An initial list would be firefox,
kdebase (konqueror), thunderbird, evolution, kdepim (kmail).
Rationale: These are the sets of packages where regressions most affect
users, and would most prevent them from Getting Their Work Done.
Furthermore, while I can accept that there may be some packages in Fedora that
cannot find a significant enough testing base for all potential updates, I
reject the notion that any desktop widely used enough that we deploy a
image or spin for it would fit into that category. I accept that this places a
larger burden on QA, and would expect them to be able to contribute testing
to this initiative.
3) All other non-security updates must either:
a) reach their specified bodhi karma threshold
b) spend some minimum amount of time in updates-testing, with a tracked
number of downloads.
Proposed time would be one week, but is open to negotiation.
Rationale: We do want additional eyes on updates wherever possible. We do
have one Fedora mirror that Fedora infrastructure controls; we should
be able to mine this server for data on updates-testing downloads.
Any update that wants to bypass these procedures would need majority
approval from FESCo.
....
Comments, questions, reasoned arguments? Part of me wonders if this should be
expanded with a sliding scale for update types (enhancements, for example, get
more stringent treatment than bugfix/security).
Bill
Re: Proposed udpates policy change
By Adam Williamson at 03/09/2010 - 16:14I've thrown together an extremely quick-n-dirty combination of Bill's
proposal and Kamil's here:
<a href="https://fedoraproject.org/wiki/User:Adamwill/Proposal:Package_update_policy" title="https://fedoraproject.org/wiki/User:Adamwill/Proposal:Package_update_policy">https://fedoraproject.org/wiki/User:Adamwill/Proposal:Package_update_policy</a>
the wording is very rough, but I hope it's good enough to form a basis
for discussion.
Re: Proposed udpates policy change
By Al Dunsmuir at 03/09/2010 - 16:13This is a sane approach.
One problem with immediate implementation would be that all packages,
no matter how insignificant would need to have tests that could be
run. Some packages in categories such as firmware or cross-compilation
tools would require specialized hardware to test fully as part of the
build or subsequent AutoQA testing.
Also sane.
Again, sanity.
I think you need to separate enhancements/etc to core components from
those for lower risk packages.
Re: Proposed udpates policy change
By Adam Williamson at 03/09/2010 - 16:20What Bill's talking about when he refers to 'autoqa tests' are generic
tests which are concerned with package quality, not really the software
in the package: stuff like do the dependencies work, are there any clear
errors in the file lists. They can be run on any RPM package, the
software in the package doesn't really matter.
Re: Proposed udpates policy change
By Al Dunsmuir at 03/09/2010 - 16:28That makes sense.
How about things like rpmlint? Perhaps that would have caught the
bind/dnssec problems where user configs were directly rewritten
without backup to rpmnew files.
Re: Proposed udpates policy change
By Adam Williamson at 03/09/2010 - 16:34We have a tool called 'rpmguard' which is more or less a cousin of
rpmlint that looks at the differences between two versions of a package
and flags up critical changes. That's what will be implemented in AutoQA
and used at this 'test point'. See
<a href="https://fedorahosted.org/autoqa/wiki/rpmguard" title="https://fedorahosted.org/autoqa/wiki/rpmguard">https://fedorahosted.org/autoqa/wiki/rpmguard</a> .
Re: Proposed udpates policy change
By Dan =?ISO-8859-... at 03/09/2010 - 15:49Bill Nottingham píše v Út 09. 03. 2010 v 14:10 -0500:
Thanks Bill, this proposal is very similar to my "dump of ideas" posted
earlier today. The only thing I would like to improve (probably in
PackageKit) is the presentation of the content in updates-testing to the
users, to make it more visible and to encourage the testing. Something
like subscription to selected subset of packages that I want to be
notified about.
Dan
Re: Proposed udpates policy change
By Al Dunsmuir at 03/09/2010 - 16:13I liked the visual presentation of download stats that Adam reference.
It may have been Ubuntu or Debian related.
I would like a tool that would present the list of installed packages,
and allow one to subscribe based on that. Also, bringing it up a
level to comps might reduce the user time. Perhaps basing the
testing of dependent package testing based on the owning packages
would simplify things too.
Al
Re: Proposed udpates policy change
By Matthew Garrett at 03/09/2010 - 15:43How about every package that's installed by default in the primary
spins?
This seems pretty sane.
Re: Proposed udpates policy change
By Adam Williamson at 03/09/2010 - 15:40I agree with the idea, but it should be better phrased. It's not the way
the test is implemented that matters, but what's being tested. The fact
that the test is done by AutoQA is really neither here nor there.
It would be safer to explicitly define the types of tests we want all
updates to pass: set out exactly what the 'obvious' hurdles every
package pushed should get over are (i.e. dependencies should be sane).
*How* we choose to test that is really just an implementation detail.
This feels broadly correct to me at present, and that's what I'll say at
the meeting if input from non-FESCo members is okay. I think Matt's
proposal is far too much of a leap at present; it may be true that we
can drive much more feedback in Bodhi than we currently get, but we
should prove that *before* we push a policy that relies on it happening,
not after. I know there's a potential chicken/egg problem there, but we
haven't even really tried very hard yet.
It also has other issues that have already been discussed, that are
mainly shortcomings in the current Bodhi process. As many have pointed
out (and QA has basically accepted), relying on a simple overall Bodhi
'score' is pretty unreliable at present, because we have no really good
process for defining what positive and negative Bodhi votes actually
mean. It's just not a robust enough process to make all updates depend
on getting a hard-defined overall Bodhi score at present. It can go
wrong in both directions; updates that shouldn't get pushed can get +3
(the obvious example being a kernel that boots fine for 7 people and
breaks for 3 would score +4), and updates that should get pushed might
not get +3.
Right now we should limit reliance on Bodhi to only critpath packages at
most, and even for that we do need to tighten up Bodhi procedures quite
urgently. And it shouldn't rely on an arbitrary combined
positive/negative score.
Re: Proposed udpates policy change
By Doug Ledford at 03/09/2010 - 15:34If it's not critpath, and it passes AutoQA, then let it go straight to
stable. There are those updates that don't need a bunch of testing.
But, by defining the critpath set, and requiring positive feedback on
critpath packages, you have covered our users asses on the really
important packages. By having autoqa in place, you've covered the vast
majority of brown paper bag problems on all packages, not just critpath.
For non critpath packages, the brown paper bag stuff is all we really
need to mandate I think.
As an example, I pushed an update direct to stable a couple weeks ago.
It was to fix a bug in an init script where my use of:
udevtrigger
udevsettle
resulted in just some deprecation warnings from udev. I replaced those
calls to calls of udevadm trigger and udevadm settle and then tested
that A) the noise was gone and B) they still worked. I tested this on
all releases before building. And that was the *only* change that I
made. This update did not deserve to go through testing, it was
pretested. I would have appreciated an autoqa check to verify that
nothing in the build system broken the new package (maybe a dep change
or something), but really the changes of even that were extremely remote
at best. So, I can see where some changes simply don't warrant a bunch
of strict rules. We can have these changes even in critpath packages,
but the whole point of critpath is that we've isolated a set of packages
that are so important that it's worth it to take a little extra time
just to make sure things are right.
So, I like the policy in as much as I like AutoQA and critpath requires
testing. But since you have both of those things, I think loosening up
and letting non-critpath packages do with just AutoQA is sufficient.
Re: Proposed udpates policy change
By Kevin Kofler at 03/08/2010 - 19:18Even if it already went to testing and sit there for ages? This will lead to
MANY updates being stalled in testing forever! Hardly any update is getting
+3 karma right now. Karma is completely useless in practice. People "gaming
the system" are going to be the only way updates can go out at all under
such a proposal.
Sure, we'd all love that kind of test coverage. It's just not practicable at
all. We need to be realistic.
In addition, it has been explained very clearly in the discussions on this
mailing list why the current karma system is a poor indicator of update
stability. Relying on it as the only way an update can go stable is just
insane.
You gotta be kidding! Just look at the current update landscape to see how
far from the truth this is.
(And I also wonder with what authority you speak in the name of FESCo as a
whole. The fact that what you're saying is so clearly wrong makes this all
the more an issue.)
Kevin Kofler
Re: Proposed udpates policy change
By Jesse Keating at 03/09/2010 - 14:13The proposal is for FESCo to adopt that position. It is not a statement
of current fact.
Re: Proposed udpates policy change
By Luke Macken at 03/09/2010 - 13:26I don't think this is ideal.
I'm having déjà vu from a lot of these discussions & proposals -- we've
been here before. When I created bodhi, it initially disallowed pushes
directly to stable. This behavior... didn't sit well with The
Community, for a variety of reasons. As you can tell by the uproar in
every single one of these threads, if these same restriction are put in
place again we will lose a lot of important contributors.
Instead of re-implementing old controversial behavior, lets look at some
recent behavioral changes in bodhi and see if we can build on top of
those to solve these problems.
For example, I recently implemented various policy restrictions for
critical path/no frozen rawhide updates for pending releases. In order
for a critical path update to get marked as 'stable' for a pending
release, it must have at least +1 karma from releng/qa members, and at
least +1 from an authenticated user. These values, of course, are
configurable.
I think a much better solution would be to require similar critical path
policies, across *all* releases, not just pending ones, while still
allowing non-critpath packages to go directly to stable.
Requiring a static amount of karma across all updates is not going to be
sufficient, especially with the current karma system (which I think
needs improvement). Especially now with the easy-karma script, people can
+1 dozens of updates at a time, reaching +3 will be almost too easy for
certain packages, and yet not not easy enough for others. Having
configurable karma thresholds for different packages seems to be
sufficient, and it allows package maintainers to use their intuition to
tweak these thresholds accordingly to fit their workflow and tester
base. For example, the kernel maintainers disable karma automation
entirely, and one could argue that this flexibility is important.
Regarding the current karma system, I think Doug Ledford brought up some
good ideas in the earlier thread 'Bodhi karma feature request', and
I know other bodhi developers agree.
With an improved karma system, stricter critpath update handling, AutoQA
integration, and giving The User more fine-grained control over what
types of updates they actually want, I think we'll be much better off
than we currently are.
luke
Re: Proposed udpates policy change
By Kevin Kofler at 03/09/2010 - 14:36We also systematically disable karma automatism for KDE updates. We found
the numeric karma to be a very poor indicator of the quality of the update.
Kevin Kofler
Re: Proposed udpates policy change
By Jon Masters at 03/09/2010 - 14:48That is an acceptable fallback. But just for the record, I consider the
critical path on my desktop to include not just kernel/udev/modules/etc.
but also GNOME, cups, and other things I use each day. I don't
personally care if KDE is updated 20 times a week, or about other
packages that aren't installed by default being rebased often.
Jon.
Re: Proposed udpates policy change
By Josh Boyer at 03/09/2010 - 15:07<a href="http://koji.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt" title="http://koji.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt">http://koji.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt</a>
josh
Re: Proposed udpates policy change
By Jesse Keating at 03/09/2010 - 15:02<a href="http://kojipkgs.fedoraproject.org/mash/branched-20100308/logs/critpath.txt" title="http://kojipkgs.fedoraproject.org/mash/branched-20100308/logs/critpath.txt">http://kojipkgs.fedoraproject.org/mash/branched-20100308/logs/critpath.txt</a> is the current critpath.
Re: Proposed udpates policy change
By Adam Williamson at 03/09/2010 - 15:02The current critical path does include most of that. You can see the
current critpath list at:
<a href="http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt" title="http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt">http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt</a>
It's generated daily, it's not a static list. It does currently include
most of the important bits of GNOME, and cups-libs.
Re: Proposed udpates policy change
By Luke Macken at 03/09/2010 - 15:01Right, the critical path package list does contain the GNOME stack, cups-libs,
etc.
<a href="http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt" title="http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt">http://kojipkgs.fedoraproject.org/mash/rawhide-20100309/logs/critpath.txt</a>
luke
Re: Proposed udpates policy change
By Jon Masters at 03/09/2010 - 15:43So then great. I thought a lot of that was on there already, but didn't
have the link handy (bookmarked). Well then, to me, at least having a
very high bar on those would probably be sufficient for now at least.
Jon.
Re: Proposed udpates policy change
By Lennart Poettering at 03/09/2010 - 11:47Two questions:
How do you plan to convince people to actually test packages? My
understanding right now is that folks who end up testing packages do so
exclusively because they themselves ran into the bug that is being
fixed. And as soon as that itch they are scratching is fixed, they
seldomly report back about this. (At least that is what I am
experiencing. In quite a few cases I know that people happily took
packages from bodhi but never reported back)
Making the user Karma logic the *only* path into the stable distribution
hence makes the distro depend on feedback by our users. And I
wonder if we really want to depend on that.
And secondly: trhere are many bugs that are only triggered in very
particular use cases or on very particular hardware. Nonetheless they
are bugs that are worth to be fixed. For those it is going to be hard to
get updates accepted simply because the number of people who could and
want to test this is minimal. (And I know what I am talking of, having
to deal with PA compatibility with a vast number of different audio
hardware and drivers).
All in all, I think the Karma logic is a good tool to speed up things,
it is not useful as the *only* path into the stable distribution.
Unless of course we create some kind of body whose sole job is to test
and provide karma to updates that are not otherwise tested by the
users. And which hence can be blamed if packages stay stuck in bodhi.
Because right now there is a substantial number of updates I push that
stay stuck in bodhi for really long times because not enough people
bother... And if in the future those packages will stay there forever
because I cannot push them myself than I'd be royally annoyed.
Lennart
Re: Proposed udpates policy change
By Tomas Mraz at 03/09/2010 - 11:27I am strongly against this proposal if it is enforced on all the
packages as a whole. It might be acceptable for critical path packages
however low profile packages which are not installed on most of the
systems (and we have huge number of such packages now in Fedora) will
not get the testing no matter what magic you will cast on the users to
attract them to testing.
A reasonable compromise might be to allow the manual push to stable
after a week or so of the package living in the testing repository.
Somehow speeding up the process of getting the package to the hands of
users once it is entered into bodhi would be also appreciable.
Re: Proposed udpates policy change
By Alexander Kahl at 03/09/2010 - 10:56- -1, I would have to orphan most of my packages and give up my current
plans of founding a Common Lisp SIG as it will die suffering the
chicken-and-egg problem if this gets approval.
I'd rather fork Fedora than accept this without massive modification
(see my first post in this thread).
Alexander Kahl
GNU/Linux Software Developer
Re: Proposed udpates policy change
By Dominik 'Rathan... at 03/09/2010 - 10:26[...]
-1 to this. I've packaged a number of things that I know just one user of.
I have no idea how many people actually use my packages or how to reach
them. Therefore it will most likely be impossible for me to get +3 on each
update and they'll never get out of testing. Do you really want this?
If this gets passed I'll probably orphan most of my packages (i.e. the ones
that have very little users). There's no point in maintaining them in Fedora
under this bureaucracy.
Regards,
R.
Re: Proposed udpates policy change
By Matthew Garrett at 03/09/2010 - 11:56No, and it's an entirely fair question. I don't expect this proposal to
be passed as is - there's going to be further discussion of the details,
and the necessary conditions for something to be considered sufficiently
tested are clearly something that we need to think about carefully. If
there's a tiny number of users then we still want the package to be
tested, but hitting +3 may be implausible.
Re: Proposed udpates policy change
By Seth Vidal at 03/09/2010 - 12:23Thanks,
-sv
Re: Proposed udpates policy change
By Matej Cepl at 03/09/2010 - 08:48Dne 8.3.2010 22:59, Matthew Garrett napsal(a):
I usually decrease required karma to +-1, but I have never experienced
package pushed to stable because of karma. Does it mean that I shouldn't
bother to fix bugs in the released distros, because new release will
newer get out of updates-testing?
Please clarify.
Thank you,
Matěj
Re: Proposed udpates policy change
By Rahul Sundaram at 03/09/2010 - 06:26I don't see how we expect that for all packages to get enough karma and
while some of them can get feedback within the current infrastructure
and considering the wide variety of packages (niche libraries for
example) it is naive to believe that we are going to accomplish and
hence my counter points are:
* We need improvements in our infrastructure (easy karma is one avenue
but Pacagekit integration and other ways to get users to provide input
needs to be in place first)
* We need to consider what we need as exceptions to this rule or more
sensibly enforce this rule only in crit path packages initially
* If a time limit is considered as a alternative we need to document
ways to escalate and file a exception if necessary and again I would
recommend only consider enforcing it for crit packages first
As it current stands I am against this proposal
Rahul
Re: Proposed udpates policy change
By =?UTF-8?B?SGHDr... at 03/09/2010 - 07:00Sounds pretty sensible.
We should also keep in mind that one size does not fit all. While core
and widely used packages should have a more conservative update path,
some packages could benefit from faster release. karma mechanism +
feedback integration in PK looks like a total win for the latter.
Promoting the use of fedora-easy-karma among contributors (kudos to
Till Maas !) would be more effective than a half baked proposition
(hasty decisions are often bad ones).
2010/3/9 Rahul Sundaram < ... at gmail dot com>:
Re: Proposed udpates policy change
By Karel Zak at 03/09/2010 - 06:28You didn't explain (in your proposal) why we need this change. Do you
have any statistics about number of regressions and bugs that have
been introduced by untested/bad updates in F-11 and F-12?
I think all such changes should be always based on real experience and
statistics otherwise the change is premature optimization.
0) Fedora strongly depends on well-motivated and non-frustrated
maintainers and open source developers. We want to increment
number of responsible maintainers who are able to use common
sense. Our mission is to keep maintainers happy otherwise we
will lost them and then we will lost users and our good position
in Linux community.
Our goal is to use technical arguments and don't introduce
non-technical discussions in our mailing lists.
Always when I see that someone is trying to introduce a new rule I
have to ask myself ... why so large project like kernel is able to
successfully exist for 20 years without a huge collection of rules?
Karel
Re: Proposed udpates policy change
By Seth Vidal at 03/09/2010 - 08:55the kernel has one rule which ends up working very well. The kernel has
the "its Linus' (and a few other people's) way or the highway".
Now, if you'd like to see fedora adopt that rule, I'd be curious to see
the results.
-sv
Re: Proposed udpates policy change
By Jaroslav Reznik at 03/09/2010 - 09:10Yes, we don't have Linus here ;-) But usually I like his decisions - mostly
strongly technical ones based on arguments = less politics.
Jaroslav
Re: Proposed udpates policy change
By Karel Zak at 03/09/2010 - 09:51It's not (only) about Linus. It's about working environment and
strong focus on technical things.
Please, read:
<a href="http://www.kernel.org/doc/Documentation/ManagementStyle" title="http://www.kernel.org/doc/Documentation/ManagementStyle">http://www.kernel.org/doc/Documentation/ManagementStyle</a>
Yes, less politics.
Karel
Re: Proposed udpates policy change
By Seth Vidal at 03/09/2010 - 10:12Less politics? Seriously?
man that's funny.
-sv
Re: Proposed udpates policy change
By Stephen John Smoogen at 03/09/2010 - 13:05Its human nature. If people discuss things and someone's proposal gets
dissed/not added it gets listed as "politics and bureaucracy", if it
gets accepted "its meritocracy at its best." You see this a lot of
times on linux-kernel list (or in the sub-email lists). You also see a
heck of a lot of politics about where a patch needs to be fixed up to
be included. It just doesn't look like Fedora politics because its not
written down and does not have to be relied on ever again. [If Linus
wants one week of patches to be in the form of haiku and the next in
the form of sonnets.. well thats what people will produce.]
Re: Proposed udpates policy change
By Joe Orton at 03/09/2010 - 05:51This seems naive to me. My experience is that there are few people
willing to provide testing karma, even for relatively high-profile
packages. It took about three months to get as many people to submit
positive testing results for an httpd F-11 update in updates-testing
recently.
Regards, Joe
Re: Proposed udpates policy change
By Mathieu Bridon ... at 03/09/2010 - 09:20If the issues this update was solbing really bothered them, they would
have provided feedback earlier.
I maintain some niche packages that almost no one uses/no one would
provide karma for. But if I'm asked for a bugfix, and I do it, I want
the people requesting it to tell me that it indeed fixes the issue and
doesn't break anything else. Why would I bother if they don't care?
Re: Proposed udpates policy change
By Michael Schwendt at 03/09/2010 - 10:11Hard to comment on as the scenario is too vague. The bugfix may be
trivial. The testing may be difficult. The added responsibility
requirements ("doesn't break anything else") are scary. The level
of participation the package maintainer asks for may be considered
too much by the user. The user may have moved on already. Do you want
to wait for the next user to find the same flaw?
Because other users judge about "your" broken product without contacting
you. That may be ordinary users, who build and spread an opinion about
Fedora's quality. Or more advanced users, who would build from source
tarball themselves and tell their friends and contacts that it works while
Fedora is broken. Or article writers, who review the distribution and may
try "some niche packages", too. The bug report from _one_ user may be the
most you get. Be thankful that somebody has taken the time to contact you.
Once you've been informed about the problem, the user expects you to take
appropriate action. Or else he would not remain a user, but would join and
become a co-maintainer or take over the package. You are the one to offer
something (on top of what upstream offers). Hence users believe that you
have bigger interest than themselves in fixing things.
It may be false expectations from users towards package maintainers,
but it's reality.
Re: Proposed udpates policy change
By Joe Orton at 03/09/2010 - 09:32Hah, right. Back in the real world, most users expect free cake and
complain if they don't get it. I would not be optimistic about
resetting expectations there. For most updates I do, I end up doing the
testing myself and ignorning automated karma pushes.
Regards, Joe
Re: Proposed udpates policy change
By Josh Stone at 03/09/2010 - 14:22We re-educate. You can have your free cake but we won't frost it until
you've helped us make sure it's properly baked.
Josh
Re: Proposed udpates policy change
By Dan =?ISO-8859-... at 03/09/2010 - 04:52Matthew Garrett píše v Po 08. 03. 2010 v 21:59 +0000:
I think first should come the answer on "what problem is this proposal
trying to solve?". Is it the plain number of updates? Is it the quality
of updates? What updates failed?
Unfortunately this tries to use the "one size fits all" method, but this
can't work in the recent Fedora with the number of source packages
attacking the 10000 mark. And for sure there are packages with many
users (millions?) and on other side packages with few users (dozens or
even less). And as others said, sometimes it's hard to get even a
response from the reporter on his own bug ...
Yes, this can again work for the mainstream packages and not for the
rest. And as result the only way to get a newer version of a
non-mainstream package will be only to upgrade the whole distro with a
waiting time up to 6 months. The new release contains both bugfixes and
new features, because the author can't maintain multiple branches at a
time and is doing releases every few months - one of the open source
principles was release early, release often.
So what are the options if the situation is so serious that something
must be done:
- start automated testing of updates - let the machine work, it can do a
lot
- divide packages into more tiers - add one(?) level between critpatch
and the rest that would require better testing (for example containing
libraries used by another package), some statistics how much are
packages installed would be needed
- start creating personal repos with updated stuff - and I always
thought this is the wrong way that is used by other distros and their
users ...
- improve the updates delivery mechanism so the user can easily consume
the regular updates and also test and comment the non-stable ones, this
would also require educating the users that there can be a newer version
in updates-testing when they are not satisfied with the stable one
- every package must have a QA person, well at least 3 ...
And yes, there are some not very nice options too ...
Dan
Re: Proposed udpates policy change
By Toshio Kuratomi at 03/09/2010 - 11:42AFAICS, it's to prevent regressions from creeping into the updates repo
unnoticed. However, it's not clear what the problematic regression::wanted
bugfix ratio is.
Plain number of updates would probably be cut down by this proposal but
policy that addresses that directly is not present in the current document
-- that's been talked about on this list in other threads though.
-Toshio
Re: Proposed udpates policy change
By Jon Masters at 03/09/2010 - 00:41Hi Matthew,
Thank you to you, Josh, and others for putting effort into these
discussions. And thanks for bringing this up in the meeting tomorrow.
Some comments inline below that might be useful, or not.
Before I get into what you wrote, can I also suggest asking Fedora users
what they want? We already have an announce list, and we have things
like firstboot, and the fedoraproject start page. It would be *trivial*
to post a survey up on the website that most users are going to see, for
the next few months, and collate actual responses from end Fedora users.
If they want rolling updates, so be it and some of us are "wrong". But
for goodness sake, let's actually ask the users about it.
Indeed. I believe this is the number one problem right now. The problem
is that a single package update can introduce compound issues with other
packages that were unforeseen and untested, especially the latter. This
is an issue even for the "slow moving" Enterprise distros, and they have
entire teams of people paid to do very thorough testing before pushing
each individual update. Fedora can't quite do that, but I believe that
is therefore even more of a reason to slow down the update pace. After
all, there's a new release in 6 months, not years later (as would be the
case with an enterprise distro). I'd rather wait 6 months for some
feature than have to take time out of the day to fix a stable box.
I believe that this is possibly too limited. Aside from the obvious
abuse potential (which will always exist no matter what happens) -
obviously someone could just hate the process and decide to have a
couple of others sign off on everything no matter what - I think it
should be necessary to have a cooling off period between pushing an
update, it being voted on, and it going out live. It doesn't have to be
weeks, but it should be long enough for the person who actually reported
some bug to test that it is fixed and for others who aren't able to
devote time every day to see the update. I suggest 3-5 days.
I also suggest /considering/ implementing rolling updates rather than
pushing everything to stable. By rolling updates, in this case I mean
implementing a technical means (and this is tricky with mirrors) by
which not every user will receive the update at once. Perhaps PackageKit
already does something with random deltas for non-security updates. I
usually do updates myself manually (I upgrade stable Fedora systems only
when I am certain to have time to reboot, and then I will bite the
bullet and apply many updates at once) so I haven't noticed that.
It might be worth /considering/ something like "volatile" on other
distros. I know that's mostly for anti-spam type stuff. But perhaps some
parallel stream similar to updates-testing where users can elect to have
new features if they want them, or retain the stable release (in which
case all they get is security fixes). Basically, fedora-updates becomes
literally a separate stream disjoint from the regular fedora stream.
Jon.
Re: Proposed udpates policy change
By Toshio Kuratomi at 03/09/2010 - 11:26Note -- in the policy as written, there's no possibility of abuse because
there's no definition of what karma +1/-1 means.
In a different thread someone asked whether we wanted people to simply +1/-1
if the update installed.
In another thread, adamw said that QA's bar for acceptance is very low (he
had some examples but I'd rather he speak up than I go try to find it in the
mailing list archives :-).
This needs to be rectified as well.
-Toshio
Re: Proposed udpates policy change
By Michal Schmidt at 03/09/2010 - 04:28Hi Jon,
please do not overload the term 'rolling updates'. That's just
confusing. Perhaps 'staggered' would convey your intended meaning
better.
But what do you mean to accomplish with it?
I guess when an update with an unforseen serious regression gets pushed
to updates you want to give us a chance to react before too many users
download it.
When the broken dbus update fiasco happened it was frustrating to know
that people in one time zone after another upgrade to the broken
build even though the regression was already known. The fixed build may
have been done quickly, but because of the long time it takes from
building a package to have it appear in updates on mirrors there was
not much Fedora could do to limit the damage.
If it is not possible to shorten the minimal from-Koji-to-mirrors delay
to something on the order of one hour (wouldn't that be awesome?), then
perhaps there should be a way to blacklist known brown-paper-bag
updates in the metalinks (which are not affected by the delay). I
believe this would be more useful than staggered updates.
Michal
Re: Proposed udpates policy change
By Jon Masters at 03/09/2010 - 05:48Agreed. Henceforth corrected because I didn't really want to go there.
You got exactly what I meant. I also think Terry's suggestion of
stability levels can easily be encoded in metadata in the yum repo, if
anyone wanted to do that. Then it could dynamically change as people
voted in Bodhi, and I could set a threshold of something awesomely high
for applying updates, or say "wait until it's a week old first" :)
Jon.
Re: Proposed udpates policy change
By Chris Weyl at 03/08/2010 - 22:00Hmm. So. I have a package, perl-Moose, that has 4,667 tests run at
build time. It depends on perl-Class-MOP, which has 2,225 tests, and
it in turn depends on perl, which has 234,776 tests run at build. On
a future note, we're working on setting up smoke testing, so when we,
say, rebuild perl-Class-MOP we also run perl-Moose's tests.
If I rebuild perl-Moose, or really, any of these packages, what sort
of manual testing would you suggest we require before pushing the
update?
-Chris