DevHeads.net

Organizing a "packager experience" objective and working group

Hello,

We had a recent discussion on this list last month about (among other
things) the current state of Pagure as a replacement for pkgdb [1].

I mentioned in that discussion that there are various issues which
have arisen from the deprecation of pkgdb that have made the packager
workflow ever so slightly worse. But it's not just pkgdb-- there are
lots of places where the packager workflow could use improvement.
There are parts of the process that are tedious and manual which could
be replaced with (partial) automation, or parts where automation
exists but is in need of improvement.

For example, there are tools (namely, the "fedora-review" script) to
automate parts of the package review process. But fedora-review has
been lagging behind the packaging guidelines for some time, and has to
be manually ran by packagers over review requests. But, there's no
reason we couldn't run fedora-review automatically over every package
submission-- which might save both reviewers and submitters a lot of
time.

Or, as another example, there's currently a lot of work going on in
the distribution to support new packaging formats-- like containers
and modules. New workflows for making containers or modules out of
existing packages are being created, and I think it's vital we make
sure these workflows and processes are designed in such a way to make
things as easy as possible for packagers.

Anyway, as part of that discussion, I was encouraged to propose a new
Fedora Community Objective focused on improving the packaging
experience and workflow in Fedora. Community Objectives are approved
by the Fedora Council and intended to be 12-18 month goals for the
entire project. The goal of this Objective would be to identify
problems with the current packager workflow(s), put together a group
of packagers interested in fixing things, and then fix them!

If this sounds like something you'd be interested in helping out with,
great! The Objective will need two things, if it's to succeed:

1. People who are interested in helping! Some people did express
interest in the other thread, but I thought I would put out a general
call for interested packagers and volunteers. Anyone who is interested
and thinks they'll have some available time is more than welcome.

2. A concrete list of goals to accomplish. What glitches are there in
the current workflow, and how can they be fixed? What do you wish was
simpler, or better, or easier to do? What, basically, would make your
life easier a packager?

I've written an initial draft proposal [2] on the wiki here, though
the list of tasks to focus on is pretty sparse at present. If you are
interested in helping out, please feel free to add your own thoughts
as well.

Sincerely,
Ben Rosser

[1] <a href="https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject.org/thread/FYNU7W6KQQWA65JWVPUFDHKUP3RX6EKR/" title="https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject.org/thread/FYNU7W6KQQWA65JWVPUFDHKUP3RX6EKR/">https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject....</a>
[2] <a href="https://fedoraproject.org/wiki/Objectives/Packager_Experience" title="https://fedoraproject.org/wiki/Objectives/Packager_Experience">https://fedoraproject.org/wiki/Objectives/Packager_Experience</a>

Comments

Re: Organizing a "packager experience" objective and working gro

By King InuYasha at 01/11/2019 - 09:55

On Thu, Jan 10, 2019 at 1:59 PM Ben Rosser <rosser. ... at gmail dot com> wrote:
Of course, I'm definitely interested!

I think when topics like this come up, I think I've complained more or
less about the same things. I hope we'll be empowered to actually make
such improvements...

I'll take a look when I've got a few moments and see if I can add
something useful there..

Re: Organizing a "packager experience" objective and working gro

By Pierre-Yves at 01/11/2019 - 09:53

On Thu, Jan 10, 2019 at 01:58:17PM -0500, Ben Rosser wrote:
I'm in :)

Pierre

Re: Organizing a "packager experience" objective and working gro

By Igor Gnatenko at 01/11/2019 - 09:10

I'm definitely interested to help (I have 6+ years experience as a
packager)!

Can we have a topic on discussion.fp.o? I think it is much easier to
discuss such things there.

Re: Organizing a "packager experience" objective and working gro

By Ben Rosser at 01/14/2019 - 16:04

On Fri, Jan 11, 2019 at 8:11 AM Igor Gnatenko
< ... at fedoraproject dot org> wrote:
Thanks! Also thanks to everyone who's expressed interest in this
thread so far. :) If no one objects, I'll go through this thread and
add people's names to the wiki page.

I wanted to start the discussion here, since most/all packagers are
subscribed to this mailing list and I wanted to make sure the entire
group saw it. But I agree it makes sense to move discussions
elsewhere, either to a separate mailing list, or discussion.fp.o, or
both. Feel free to start a topic there! I guess this would go under
the "project conversation" section?

I was also thinking about creating an IRC channel... does that sound
like a good idea? (Maybe #fedora-packaging, if it is not in use? Or
maybe #fedora-packaging-qol or something instead.)

Cheers,
Ben Rosser

Re: Organizing a "packager experience" objective and working gro

By Kevin Fenzi at 01/30/2019 - 16:07

On 1/14/19 12:04 PM, Ben Rosser wrote:
I'd really think this would fall under the #fedora-devel channel.
It's not all that busy and this definitely affects developers.

kevin

Re: Organizing a "packager experience" objective and working gro

By Kevin Fenzi at 01/10/2019 - 18:42

On 1/10/19 10:58 AM, Ben Rosser wrote:
Well, the issues in the past that have come up with that:

* non free submissions. What if someone submits non free content and we
build and host it? That would not be great.
* DOS or security concerns. This is possibly not as big of a deal as it
was in the past, and could be mitigated by using copr or a seperate koji
just for this purpose.

agreed!

I'm super busy, but I think this is quite important, so I would be happy
to help out with history of things, getting you all access to make the
changes you want or at least pointing you to the right people/place to
submit changes.

Yes please, and then we should try and prioritize these since there's
likely to be a bunch. :)

Thanks a bunch for working on this!

kevin

Re: Organizing a "packager experience" objective and working gro

By Matthew Miller at 01/11/2019 - 11:18

On Thu, Jan 10, 2019 at 02:42:05PM -0800, Kevin Fenzi wrote:
Can we apply the same "flag and remove" approach as currently used in Copr?

Re: Organizing a "packager experience" objective and working gro

By Michael Cronenworth at 01/11/2019 - 13:35

On 1/11/19 9:18 AM, Matthew Miller wrote:
I'd rather have a licensing sign-off step. Allow fedora-review to automate the spec
review and build test, but break out the licensing check. Maybe in the future we can
automate that, too, but breaking out the mountain that is package review into a
small rock would still accelerate reviews.

Re: Organizing a "packager experience" objective and working gro

By Ben Rosser at 01/14/2019 - 16:18

On Fri, Jan 11, 2019 at 12:37 PM Michael Cronenworth < ... at cchtml dot com> wrote:
Well, part of fedora-review is running the license check script. I
don't think it would be too difficult to split this off into a
separate automated step, that runs over the src.rpm on the Bugzilla
ticket. Obviously this won't catch everything, but it can at least
alert the submitter (and any potential reviewers) to obvious licensing
problems. Perhaps if the license check passes, then the rest of the
review automation could run; otherwise, it has to be manually
triggered.

Or maybe we could just make the review automation use copr directly,
rather than koji, if it's easier to remove things from copr? The
packages/builds could then be deleted from copr once the package gets
approved (or removed if the review is closed WONTFIX, or something).

Ben Rosser

Re: Organizing a "packager experience" objective and working gro

By King InuYasha at 01/14/2019 - 16:23

On Mon, Jan 14, 2019 at 3:19 PM Ben Rosser <rosser. ... at gmail dot com> wrote:
I'm working on packaging Cavil[1] as an option to replace the
licensecheck stuff we currently use. The openSUSE guys have been using
this as part of their semi-automated package review process for years
now[2], and it may help us move towards less human involvement in
reviews, too.

[1]: <a href="https://github.com/openSUSE/cavil" title="https://github.com/openSUSE/cavil">https://github.com/openSUSE/cavil</a>
[2]: <a href="https://github.com/openSUSE/openSUSE-release-tools/blob/master/legal-auto.py" title="https://github.com/openSUSE/openSUSE-release-tools/blob/master/legal-auto.py">https://github.com/openSUSE/openSUSE-release-tools/blob/master/legal-aut...</a>

Re: Organizing a "packager experience" objective and working gro

By Stephen John Smoogen at 01/11/2019 - 12:24

On Fri, 11 Jan 2019 at 10:19, Matthew Miller < ... at fedoraproject dot org>
wrote:

One of the problems we are running into is that koji is a hammer and we
keep using it to do everything from putting in screws to painting the car.
Sure we can add some layers (bodhi+mbs+pungi+..) and make the hammer more
screwdriver like or more brush like.. but in the end.. it is going to leave
dent marks in the wall and car.

The issue here is that a good portion of what we are looking at isn't a
distribution. It may become a distribution later, but we are mainly talking
about someone wanting packageA to be usable in some way. [It doesn't mean
there is any work to make it integrated with anything else.. just that it
is usable.]

If you want a build system which has different requirements, make or clone
a different build system for those requirements.

* FBS == koji + bodhi + mbs + pungi + osbs + etc etc etc.

Re: Organizing a "packager experience" objective and working gro

By Stephen John Smoogen at 01/11/2019 - 12:35

If we want a build system which has different requirements, let us make or
clone a build system for those requirements.

Re: Organizing a "packager experience" objective and working gro

By King InuYasha at 01/11/2019 - 11:50

On Fri, Jan 11, 2019 at 10:19 AM Matthew Miller
< ... at fedoraproject dot org> wrote:
My understanding is that it's a lot harder to completely remove stuff
from Koji, Dist-Git, etc. than it is from Copr.

And there's the whole "Red Hat really doesn't want it in there to
begin with" thing...

Re: Organizing a "packager experience" objective and working gro

By Artur Iwicki at 01/10/2019 - 15:47

Thanks for bringing some attention to this, Ben. I don't do a whole lot of packaging, but I admit there are some places where the workflow could be improved. I'd be willing to join the group and help where I can.

Some notes off the top of my head that I ran into recently:

- fedpkg hides quite a few tools underneath, and it's not always clear which tool is running. This becomes a nuisance when a tool requests a password, since quite often you just get a "Password: " prompt. I think bodhi does exactly that, and every time it does I scratch my head trying to remember if I should provide the GPG passphrase, FAS account password, or something else.

- Related to the previous one, error messages could be improved. Recently I had to renew my Pagure token so I could request a new repo. I renewed my token, found the config file, updated it... and nothing changed. Turned out it was the wrong config file - old, legacy location maybe? Don't know. My point is, it'd be helpful if every error that says something like "value X in config is wrong" would also specify which config file it's talking about.

- fedpkg clone performs the clone over ssh, instead of HTTPS, so if you're not logged into your FAS account, you get an "access denied" error, which is as amusing as it's irritating, given that the package repos are public.

- Now that I've mentioned it, maybe we should add something like "fedpkg fas-login"? Personally I've put "alias koji-init='kinit my-FAS-account@my-domain.org'" in my .bashrc, because looking up how to solve the "koji says I'm unauthenticated" error got boring after the third time.

Cheers,
A.I.

Re: Organizing a "packager experience" objective and working gro

By Till Maas at 01/15/2019 - 10:20

IMHO you can take this one step further. Instead of telling that one is
unauthenticted it should start the authentication and ask for
credentials.

Kind regards
Till

Re: Organizing a "packager experience" objective and working gro

By =?ISO-8859-1?Q?... at 01/11/2019 - 05:08

Dne 10. 01. 19 v 20:47 Artur Iwicki napsal(a):

If you used Gnome Online Accounts, you would not need commands/scripts
like this.

Vít

Re: Organizing a "packager experience" objective and working gro

By =?iso-8859-1?q?... at 01/11/2019 - 08:51

Vít Ondruch < ... at redhat dot com> wrote:
I've seen that mentioned a few times. What exactly is this "Gnome
Online Accounts", and is there a reasonable way I can use it without
having to endure the rest of Gnome 3?

I want my FAS passkey stored in a keyring, encrypted with a master
passphrase, and I don't want to have to authenticate in advance with a
separate command. When I do for example "fedpkg build", I want to be
prompted for the master passphrase to unlock the keyring, unless the
keyring is already open. Then the FAS passkey shall be fetched from
the keyring and used in the Kerberos authentication, while the master
passphrase never leaves my workstation. But I don't want this at the
cost of having my productivity hampered by Gnome 3.

I have poked around trying to find some "Online Accounts" program or
package, but I didn't find anything I could run from another desktop.

Björn Persson

Re: Organizing a "packager experience" objective and working gro

By King InuYasha at 01/11/2019 - 09:16

On Fri, Jan 11, 2019 at 7:52 AM Björn Persson <Bjorn@rombobjörn.se> wrote:
GOA is an aspect of the GNOME Desktop that supports linking online
accounts to your user. When you log into Google, ownCloud or
Nextcloud, and so on during initial setup, those are stored in GOA.

GOA has the ability to store kerberos logins and initialize them as
part of your login session, too.

You cannot leverage GOA without the GNOME Desktop. KDE has its own
system, but I am unsure if it has Kerberos support like GOA does.

Pantheon Desktop has its own thing too, but that doesn't support Kerberos.

Sorry, you have to take the good with the bad in this case, sadly. I
live with the constant kinit runs because GNOME is too much for me to
bear. I consider it an acceptable trade-off really. The only reason
GOA has Kerberos support is because of all the enterprise desktop
stuff.

Re: Organizing a "packager experience" objective and working gro

By =?iso-8859-1?q?... at 01/11/2019 - 09:46

Neal Gompa < ... at gmail dot com> wrote:
Every time I log in locally on my workstation? That's not good. I
suppose that's suitable in a centralized corporate or university
network, but as an infrequent Fedora contributor I'll of course log in
only when I actually access Fedora systems.

But that doesn't matter to me as long as it's tied to Gnome 3.

Björn Persson

Re: Organizing a "packager experience" objective and working gro

By Christopher Brown at 01/11/2019 - 05:42

That's good to know!

I have updated
<a href="https://fedoraproject.org/wiki/Join_the_package_collection_maintainers" title="https://fedoraproject.org/wiki/Join_the_package_collection_maintainers">https://fedoraproject.org/wiki/Join_the_package_collection_maintainers</a> to
reflect this.

Thanks Vit.

Re: Organizing a "packager experience" objective and working gro

By =?ISO-8859-1?Q?... at 01/11/2019 - 06:22

Goog small update, thx. I moved the hint into the "Kerberos" admon and
now it is almost perfect IMO ;)

Just noticed the note about ~/.fedora.upn file, what is it about? I
don't have such file on my system, therefore I assume I don't need it,
because my FAS name is the same as my system user name. Is that correct
assumption? Or on different page I see "This is not actually needed for
Kerberos but for other tools", so what are the other tools?

Vít

Dne 11. 01. 19 v 10:42 Christopher Brown napsal(a):

Re: Organizing a "packager experience" objective and working gro

By Artur Iwicki at 01/11/2019 - 06:28

Just checked myself, I don't have the file either, and I use a separate local account for RPM packaging (that uses a different name than my FAS login). No idea what that's about, I can't recall any tool ever complaining about the file missing.

Re: Organizing a "packager experience" objective and working gro

By Fabio Valentini at 01/11/2019 - 06:45

On Fri, Jan 11, 2019 at 11:29 AM Artur Iwicki < ... at fedoraproject dot org> wrote:
For example, ~/.fedora.upn is used by fedpkg as the "authoritative"
source of the FAS username.
But, if that file isn't present, it just silently falls back to
checking kerberos for the username, and if that doesn't work, it uses
$USER.

(See: the Command.*user* methods in __init__.py of both rpkg and fedpkg)

Fabio

Re: Organizing a "packager experience" objective and working gro

By Kevin Fenzi at 01/10/2019 - 18:57

On 1/10/19 11:47 AM, Artur Iwicki wrote:
Yeah, thats bad and should be fixed.

Yep. another good one to clean up. :)

Well, there's some history here. packagers can use ssh because they have
actual accounts on the machine (restricted to just git commands). If you
are not in the packager group however you cannot use ssh.

Since support for https pushing was added, we should likely change the
default in fedpkg to just use that and only setup ssh if asked. For
https pushing you only need to have a browser and a kerberos ticket the
first time you clone the repo (to get a token). After that you have a
token and don't need to login unless you get a new token somewhere else.
(Only one is valid at a time).
Well, not sure how much it matters if we switch to https pushing.. but
sure.

kevin

Re: Organizing a "packager experience" objective and working gro

By Dridi Boukelmoune at 01/10/2019 - 18:36

Mine is called fedinit :)

I think fedpkg login would be enough since to my knowledge FAS is
always the authority (except for rhbz). Even better, when tools like
koji are invoked via fedpkg the latter could take care of checking
whether you have a valid kerberos ticket (or whatever it is that
kerberos works with) and kinit for you when it's not the case.

Dridi

Dridi