DevHeads.net

New Go Packaging Guidelines landed in rawhide (koji) today

Hi,

Fedora’s new Go packaging macros landed in rawhide (koji) today.

The corresponding Fedora Go packaging conventions are therefore
EFFECTIVE for new rawhide builds. For the first time in Fedora’s
history, we will be able to perform Go package builds conforming to an
approved Fedora Packaging Guideline.

Packaging documentation:
<a href="https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/" title="https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/">https://eclipseo.fedorapeople.org/guidelines/packaging-guidelines/Golang/</a>
and approval: <a href="https://pagure.io/packaging-committee/issue/382" title="https://pagure.io/packaging-committee/issue/382">https://pagure.io/packaging-committee/issue/382</a>
The go-rpm-templates package provides more complete info.

F31 change page:
<a href="https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines" title="https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines">https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines</a>
and approval: <a href="https://pagure.io/fesco/issue/2120" title="https://pagure.io/fesco/issue/2120">https://pagure.io/fesco/issue/2120</a>

While the guidelines will feel familiar to anyone who created a Fedora
Go packages in the last two years, they DO include a backwards-
incompatible change. Making GOPATH manipulation robust required moving
the corresponding logic to %prep with a new %goprep macro.

Therefore, existing specs are expected to fail without the addition of
the %goprep call.

This is of course not the end of the road, just a key step.

It opens the way to a mass cleanup and refresh of the Fedora Go stack.
<a href="https://pagure.io/packaging-committee/issue/901" title="https://pagure.io/packaging-committee/issue/901">https://pagure.io/packaging-committee/issue/901</a>

A preview of this refresh is available here:
<a href="https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/" title="https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/">https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/builds/</a>

Enormous thanks to
– Robert-André Mauchin (eclipseo) for the gigantic work done reviewing
updating and cleaning-up all those packages, and to
– Elliott Sales de Andrade (Qulogic), that picked up maintenance of
golist and fixed many of its long-standing bugs and limitations.

Many thanks to the mock, rpm and redhat-rpm-config maintainers,
that integrated the changes, we built upon (Igor Gnatenko, Florian
Festi, Miroslav Suchý, Panu Matilainen)

The macro set supports Go DynamicBuildRequires
<a href="https://fedoraproject.org/wiki/Changes/DynamicBuildRequires" title="https://fedoraproject.org/wiki/Changes/DynamicBuildRequires">https://fedoraproject.org/wiki/Changes/DynamicBuildRequires</a>

They will be usable in mock as soon as rpm 4.15 lands
<a href="https://fedoraproject.org/wiki/Changes/RPM-4.15" title="https://fedoraproject.org/wiki/Changes/RPM-4.15">https://fedoraproject.org/wiki/Changes/RPM-4.15</a>

Use in koji or copr will have to wait for the corresponding refresh
buldsystem-side. So this part of the change is a technology preview for
now.

Best regards,

Comments

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Robin 'cheese' Lee at 06/29/2019 - 00:03

On Sat, Jun 8, 2019 at 10:35 PM Nicolas Mailhot via devel
< ... at lists dot fedoraproject.org> wrote:

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Vitaly Zaitsev ... at 06/29/2019 - 03:26

Hi,

Yes, sure, if you can that would be appreciated. The vast majority of
packages is easy to clean up (just adapt the templates in go-rpm-
templates or use go2rpm), it's just there is an awful lot of them, so
Robert-André Mauchin (eclipseo) can not do all of them in a single
pass.

If you hit one of the cases where a project needs weird stuff, or if
you do not understand something, there is help available here and in
the #fedora-golang channel.

Please just state here (or tell eclispeo directly) what you will
convert, so you do not end up doing the same work in parallel.

Quite often, eclipseo will have prepared things in his copr, just not
built them in koji yet.
<a href="https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/" title="https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/">https://copr.fedorainfracloud.org/coprs/eclipseo/golang-ng/</a>

So taking up from where he prepared, and checking the result builds in
koji and works for you, will accelerate things.

@eclipseo: please correct if I wrote something that does not make
things easier your side

Regards,

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Bob Mauchin at 07/02/2019 - 15:50

On Saturday, 29 June 2019 09:26:20 CEST Nicolas Mailhot via devel wrote:

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Fabio Valentini at 06/30/2019 - 05:53

On Sat, Jun 29, 2019 at 10:15 AM Nicolas Mailhot via devel
< ... at lists dot fedoraproject.org> wrote:
It's great to see that all this stuff is finally coming together.

I've tried to fix some of my broken packages locally, but I had
trouble getting go2rpm to run from the sources -
because AFAICT, go2rpm isn't packaged for fedora yet. Is there
anything blocking a package or has just nobody worked on it yet?

Fabio

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Jakub Cajka at 06/12/2019 - 04:39

I thought that we have agreed on Go SIG meeting with eclipseo to do this in side tag along with golang rebase(to avoid 2 rebuilds), so there is no observable breakage(if any would occur) for package builds and all packages "just" pop in using new macros and following new guidelines. Currently following packages are FTBFS due to this change <a href="https://apps.fedoraproject.org/koschei/affected-by/go-srpm-macros?epoch1=0&amp;version1=2&amp;release1=19.fc30&amp;epoch2=0&amp;version2=3.0.8&amp;release2=3.fc31&amp;collection=f31" title="https://apps.fedoraproject.org/koschei/affected-by/go-srpm-macros?epoch1=0&amp;version1=2&amp;release1=19.fc30&amp;epoch2=0&amp;version2=3.0.8&amp;release2=3.fc31&amp;collection=f31">https://apps.fedoraproject.org/koschei/affected-by/go-srpm-macros?epoch1...</a> (thanks to qulogic for this query). And still we will have to do the side tag rebuild(for this change). We could have done that in one pass without causing any observable breakage for anyone.

It seems that this change has been accepted as Self Contained Change but IMHO it is System Wide Change as it seems to affect (nearly) all Go based packages in distribution(and will require work/attention of people that are not change owners, actually not accounted for in change proposal). For past several releases I have been doing rebase of Go compiler change(yet to be filed for F31) that is IMHO comparable(maybe a bit smaller) in scope and they were always deemed by FESCO as System Wide Changes. This really leaves me confused. Could someone from FESCO clarify?

When this has been discussed, new macros have been presented to me as backwards compatible(i.e. current packages will work and build as is, although requiring refresh to adopt new features), so it has not been concern for me. Other issue that I have is that you have not communicated this change(landing the new macro package) prior it happening here or elsewhere. I'm a Go SIG member(and maintainer of the previous macros packages, which by the way are still out there) and I have not been aware of this pending change landing now.

I would much appreciate if you would communicate a bit more before landing such a big changes(go-rpm-package) in future, it would made possible to avoid some breakages, collect feedback and improve overall packagers experience. I think that communication is not easy, at least not for me, and I don't think that it is my strong skill, but we shouldn't resign on it as it is IMO crucial part of the Fedora community. Is there something that can I do to improve the information flow from my side?

JC

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Zbigniew =?utf-... at 06/12/2019 - 10:20

On Wed, Jun 12, 2019 at 04:39:07AM -0400, Jakub Cajka wrote:
<a href="https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes" title="https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes">https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes</a> says
So in this case, even though the change affects so many packages, it
falls into the "self contained change" category.

That said, the difference between "system-wide" and "self-contained"
boils down to two things: some additional data required in the change
page, and filing the change a bit earlier. In this case the additional
data is mostly there in the change page, and the change was filed early,
so even if we were to change the Change to "system-wide", the effect
would be cosmetic.

Zbyszek

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Jakub Cajka at 06/14/2019 - 08:01

Thank you for clarification :). IMO it would be great if that has been recorded in <a href="https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes" title="https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes">https://fedoraproject.org/wiki/Changes/Policy#Self_contained_changes</a>, this approach can't be really inferred from it(at least I don't see it there).

With that popped on my mind that may be calling it "early/late window change" would fit better along with recording the complexity of the change, rather then system wide, contained change. But this is really just my brain storm.

Unfortunately no early changes for me as Golang release are aligned late in to the Fedora devel cycle.

JC

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Vitaly Zaitsev ... at 06/12/2019 - 05:23

Le 2019-06-12 10:39, Jakub Cajka a écrit :

<a href="https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#Summary" title="https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines#Summary">https://fedoraproject.org/wiki/Changes/Adopt_new_Go_Packaging_Guidelines...</a>

“ This proposal consists of:
Packaging the new Go macros: go-rpm-macros
Getting the Guidelines approved by the FPC
Updating all Go libraries with the new macros
Mass-rebuilding all the Go package in a side tag ”

You complain that step 3 and 4 are separate, but that’s how it was
planed from the start up and approved in the change page.

You're conflating merging two mass Go package rebuilds (one for the new
Go compiler, and another for the new, and first, Go packaging
guidelines) with merging step 3 and 4 (which would have had other
drawbacks, that were never discussed, because that's not how we planed
things).

And BTW it was already so in
<a href="https://pagure.io/GoSIG/go-sig/issue/20" title="https://pagure.io/GoSIG/go-sig/issue/20">https://pagure.io/GoSIG/go-sig/issue/20</a> 6 months ago (though this page
is obsolete, you made us rewrite the plan in so many formats over time
I've lost track or what is up to date or not. The change page is up to
date, it’s the most recent rewrite)

Sincerely,

Re: New Go Packaging Guidelines landed in rawhide (koji) today

By Jakub Cajka at 06/14/2019 - 06:10

I guess that we have not agreed on the SIG meeting then. I don't complain and this is not in any ways personal, keep it on mind please. The change proposal predates that SIG discussion as other bit predates the change proposal. I'm pointing out that we could have avoided any breakage if we did few thing slightly differently. Currently by your actions there are several FTBFS packages, it is not really a serious issues(as I'm sure that eclipso will fix up all the packages that need it in time for Fedora 31, kudos for committing for that work :)) but it is unnecessary and avoidable inconvenience that I(and I guess others too) will have to account for(spend some time on). In my case preparing for Go rebase(I do scratch rebuilds). One of the points been also possibility to fit in the Go rebase in to that side tag, but after further discussion with eclipseo on Wednesday it will make more sense to use regular mass-rebuild for that(as I usually do) assuming that the side tag rebuild will conclude 1-2w prior to it(so I will be able to observe dist git in coherent shape).

JC