RFC: make $releasever return "rawhide" on Rawhide

Hello devel list,

this is a request for comments for a recent proposal I filed at releng
<a href="" title=""></a>

In short, package managers on Rawhide would no longer replace $releasever
variable with a numerical value (like '29' at this moment, soon '30'), but
with 'rawhide' string instead. I hope this change will make life a bit
easier for third-party repos maintainers, for users, for developers and
sysadmins, and for release engineering. The technical implementation can be
seen in the ticket (it's the 'Proposed solution 1'), together with a long

To provide a longer explanation of the improvements I expect this to bring:
* Third-party repo maintainers will no longer need to provide two different
repo files, one for stable Fedora releases using $releasever in URLs, and
one for Rawhide hardcoding 'rawhide/' in URL and avoiding $releasever in
URL. (Technically, two repo files are not needed if you always use a
numbered dir even for Rawhide, but that's maintenance-heavy, because you
need to change the directory number precisely at Branching time). This
involves COPR as well.
* Users will be able to run commands like "dnf ... --releasever=28" even on
Rawhide. That doesn't work at the moment, because most repo files don't use
$releasever and instead have 'rawhide/' hardcoded.
* Developers and sysadmins will be able to use the same approach regarding
repositories for stable Fedora releases and Rawhide. Rawhide will no longer
be different, requiring special treatment. For example, the same repo URL
can be used to install a system, or the same URL can be used to add an
additional repository to an existing system. As an engineer working on
automation, I was always annoyed how I need to special-case Rawhide
everywhere (and of course, maintain a config file that states which release
number Rawhide currently maps to). That will hopefully be no longer
necessary, or very much reduced.
* Fedora release engineers should be able to get rid of
fedora-repos-rawhide (again, hardcoding 'rawhide/' in URL), and use the
standard repo files instead (making use of $releasever).

There might be other advantages, which I haven't tested or though of.

There are also disadvantages. The only one I know of at this moment, is
that PackageKit is currently incompatible with this change (it uses custom
logic for populating $releasever, different from dnf logic) and will need

Fedora releng has pre-approved this change in the ticket, and the point of
this email is to ask for more feedback from all of you. I'd appreciate if
you could help us identify edge cases we haven't thought of, or point out
which tools would be incompatible with this change, so that we can track
them and discuss it with their developers.



Re: RFC: make $releasever return "rawhide" on Rawhide

By King InuYasha at 08/02/2018 - 08:04

On Tue, Jul 31, 2018 at 9:59 AM Kamil Paral < ... at redhat dot com> wrote:
This might surprise you, but I actually prefer the current way. It
makes activating Rawhide an explicit action that can stay carried
forward. The other thing is, realistically, few third party folks try
to build for Rawhide because Rawhide snapshots are either too old or
too broken.

Case in point, the Docker containers for Rawhide effectively look like
Fedora 28, so what's the point? And upgrading to the latest released
compose just breaks everything, so it's not useful there either.

This change makes no sense unless we were actually going to make
Rawhide something that people could rely on. And I'm not sure that's a
good idea, since we have a nice cadence of releasing every 6
months(-ish). It's already too hard to keep Rawhide working because of
GCC breakages and the DNF stack work, and upstreams rely on our
Rawhide tree to suss out these kinds of things.

And I would argue that special casing Rawhide is sort of the point,
but I have no objection to making dnf --releasever=rawhide distro-sync
also work. I just don't think it's smart to drop the release number
thing and the fedora-repos-rawhide package.

Re: RFC: make $releasever return "rawhide" on Rawhide

By Kamil Paral at 08/03/2018 - 07:25

It's going to stay explicit and persistent, as explained by Kevin.

I'm not sure if you're aware, but that's exactly what many people are
attempting to do. There are plans for automated gating in Rawhide. And
since I was one of those who were at least partially involved in this, I
proposed this change so that life is easier whenever you try to automate
stuff across releases, including Rawhide. I'm not saying it's a silver
bullet, and I hope I haven't missed any important use cases (hence this
discussion), but it should be one step closer to more consistency across
releases, therefore easier operation and automation.

Re: RFC: make $releasever return "rawhide" on Rawhide

By Kevin Fenzi at 08/02/2018 - 18:24

The same for the proposed change, once you install fedora-release from
rawhide you are on rawhide until and unless you intentionally switch.

I've been looking into this the last week or so by chance. Rawhide does
compose containers every day with rawhide compose, they are just not
correctly uploading to our registry. Hopefully this will be fixed soon.

I don't think the answer to something being old or broken is to sigh and
wander off. We need to fix those things, and I think we are making
progress on doing so.

I'm not sure I follow here... you don't think we should make rawhide
something to rely on because we have regular releases?

In any case I think rawhide is very useful and without it our stable
releases would be vastly more diffcult. We can definitely do better to
make it stable, but I think it's quite usable.
The number will keep working too. We can make that an alias in
mirrormanager. So, for example if we had this implemented now and we
branched 29 off, '29' would point to the branched release, '30' or
'rawhide' would point to rawhide. If you installed fedora-release from
rawhide it would keep you on rawhide, if you install from branched or
distro-sync to the branched fedora-release (by doing a 'dnf
--releasever=29 distro-sync fedora-release') you go on branched. This
means you don't need to worry about fedora-release-rawhide and
enabling/disabling repos, and makes everyone's life easier.


Re: RFC: make $releasever return "rawhide" on Rawhide

By Kamil Paral at 08/03/2018 - 07:35

To be fair here, if COPR and other third party inherited the "single repo
for all releases, using $releasever" approach as suggested, the number
would work for internal Fedora repos (because it's easy to do so in
mirrormanager), but it wouldn't work for COPR and other third-party repos
(because they don't have a mirrormanager). Of course they could make it
work by maintaining symlinks, just as now they can make it work by
maintaining symlink - but they don't.

The question is what is the important use case for using a number in repo
network requests, if we make it work by default (using releasever=rawhide).
Would anyone need to side-step dnf to perform their own calls, and this
would complicate the life for them? I don't know, tell me. (We could make
the releasever value easily obtainable in that case, either through dnf or
through evaluating an rpm macro).

Re: RFC: make $releasever return "rawhide" on Rawhide

By King InuYasha at 08/02/2018 - 21:19

On Thu, Aug 2, 2018 at 7:12 PM Kevin Fenzi < ... at scrye dot com> wrote:
I'm saying that if we try to pull off something similar to openSUSE
Tumbleweed with our current infrastructure and tooling, upstreams like
GCC and glibc could no longer leverage our Rawhide to validate their

It's not usable whenever we have compiler transitions or add more
weird GCC plugins, and that's something I've accepted will never
change as long as we're the distribution that helps make GCC great.
And that's fine.

But the problem is that it does make things slightly harder. And
you're not quite correct about this. The way that DNF gets this value
is through identifying the package that provides "system-release" or
"distribution-release" and identifies the version set for the package.
That version is what propagates to set $releasever. Hilariously,
PackageKit independently reads VERSION_ID from os-release(5) to
determine this. These don't always agree. And in addition, it's
impossible to stay on Rawhide through PackageKit without the controls
through fedora-repos-rawhide forcing it.

And how do you propose people sync down from Rawhide to "branched"? Or
sync up from an old Rawhide to "branched"/"stable" which this change?
From what I can tell, that wouldn't be easy.

Re: RFC: make $releasever return "rawhide" on Rawhide

By Kamil Paral at 08/03/2018 - 07:50

Not quite. I have tested the patch in "Proposed solution 1" in the releng
ticket and it works. The code is here:
<a href="" title=""></a>

If you set
system-release(releasever) = value
then 'value' is used instead of identifying the package version.

If you see a logical mistake somewhere, please tell me. But in my testing
dnf correctly used 'value' (i.e. 'rawhide') with this patch.

Yes, that's mentioned in the ticket and in the introductory email here.
I'll need to ask PackageKit developers to use the DNF logic instead of
using a custom one. This is actually already "broken" - having two package
managers, each detecting releasever in a different manner - it just has
never been a problem in Fedora, because they've both arrived at the same
answer so far.

Again, it's in the ticket. Why wouldn't this work? (Once Branched == F29
and Rawhide == F30)

sudo dnf distrosync fedora-release\* --releasever=29

It should work with the same command as above, shouldn't it?

Re: RFC: make $releasever return "rawhide" on Rawhide

By Michal Novotny at 08/02/2018 - 08:14

If we can make rawhide something that people can rely on,
the actual releases will benefit from it as well.

Re: RFC: make $releasever return "rawhide" on Rawhide

By =?ISO-8859-2?Q?... at 08/02/2018 - 06:23

Dne 31.7.2018 v 15:04 Kamil Paral napsal(a):
Just note that fedora-rawhide.repo has:

Which now points to /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-29-x86_64 (on x86_64 machine).

So either this file need to be renamed to RPM-GPG-KEY-fedora-rawhide-x86_64 and updated on every branching or symlink
need to be provided.


Re: RFC: make $releasever return "rawhide" on Rawhide

By Kamil Paral at 08/03/2018 - 05:11

Interesting, this is something Kevin mentioned during the releng meeting,
which I didn't completely get:

17:09:15 <nirik> also would fix a long standing issue around moving to
rawhide from a stable relase and gpg key...17:09:35 <nirik> because
the gpg key file is called 'whatever-rawhide' not 'whatever-29'

It seems he was mistaken then?

But yes, the files would have to be renamed, thanks for a reminder.

Re: RFC: make $releasever return "rawhide" on Rawhide

By Kevin Fenzi at 08/04/2018 - 16:59

On 08/03/2018 02:11 AM, Kamil Paral wrote:
Yeah, we finally did change it... for a long time it was rawhide which
confused people who passed the number.

Likely here we want a link from "rawhide" to the current number and
change that at each branching.


Re: RFC: make $releasever return "rawhide" on Rawhide

By =?ISO-8859-1?Q?... at 08/01/2018 - 04:36

I'm somehow missing the point probably, but as a rawhide user, when I
want to take some package from F28 (typically kernel), then I have to do
"sudo dnf update --disablerepo=* --enablerepo=updates{,-testing}
kernel\* --release 28", i.e. the problem is that I have to enable the
"updates" repositories and that is about organization of the packages,
not about the 28 vs Rawhide. So how would your proposal help with this?


Dne 31.7.2018 v 15:04 Kamil Paral napsal(a):

Re: RFC: make $releasever return "rawhide" on Rawhide

By Kamil Paral at 08/01/2018 - 07:09

The current situation is that 'fedora', 'updates' and 'updates-testing'
repos are disabled in Rawhide, and instead a single 'rawhide' repo is
enabled (with hardcoded paths). It's true that if you specifically disable
the rawhide repo and instead enable the fedora/updates/updates-testing ones
(note: you're missing the fedora repo in your example), the command does
work. But it's again one example of what you need to do differently on
Rawhide than on other releases. I'd like to see the same approach wherever
you are.

With the new approach, you could do simply this:
$ dnf list/download/repoquery/whatever package --releasever=28
the same way as you do this on other releases.

Now, there are two ways how this can be handled by Fedora releng. Either
they only enable the 'fedora' repo on Rawhide, and then if you wanted to
access updates/updates-testing repos with this command line, you'd need to
add --enablerepo=updates and/or --enablerepo=updates-testing. (You can say
that this is still consistent with stable releases, it's just that the
general expectation is that 'updates' repo is always enabled). Or they
enable 'updates' repo by default on Rawhide as well (the same way stable
releases have it), and they point it to an empty repo dir (they can even
set it to never expire or almost never). In that case no --enablerepo will
be necessary, and this will be 100% matching stable releases approach.

I admit the end-user benefit here is very small (except for consistency and
perhaps documentation). When you start automating tasks and need to run
such commands on different Fedora releases including Rawhide, the benefit
might be larger.

Re: RFC: make $releasever return "rawhide" on Rawhide

By =?ISO-8859-1?Q?... at 08/01/2018 - 07:56

Dne 1.8.2018 v 13:09 Kamil Paral napsal(a):
Obviously I am not enabling "fedora" repo, because it is too old. But
now I see what you are talking about. I should have enabled "fedora"
repo all the time and it should work the same way for f28 as for the
rawhide. This would make the life easier indeed.