DevHeads.net

dropping %systemd_requires from most packages (guidelines change and mass package update proposal)

Hi,

let's drop the requirement and ordering on systemd (as implemented by
%systemd_requires) from packages which provide systemd units.

I now filed [1], which removes the recommendation to use %systemd_requires.
Quoting from that ticket:

Nowadays systemd.rpm does a preset-all call when it is installed.
This means that individual packages which provide systemd units and
call %systemd_post in their %post will work fine no matter if they
are installed *before* or *after* systemd.

If installed *after*, the sequence is:
...
1. install systemd, systemctl preset-all is called
2. install package with a.service, %systemd_post calls systemctl preset a.service
(a.service is enabled if presets say so).

If installed *before*:
...
1. package is installed, %systemd_post calls systemctl preset a.service, but
nothing happens because /usr/bin/systemctl is not found.
(The scriptlet is conditionalized using [ -x /usr/bin/systemctl ], so no
error or warning is emitted.)
2. systemd is installed, systemctl preset-all is called
(a.service is enabled if presets say so).

If systemd is not installed at all:
the service is never enabled, and there's generally no way to run the service
using systemd. This might happen for example when building a custom container
image or systemd portable. I think that's totally reasonable, if systemd should
be used in the image, it should be declared explicitly, and not pulled in as
a dependency of some random package.

During upgrades:
%systemd_postun_with_restart calls systemctl try-restart. If systemd is not
installed, than the service is obviously not running, so there's nothing to
restart.

The advantages of removing the dependency are:
1. it is easier to build custom images of various sorts, because systemd is
a big package and pulls in a *lot* of dependencies. For specialized use cases
this is totally unnecessary.
2. when those depencencies are removed, rpm has more freedom to order
the transaction. In case there are other constrains on the ordering
(that actually matter), there's less chance of circular dependencies and
it's more likely that rpm will be able to honour those other constraints.
3. "less is more" in general.

Please note that this applies to calls to systemctl preset and
systemctl try-restart. If the package calls some other systemd tool,
for example systemd-tmpfiles or systemd-sysusers, or otherwise requires
systemd to be installed, it should retain appropriate dependencies.

If this change is accepted by FPC, I'll do a mass package change using
pp privs to remove %systemd_requires use in obvious cases, i.e. when the
scriptlets simply call %systemd_{post,postun,preun}. I think it'd be great
to get rid of those dependencies for F31.

[1] <a href="https://pagure.io/packaging-committee/pull-request/890" title="https://pagure.io/packaging-committee/pull-request/890">https://pagure.io/packaging-committee/pull-request/890</a>

Zbyszek

Comments

Re: dropping %systemd_requires from most packages (guidelines ch

By King InuYasha at 05/09/2019 - 16:56

On Thu, May 9, 2019 at 10:02 AM Zbigniew Jędrzejewski-Szmek
< ... at in dot waw.pl> wrote:
In general, I think this idea has some solid foundation. But I think
it'd be dangerous to remove any kind of ordering hints from the
solution, because that means we'd need logic to handle system
bootstrap that needs to include systemd in DNF, and I definitely don't
want hard coded logic of any kind.

Would transitioning from %{?systemd_requires} to %{?systemd_ordering}
instead work to do what you want? That still offers the necessary
ordering hints to ensure systemd is set up early in a large
transaction that includes everything.

I don't relish the potential bugs that would come from losing all of
those hints in packages that we got from %{?systemd_requires}.
However, %{?systemd_ordering} would fix that issue.

Re: dropping %systemd_requires from most packages (guidelines ch

By Zbigniew =?utf-... at 05/09/2019 - 17:27

On Thu, May 09, 2019 at 04:56:25PM -0400, Neal Gompa wrote:
What system bootstrap? Please explain.

If you are talking about an initial installation in the sense of
'dnf --installroot=... install <long list of packages>', then it
shouldn't matter whether systemd is installed early or late. It is
not started in this transaction, and it should be totally OK (*) to install
it at very end.

(*) As mentioned in my original e-mail, if packages have scriptlets
which use other systemd tools, this does not apply. But in that case those
packages need an explicit dependency.

No, neither should be needed.

Zbyszek

Re: dropping %systemd_requires from most packages (guidelines ch

By King InuYasha at 05/10/2019 - 08:21

On Thu, May 9, 2019 at 5:27 PM Zbigniew Jędrzejewski-Szmek
< ... at in dot waw.pl> wrote:
systemd's preset functionality only kicks in when it installed and the
preset files exist. This means that for those scriptlets to actually
work when systemd is requested, it needs to be early in the install
order sequence.

I know that when using the systemd macros, they don't allow the
package to fail if systemd isn't available. But I strongly disagree
that allowing systemd to be installed more or less randomly is "fine"
when making full system installs (such as through Anaconda net-install
or dnf --installroot= install). With the way we call to systemd for
presets, it makes much more sense to use %{?systemd_ordering} so that
it works when systemd needs to be part of the system.

And the OrderWithRequires statement is supported in RHEL/CentOS 7, so
%{?systemd_ordering} can be backported via the epel-rpm-macros package
so that it works as intended.

Re: dropping %systemd_requires from most packages (guidelines ch

By Zbigniew =?utf-... at 05/10/2019 - 10:33

On Fri, May 10, 2019 at 08:21:24AM -0400, Neal Gompa wrote:
Yes.

No. (The scriplets themselves don't do anything, but equivalent effect
is provided by systemd package scriptlets which call preset-all.)

No.

Yes.

OK, I think I know where the misunderstanding lies.

You seem to assume that %systemd_ordering does something useful, i.e.
that it makes sense to install systemd early. It does not (*).
Please re-read my proposal, I think I explain pretty clearly why not.

Zbyszek

(*) As far as 'systemctl preset' is concerned. If the scriptlets call
other systemd tools, an ordering might still be needed.

Re: dropping %systemd_requires from most packages (guidelines ch

By =?UTF-8?Q?M=c3=... at 05/09/2019 - 10:19

On 5/9/19 9:00 AM, Zbigniew Jędrzejewski-Szmek wrote:
Is this true for the version of systemd in RHEL 7 and compatible as well?
How will this affect EPEL packages?

Re: dropping %systemd_requires from most packages (guidelines ch

By =?UTF-8?B?TWlyb... at 05/09/2019 - 14:15

On 09. 05. 19 16:19, Mátyás Selmeci wrote:
It will not affect EPEL packages because it will only be done on the master
branch and the EPEL packages are not built from the master branch.

Re: dropping %systemd_requires from most packages (guidelines ch

By Zbigniew =?utf-... at 05/09/2019 - 10:46

On Thu, May 09, 2019 at 09:19:32AM -0500, Mátyás Selmeci wrote:
systemd in RHEL generally follows the changes in Fedora, with a delay.
If this is changed in F31, then it wouldn't filter down to RHEL until
the next RHEL release. Similarly, such changes should not be propagated to
packages in EPEL7.

Zbyszek

Re: dropping %systemd_requires from most packages (guidelines ch

By J.C. Cleaver at 05/09/2019 - 13:21

On 5/9/2019 7:46 AM, Zbigniew Jędrzejewski-Szmek wrote:
RHEL8 has been out for all of two days. EPEL8 is still to come.

So long as EPEL (and Rawhide, tbh) are under the aegis of Fedora, it
would be nice at least /some/ effort was made not to toss over
incompatible changes, or a broad need for dist conditionals, across the
package ecosystem with such cavilerity.

-jc

Re: dropping %systemd_requires from most packages (guidelines ch

By =?UTF-8?B?TWlyb... at 05/09/2019 - 14:12

On 09. 05. 19 19:21, Japheth Cleaver wrote:
It's called branches.

Re: dropping %systemd_requires from most packages (guidelines ch

By Rex Dieter at 05/09/2019 - 16:39

And that's fine, being able to easily share .spec's between branches is
still nice-to-have.

I still personally use and rely on synchronized .spec's (in most branches) a
*lot*, and will likely continue to do so at leaste until I fully grok and
embrace modules.

-- Rex

Re: dropping %systemd_requires from most packages (guidelines ch

By Zbigniew =?utf-... at 05/09/2019 - 14:40

On Thu, May 09, 2019 at 08:12:15PM +0200, Miro Hrončok wrote:
Also, if for some reason I don't grok one absolutely needs to use the
exact same spec file for Fedora 31+ and EPEL7 (which is based on F19),
than keeping the dependency as it is now is also an option. One extra
unneeded dependency is not the end of the world.

Zbyszek

Re: dropping %systemd_requires from most packages (guidelines ch

By Nico Kadel-Garcia at 05/09/2019 - 19:14

On Thu, May 9, 2019 at 2:40 PM Zbigniew Jędrzejewski-Szmek

It's useful to have one spec file. Maintaining 2 or 3 spec files means
maintaining and keeping synchronized 2, or 3, synchronized git
branches or shoving in logic to detect different build configurations.
This kind of logic is already in practice for RHEL, EPEL, and Fedora
for many packages, even though it is often applied inconsistently.

Re: dropping %systemd_requires from most packages (guidelines ch

By J.C. Cleaver at 05/10/2019 - 02:21

On 5/9/2019 4:14 PM, Nico Kadel-Garcia wrote:
Not to mention that this is more or less the entire point of having an
SRPM available. Packaging means that if I need a differing version of
something on my system, I can often grab the latest .src.rpm and
recompile locally, and -- subject to ABI compatibility restrictions
(which will be being taken into account anyway) -- this often Just Works.

No RH SysEng or systems administrator is going to head to a Fedora git
branch and check out individual components as a starting point. Thus,
every tiny structural change like this made without regard to backwards
compatibility (by, say, making a macro a no-op at the distro level
instead) makes it harder and harder to actually leverage what's in the
RPM ecosystem.

-jc

Re: dropping %systemd_requires from most packages (guidelines ch

By =?UTF-8?Q?Bj=c3... at 05/09/2019 - 13:56

Am Donnerstag, den 09.05.2019, 10:21 -0700 schrieb Japheth Cleaver:

The impact of this change is not too bad considering a few things:

The use of the `%systemd_requires` macros must be changed to
`%{?systemd_requires}` in each spec file. That way rpmbuild will
evaluate whether the macro is defined and after that will fill in the
body of the macro. If it is not defined rpmbuild will simply replace
the `%{?systemd_requires}` with %nil.

That way the definition of that macro can be fully dropped from F-31+
(and thus get reintroduced at any time later, if needed for $reasons),
while still keeping the spec file fully compatible with any prior
releases (including EPEL / RHEL).

Cheers,
Björn

Re: dropping %systemd_requires from most packages (guidelines ch

By Zbigniew =?utf-... at 05/09/2019 - 14:48

On Thu, May 09, 2019 at 07:56:22PM +0200, Björn 'besser82' Esser wrote:
My idea was to keep %systemd_requires defined, but to simply remove
its use from packages in the master branches. I think that's preferable
because it makes the transition smoother: existing packages keep
working, and the dependency can be removed one-by-one without any
flag day. If some packages are not changed, there's no issue apart
from an extra dep.

OTOH, if we stopped defining %systemd_requires, it'd be necessary to
go over all packages and make sure that they don't need the dependency
for some other reason, and if they do, add the dependency in a different
form, so that it still declared when %systemd_requires go away. This
seems like more churn and more chance for breakage.

I don't think that's worth the trouble. As stated elsewhere in the thread,
we have branches for this. And even if one wants to keep the branches
the same, either conditionalizing or even keeping the extra dep are
perfectly viable options.

Zbyszek

Re: dropping %systemd_requires from most packages (guidelines ch

By Nico Kadel-Garcia at 05/09/2019 - 14:09

On Thu, May 9, 2019 at 1:57 PM Björn 'besser82' Esser
< ... at fedoraproject dot org> wrote:
RHEL or EPEL compatibility will definitely require other factors, for
RHEL 6 and 7 as well as RHEL 8. The "with_python3" options will
require parallel "with_python2" options for compatibility with RHEL 6,
and ideally disable with_python2 for Fedora 30 and 31, even if they
were simply discarded as compilation for Fedora 30. "Suggests:"
options for dnf will need to be enabled for RHEL 8, but not older
versions of RHEL. I've found it useful to compile such packages with
mock for RHEL 7 and the current Fedora to verify compatibility. The
python tests, in particular, may not have occurred with the other
version of python. (I just ran into this with python3 and subversion.)

And do not get me *going* on RHEL 8's decision to split its
installation media to have two different dnf repositories on the same
installation disk, one in a subdirectory called "BaseOS" and another
called "Appstream" for no reason I can figure out. It means yum repos
or mock setups from the DVD image need to list both repositories
distinctly, for no reason I can imagine.