DevHeads.net

starting services in fedora

In [0] it was reported that after installation of pcsc-lite in Fedora,
no smart cards were functioning at the system. After rebooting Fedora
everything was functioning as expected.

The issue is that the pcsc daemon uses a socket-activated unit
which is installed by dnf, but not started (and hence the windows-
reminding behavior).

The text on our default services [1] mentions that a service is
"enabled by default on package installation" though there is a
distinction between 'enable' and 'start'.

The reason behind that is probably summarized in [2]:
which is understandable, but does not necessarily lead to good user
experience in Fedora. Zbigniew made a good summary of what needs to be
done for that to work [3].

What do you think overall, is that something that makes sense to
address in general, or in socket activated services only, or not at
all?

regards,
Nikos

[0]. <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1545027" title="https://bugzilla.redhat.com/show_bug.cgi?id=1545027">https://bugzilla.redhat.com/show_bug.cgi?id=1545027</a>

[1].
<a href="https://fedoraproject.org/wiki/Packaging:DefaultServices" title="https://fedoraproject.org/wiki/Packaging:DefaultServices">https://fedoraproject.org/wiki/Packaging:DefaultServices</a>

[2]. <a href="https://fedoraproject.org/wiki/EPEL:SysVInitScripts?rd=Packaging:S" title="https://fedoraproject.org/wiki/EPEL:SysVInitScripts?rd=Packaging:S">https://fedoraproject.org/wiki/EPEL:SysVInitScripts?rd=Packaging:S</a>
ysVInitScript#Why_don.27t_we

[3]. <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1545027#c29" title="https://bugzilla.redhat.com/show_bug.cgi?id=1545027#c29">https://bugzilla.redhat.com/show_bug.cgi?id=1545027#c29</a>

Comments

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 00:41

Hi all,

I'm a bit surprised that there's no reply to this fairly explosive
idea. Maybe Nikos' text is too long, so let me summarize:

tl;dr: the proposal is to start services immediately during
installation (in %post), iff they are enabled in presets and the
system is live (not a chroot or such).

This would mean that e.g. after 'dnf install gpm' gpm would be running
when dnf exits.

Zbyszek

On Mon, Apr 16, 2018 at 04:13:51PM +0200, Nikos Mavrogiannopoulos wrote:

Re: starting services in fedora

By Kevin Fenzi at 04/17/2018 - 14:05

On 04/16/2018 10:41 PM, Zbigniew Jędrzejewski-Szmek wrote:
Well, I marked it to reply to, but just hadn't gotten to it yet. ;)
Not everyone can immediately answer list posts. :)

So, as it is mentioned downthread the problems I see with this is:

a) making sure we reliably detect if we are in a live system or not
and
b) services that have assumed our current behavior and are not useful,
or are even insecure when started by default.

I think a is solveable if it already hasn't been...

For b, we will want to have a long ramp (so perhaps make this a 29 or
even a f30 change) and just fix up these things that have problems
starting / being usable / being secure when started right after install.

I can see this being helpfull in some cases for sure.

kevin

Re: starting services in fedora

By Samuel Sieb at 04/17/2018 - 14:46

On 04/17/2018 12:05 PM, Kevin Fenzi wrote:
I would assume that services that require configuration before being
useful would not be enabled by default.

Re: starting services in fedora

By R P Herrold at 04/17/2018 - 15:12

I thing this is a mistaken assumption, and that we are moving
into matters of sysadmin taste

The SSHD is enabled by default, and likely to remain so

We pre-inject a management key for root in a %post stanza of
an install, but we still need to go in and add restrictions to
keyed access only, down in
/etc/sshd/
and IP range lockdowns

The Rsyslogd is enabled by default, but has essentally useless
default settings -- no Mark service, no UDP listener, local
host loggin only

-- Russ herrold

Re: starting services in fedora

By Samuel Sieb at 04/17/2018 - 17:11

On 04/17/2018 01:12 PM, R P Herrold wrote:
Both of those are also examples of packages that you are unlikely to be
installing separately. Those both get installed by default on the
initial installation so wouldn't be affected by what is being discussed
in this thread.

Re: starting services in fedora

By Ian Pilcher at 04/17/2018 - 11:00

On 04/17/2018 12:41 AM, Zbigniew Jędrzejewski-Szmek wrote:
What if gpm is pulled in as a dependency?

(gpm may not be the best example here, but Avahi definitely is pulled in
as a dependency sometimes.)

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 13:53

On Tue, Apr 17, 2018 at 11:00:55AM -0500, Ian Pilcher wrote:
Doesn't matter. Essentially, the thinking is that if it is safe and
useful to start something after a reboot, it should be OK to start it
immediately after installation too.

Zbyszek

Re: starting services in fedora

By Nico Kadel-Garcia at 04/17/2018 - 18:42

On Tue, Apr 17, 2018 at 2:53 PM, Zbigniew Jędrzejewski-Szmek
< ... at in dot waw.pl> wrote:
It's very application and local configuration specific. The "nginx"
software, for example, may expect some very real configuration
dependencies before startup to avoid conflicting with an active
webserver on the same port. When configuration tools like chef or
ansible install new configurations, the default configuration may not
even be compatible with what they have previously set up, and the
service restart would thus fail.until the rest of the deployment is
completed. But if the rpm installation reports failure because of the
failed service start, the configuration software may never be able to
get past that reported failure.

Re: starting services in fedora

By Stephen John Smoogen at 04/17/2018 - 20:45

On 17 April 2018 at 19:42, Nico Kadel-Garcia < ... at gmail dot com> wrote:
It seems to me that we are hitting the classic workstation/server
divide here. The workstation user wants gpm and other things to start
up when they install it. They expect it to work out of the box and not
need additional configuration. The server person expects that there is
going to be a LOT of special cases that need configuration or multiple
big use cases which conflict. That means having services started could
be worse than not having them wait until a reboot.

Can we start looking at this from that sake versus trying to come up
with a single solution that meets one sides problems but not the
others. It may be that the workstation needs a "yolo" flag which
enables and starts any service which meets certain needs because that
is what that audience is expecting even if that service would not want
to be started in a server environment. [To use the gpm example, if gpm
is installed and started on certain servers that would be a security
finding which could cause fines to the user... while in a workstation
it would be the opposite :)]

Re: starting services in fedora

By Lennart Poettering at 04/17/2018 - 02:44

Hmm. While many services make a ton of sense to just be started on
install, for others this doesn't really apply... i.e. starting the
smartcard stuff or gpm right-away makes sense, since they work
out-of-the-box in a sensible, comprehensive way. This is very
different for server stuff like httpd however, which is only vaguely
useful unless configured by the admin to serve the data it's supposed
to serve... Hence I am pretty sure there needs to some per-case
deliberation in place.

That said, maybe Fedora's service preset files these days are
carefully enough written and already formalize such deliberation?

Lennart

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 04:06

On Tue, Apr 17, 2018 at 09:44:45AM +0200, Lennart Poettering wrote:
Pfff. Yes they are. See <a href="https://fedoraproject.org/wiki/Packaging:DefaultServices" title="https://fedoraproject.org/wiki/Packaging:DefaultServices">https://fedoraproject.org/wiki/Packaging:DefaultServices</a>.

Zbyszek

Re: starting services in fedora

By R P Herrold at 04/17/2018 - 11:41

Is packager install sequencing aware of all of the
various systemd dependency resolution qualifications?
Wants
After
Conflicts
WantedBy
(there are a passel more)

I know I need to go in and manually create and add files like:
/etc/systemd/system/var-ftp-pub-nfs-mirror2.mount

and then link in that file in:
/etc/systemd/system/machines.target.wants/

to get NFS working as I want -- I cannot imagine that** any **
install tool knows how to read my desires as a deploying owner

-- Russ herrold

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 14:05

On Tue, Apr 17, 2018 at 12:41:15PM -0400, R P Herrold wrote:
To make this work, we could either require that maintainers of A add
Requires(post): B, or delay the starting of services until the end
of the transaction, using a transfiletrigger. This second approach
is much more attractive. Actually we already od a delayed
'systemctl daemon-reload' after the transaction, and we could start
the services after that.

Thanks for pointing this out!

Zbyszek

Re: starting services in fedora

By R P Herrold at 04/17/2018 - 15:01

Thank you ... but:

you trimmed off and did not respond to the harder part of my
real-world example

herrold earlier:

which in this case is a RO NFS mount of a third party SAN
device, and which contains site specific matter needed for an
install needs to access to be useful

There are companion files, such as one with a RW:
/etc/systemd/system/home-nfs.mount
and more, and the RW case is much 'harder' to solve
(rootsquash, NFSv4, restricted IP ranges, more). This is for
a workstation class unit

How is chasing down a rabbit hole of unknowable configuration
possibilities, to start things during deployment, and before
hardening even vaguely 'solveable' even with unlimited **
packager ** effort? Augeas sort of tried to do this, and got
mired in complexity quicksand. Trying to enable install time
startups is in no way a 'costless' decision and adding new and
ill-defined 'requirements' for unclear reasons will tend to
reduce packager willingness to participate

As I pointed out, install order matters, and in testing alone,
the big O() complexity testing matrix explodes at a O(N^M)
rate. That is, it is simply untestable in very short order

And just WHY do we want to start services during deployment,
and before hardening? Why would we WANT to enable services
_before_ application of potential security updates recognized
and released after a media freeze? Setting up the firewalld,
particularly with the demise and eradication of host name
based resolution wrappers, is not an install time task at all,
other than
'deny all but ssh'

I do not understand the use case at all

-- Russ herrold

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 15:31

On Tue, Apr 17, 2018 at 04:01:04PM -0400, R P Herrold wrote:
Yep, I did.

Yes, I don't think we could deal with arbitrary complexity here.
But all that stuff is not something that would be subject
to being enabled by default anyway. And...

... the guidelines say that
"if a service does not require manual configuration to be functional
and does not listen on a network socket for connections originating on
a separate physical or virtual machine [...] may be enabled by
default" or "Some services which are permitted to be enabled by
default as specific exceptions."

So the default presets are mostly for services which are understood to
be "safe" (for some definition of safe, it's always a judgement call,
e.g. sshd is enabled by default).

Another thing to consider is that after a service has been
enabled, it is in principle always possible for the service to be
started:
- first, a machine might be restarted because of a power failure,
kernel panic, or a user tripping over a cable, in which case the
service will be started on reboot
- second, a transaction might be started which pulls in the target
which pulls in the service in question, and the service might
be started as a side effect.

So if a service is enabled by default and needs tuning before being
started, this tuning better happen *before* the service is enabled,
i.e. before the package is installed, if it will be enabled in %post.

I see two scenarios:
- one, you install from an image, w/o pulling updates from the
network, restart the machine, and pull in updates.
- two, you install w/ network access, pulling updates for any
packages for the initial installation.

In the second case, updated packages are installed, and in the first
case, a reboot happens before the update, so any enabled services will
be started. In both scenarios starting them immediately after
installation doesn't change things.

The use case is that for simple packages, it's simpler for the user
if the service is available immediately. My initial example with gpm
is actually good here: do "sudo dnf install -y gpm", move mouse, voilà.
Also the case from <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1545027:" title="https://bugzilla.redhat.com/show_bug.cgi?id=1545027:">https://bugzilla.redhat.com/show_bug.cgi?id=1545027:</a>
a user installs some daemon for smartcards, and expects it to be
functional without rebooting.

Zbyszek

Re: starting services in fedora

By John Florian at 04/23/2018 - 08:30

On 2018-04-17 16:31, Zbigniew Jędrzejewski-Szmek wrote:
.... unless your a lefty ;-)

OT: Why is why I wish sddm was handedness-agnostic.  It's too simple to
need anything more than click awareness, the type of click shouldn't
ever matter.

Re: starting services in fedora

By R P Herrold at 04/17/2018 - 15:47

pcsc-lite is an example with a poor history set of interaction
with the systemd and itself in a 'no card present' state

I've been bitten by that Heisenbug before [1], and was
basically told to put black tape over the warning light on the
dashboard

-- Russ herrold

1. <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1046685" title="https://bugzilla.redhat.com/show_bug.cgi?id=1046685">https://bugzilla.redhat.com/show_bug.cgi?id=1046685</a>

Re: starting services in fedora

By Zbigniew =?utf-... at 04/18/2018 - 01:34

On Tue, Apr 17, 2018 at 04:47:14PM -0400, R P Herrold wrote:
That issue is orthogonal to the issue at hand.
If it logs a spurious warning at every boot if installed, one
extra warning after installation doesn't really matter.

Zbyszek

Re: starting services in fedora

By Petr Pisar at 04/17/2018 - 00:46

On 2018-04-17, Zbigniew Jędrzejewski-Szmek < ... at in dot waw.pl> wrote:
-- Petr

Re: starting services in fedora

By Jakub Jelen at 04/17/2018 - 02:23

On Tue, 2018-04-17 at 05:46 +0000, Petr Pisar wrote:
This determination is on systemd and if anyone should know that, it
should be a init, isn't it?

From the comment #30 in bug #1545027, there is a reference to snapd
spec file, that is already doing this (using systemd).

I would not say this is a problem today, we would not be able to tell
if we can or can not start a service. systemd is powerful and can
simplify these things for us.

The problem is that it does not matter for most of the technical people
to run one more command "systemctl start foo" after "dnf install". But
it really does for non-technical ones. They usually have to reboot to
make things work (hello M$). This is also the reason why the snapd has
this post script -- it is much more (likely) to be used by non-
technical people than the smart cards daemon and less likely to be
installed out of the box.

If we want to make it simpler to use for *any* user, we should go this
way and allow starting enabled services after installation, either by
explicit post scriptlet or automatically inside of existing systemd
scriptlets.

[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1545027#30" title="https://bugzilla.redhat.com/show_bug.cgi?id=1545027#30">https://bugzilla.redhat.com/show_bug.cgi?id=1545027#30</a>

Jakub Jelen
Software Engineer
Security Technologies
Red Hat, Inc.

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 02:33

On Tue, Apr 17, 2018 at 09:23:24AM +0200, Jakub Jelen wrote:
Yep, usability and all that. One step less is always good.

My idea was to actually make this opt-out: if a service is enabled in
presets, make the default scriptlet start it too, and then provide a
new scriptlet that would give *current* behaviour and to opt-out of
autostarting.

Starting immediately should be OK for those services which satisfy the
criteria for being enabled by default, and making this automatic and
uniform would simplify things for users.

Zbyszek

Re: starting services in fedora

By =?ISO-8859-2?Q?... at 04/17/2018 - 01:30

Dne 17.4.2018 v 07:46 Petr Pisar napsal(a):
+1

Miroslav

Re: starting services in fedora

By Zbigniew =?utf-... at 04/17/2018 - 02:23

On Tue, Apr 17, 2018 at 08:30:59AM +0200, Miroslav Suchý wrote:
Actually we do. If systemd is running and dnf is installing packages
into the same root from which it is running, it's a live system.
Otherwise, it's not. This is what systemctl checks.

Are you aware of any circumstances where the existing checks done
before 'systemctl start' are not enough?

Zbyszek

Re: starting services in fedora

By =?ISO-8859-2?Q?... at 04/17/2018 - 08:15

Dne 17.4.2018 v 09:23 Zbigniew Jędrzejewski-Szmek napsal(a):
Hmm, this will work for Mock. Thou I am not sure about others environments.

Miroslav

Re: starting services in fedora

By Petr Pisar at 04/17/2018 - 03:52

On 2018-04-17, Zbigniew Jędrzejewski-Szmek < ... at in dot waw.pl> wrote:
You mean systemctl can talk to a systemd via
/var/run/dbus/system_bus_socket. Ok.

-- Petr

Re: starting services in fedora

By Michal Sekletar at 04/17/2018 - 04:51

IIRC, systemctl detects that it is running in chrooted environment
(i.e. system isn't "live") by looking at inode number of /. If the
number that systemctl sees (/proc/<SYSTEMCTLPID>/root) and one that
PID 1 see are different than systemctl assumes it is chrooted and
refuses to serve service start requests.

Michal