DevHeads.net

RFC: Alternative -devel packaging

Consider a library like libGL. At runtime, you want the drivers it
might load to be installed. But when building an application, you just
need the library itself. If the drivers themselves have non-trivial
dependencies, the buildroot is more likely to fail to compose.

The problem, in a sense, is that the devel package requires the package
providing the API library, _and_ said required package requires the
drivers. What if instead you had (in pseudo-spec):

%package devel
Requires: %{name}-sdk

%package sdk

%package libs
Requires: foo-drivers

where -libs and -sdk have identical %files, but -sdk has automatic
provides for library sonames turned off (so it never satisfies a
runtime dependency). This would duplicate some content on the mirrors,
but not the installed system, and it would let you compose a buildroot
with only the API surface you link against.

Is this all dodging the issue that nothing agrees what Recommends:
means? And that nobody's compose tools or comps files really understand
what drivers they want, and that propagating that information is an
unbounded task? Well, yeah, a bit. I still think it'd solve a real
problem, I'm just wondering if the idea is too ugly to consider.

- ajax

Comments

Re: RFC: Alternative -devel packaging

By Nicolas Mailhot at 08/28/2018 - 06:47

Le 2018-08-07 17:33, Adam Jackson a écrit :
That's a boostraping problem. The general solution is to make our build
tools bootstrap aware, so they activate bootstrap mode as needed
automatically, instead of forcing packagers to switch the conditional
manually in spec files each time a bootstraping situation arises.

Regards,

Re: RFC: Alternative -devel packaging

By =?ISO-8859-1?Q?... at 08/28/2018 - 09:58

Dne 28.8.2018 v 13:47 Nicolas Mailhot napsal(a):
Just FTR, the logic is more or less there.

<a href="https://fedoraproject.org/wiki/Packaging:Guidelines#Bootstrapping" title="https://fedoraproject.org/wiki/Packaging:Guidelines#Bootstrapping">https://fedoraproject.org/wiki/Packaging:Guidelines#Bootstrapping</a>
<a href="https://pagure.io/packaging-committee/issue/789" title="https://pagure.io/packaging-committee/issue/789">https://pagure.io/packaging-committee/issue/789</a>

together with:

<a href="https://src.fedoraproject.org/rpms/fedora-release/pull-request/7" title="https://src.fedoraproject.org/rpms/fedora-release/pull-request/7">https://src.fedoraproject.org/rpms/fedora-release/pull-request/7</a>

So as long as you do the boostrapping iteration first with "%global
boostrap 1", then the packages have the "~bootstrap" suffix, if you do
subsequent build without the %{boostrap} macro defined, the suffix is
omitted.

Now the only issue is how to inject the "bootstrap 1" macro into the
buildroot. Obviously the tooling does not support it directly, but:

1) Modules can do this AFAIK
2) It is not that hard to adjust some basic RPM package shipping some
%{_rpmconfigdir}/macros.d/macros.* file to have this macro set.

V.

Re: RFC: Alternative -devel packaging

By Nicolas Mailhot at 08/28/2018 - 11:35

Le 2018-08-28 16:58, Vít Ondruch a écrit :

I hadn't seen the PR, it's a nice enhancement over what guidelines
already included. Kudos to everyone involved.

Having to set manually
%global boostrap 1
is the missing part I had in mind.

You need build tool support to scale to language-wide or distro-wide
mass rebuilds:
1. you do not want to waste time building in bootstrap mode anything
that could build directly in full mode
2. you do not want humans to baby sit the progress of the whole mass
rebuild

So the system should
A. build by itself as much as possible without bootstrapping anything,
B. if A does not succeed, try to solve holdovers (anything that failed
build) with a bootstrap pass,
C. if B succeeds, rebuild everything that built in B and includes
bootstrap conditionals in full mode
D. if A or C succeeded, rebuild everything against the result to make
sure it is actually self hosting.

All this without human intervention to set a specific mode or select
what to bootstrap or not.

(even without bootstrapping we should do the last D. part but I fear we
often forget it expect for specific package sets where the maintainer
has already been bitten)

Regards,

Re: RFC: Alternative -devel packaging

By Adam Jackson at 08/28/2018 - 08:27

On Tue, 2018-08-28 at 13:47 +0200, Nicolas Mailhot wrote:
If you consider buildroot size to be a metric to be reduced - and
clearly people do, see BuildRequires: gcc - then this is not just a
bootstrapping issue.

Is there a proposal anywhere for the kind of bootstrap awareness you
describe?

- ajax

Re: RFC: Alternative -devel packaging

By Nicolas Mailhot at 08/28/2018 - 09:14

Le 2018-08-28 15:27, Adam Jackson a écrit :
Sure. However I suspect your solution would work fine at first and then
quickly degenerate in tricky situations as soon as the two packages
which are supposed to provide the same files lose sync (especially if
several elements in the build compose use your pattern).

Losing sync can be just: full version can not install due to broken
deps, so your builds work with the reduced version, and is then used on
user systems with previous full versions, as the new one does not
install. Of course you could add a version-lock between full and reduced
version, so your build refuses to install if there is not an exact
match, but then you have poisoned the repo with a package that built,
but is not instalable.

It was described about a month ago as part of the discussion on mass
rebuild tools.

Basically, if building a set of packages does not succeed, and part of
the failing packages have bootstraping logic, the build tool should try
to rebuild those in reduced mode automatically, use the result to finish
all the other builds, and then rebuild everything that was build in
reduced mode in full mode. At every step in the process there is only
one package that provides the same dep.

Regards,

Re: RFC: Alternative -devel packaging

By =?ISO-8859-1?Q?... at 08/07/2018 - 12:25

On Tue, 2018-08-07 at 11:33 -0400, Adam Jackson wrote:
In my point of view, in opencv package , sdk should require -devel not
the inverse

%package sdk
Requires: %{name}-devel

or doc contains the sdk examples :
%package doc
Summary: docs files
Requires: %{name}-devel = %{version}-%{release}

In Virtualbox package, I put part of sdk in devel sub-package and the
other part in pyhton2-%{name}

%files devel
%{_libdir}/virtualbox/sdk
%exclude %{_libdir}/virtualbox/sdk/bindings/xpcom/python

%files -n python2-%{name}
%{_libdir}/virtualbox/*.py*
%{python2_sitelib}/virtualbox
%{python2_sitelib}/vboxapi*
%{_libdir}/virtualbox/VBoxPython2_7.so
%{_libdir}/virtualbox/sdk/bindings/xpcom/python

in resume I do not have any sdk sub-package , and or put that files on
docs or in devel dependent on importance ...

Re: RFC: Alternative -devel packaging

By Adam Jackson at 08/07/2018 - 16:06

I'm not so much concerned with the _names_ of the subpackages, as with
the idea of packaging the same files in multiple packages and being
careful with dependencies. I'm not sure I see an example of anything
like that in opencv or virtualbox-guest-additions, but I might be
misreading you.

- ajax