DevHeads.net

gcc/gcc-c++ removal from buildroot and more

Hello everyone,

today we finally dropped gcc and gcc-c++ from the buildroot
<https://fedoraproject.org/wiki/Changes/Remove_GCC_from_BuildRoot>. This
made 12 packages go away along with 134MB installed size. This means that
you need to add gcc/gcc-c++ in the BuildRequires (guidelines stated this
for few years but not many were following them).

Also Mark fixed bug today which was pulling in systemd in the buildroot
which was pulling gnutls/libgcrypt/nettle and stuff like that. I don't have
exact numbers what we saved here.

But looking into simple package build for f28 and f29 I see some nice trend.

F28:
DEBUG util.py:439: Install 179 Packages
DEBUG util.py:439: Total download size: 146 M
DEBUG util.py:439: Installed size: 570 M

F29:
DEBUG util.py:439: Install 144 Packages
DEBUG util.py:439: Total download size: 87 M
DEBUG util.py:439: Installed size: 425 M

Comments

Re: gcc/gcc-c++ removal from buildroot and more

By Carlos O'Donell at 07/26/2018 - 15:19

On 07/09/2018 07:03 PM, Igor Gnatenko wrote:
Great job!

In November 2015 (almost 3 years now) we added this requirement to
the packaging guidelines specifically for C and C++ :-)

"Packaging:C and C++"
<a href="https://fedoraproject.org/wiki/Packaging:C_and_C%2B%2B" title="https://fedoraproject.org/wiki/Packaging:C_and_C%2B%2B">https://fedoraproject.org/wiki/Packaging:C_and_C%2B%2B</a>

Cheers,
CArlos.

Re: gcc/gcc-c++ removal from buildroot and more

By =?UTF-8?Q?Tomas... at 07/23/2018 - 08:51

On Tue, 10 Jul 2018 at 00:17, Igor Gnatenko
< ... at fedoraproject dot org> wrote:
[..]
Igor can you post full list of packages about F29?

kloczek

Re: gcc/gcc-c++ removal from buildroot and more

By Igor Gnatenko at 07/23/2018 - 09:54

On Mon, Jul 23, 2018 at 3:43 PM Tomasz Kłoczko <kloczko. ... at gmail dot com>
wrote:

Can you describe which list of packages you are asking for?

kloczek

Re: gcc/gcc-c++ removal from buildroot and more

By =?UTF-8?Q?Tomas... at 07/23/2018 - 12:56

On Mon, 23 Jul 2018 at 15:50, Igor Gnatenko
< ... at fedoraproject dot org> wrote:
[..]
I've been asking about names of those 144 packages.
As long as concern about used disk space used during Fedora packages
builds was main reason of introduction direct BRs for C/C++ compiler
I'm 100% sure that it is possible to reduce even more such minimal
packages set.
Simple it would be good to have look on this list and I'm sure that
between many of those packages from such minimal list are some
dependencies which could removed/corrected to allow form even smaller
list.
Second propose of publishing names of those packages is to probably
tweak list of current BRs as it is not possible to build even simplest
package without any of those packages.

Just please post here result of the command like "rpm -qa --qf
'%{NAME}\n' | sort | xargs"

kloczek

Re: gcc/gcc-c++ removal from buildroot and more

By =?ISO-8859-1?Q?... at 07/10/2018 - 06:14

Thank you for pushing this forward!

One question though. I see that this works in Koji, but trying to test
this locally it does not work.

1) $ mock -r fedora-rawhide-x86_64 rubygem-abrt-0.3.0-3.fc29.src.rpm
--enablerepo=local

This still installs gcc.

2) $ mock -r fedora-rawhide-x86_64 rubygem-abrt-0.3.0-3.fc29.src.rpm
--disablerepo=* --enablerepo=local

This fails:

~~~

Warning: Module or Group 'buildsys-build' does not exist.
Error: Nothing to do.

~~~

Trying to get the latest comps from local repository:

~~~

<a href="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/db824c6dd6664943bcec0a8fcd3bda7fc855e5a787623c0da6d49deb79438fc7-comps.xml" title="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/db824c6dd6664943bcec0a8fcd3bda7fc855e5a787623c0da6d49deb79438fc7-comps.xml">https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/...</a>

~~~

This has "build" group which references gcc/gcc-c++ and does not
reference any buildsys-build group. So where is this file coming from?
Why does it differ from regular Fedora repos?

Thx

Vít

Dne 10.7.2018 v 01:03 Igor Gnatenko napsal(a):

Re: gcc/gcc-c++ removal from buildroot and more

By Kevin Fenzi at 07/10/2018 - 13:31

On 07/10/2018 03:36 AM, Igor Gnatenko wrote:
Looks like Mohan merged it a while back now...

kevin

Re: gcc/gcc-c++ removal from buildroot and more

By =?ISO-8859-1?Q?... at 07/23/2018 - 04:47

Dne 10.7.2018 v 19:31 Kevin Fenzi napsal(a):
The question is not about Fedora comps, but about comps from the Koji
repository:

<a href="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/" title="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/">https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/</a>

specifically:

<a href="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/db824c6dd6664943bcec0a8fcd3bda7fc855e5a787623c0da6d49deb79438fc7-comps.xml" title="https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/db824c6dd6664943bcec0a8fcd3bda7fc855e5a787623c0da6d49deb79438fc7-comps.xml">https://kojipkgs.fedoraproject.org/repos/rawhide/latest/x86_64/repodata/...</a>

Where are these coming from? How to fix them? I'd love to see the
examples above to behave correctly, i.e. to not install gcc.

I opened releng ticket to track this: <a href="https://pagure.io/releng/issue/7649" title="https://pagure.io/releng/issue/7649">https://pagure.io/releng/issue/7649</a>

V.

Re: gcc/gcc-c++ removal from buildroot and more

By Kevin Fenzi at 07/24/2018 - 13:48

As you can see if you look it notes those are generated by koji.

koji has a 'groups' concept. It uses different groups for different
things. There's a srpm-build group that it uses when building the
initial src.rpm, there's a build group it uses for building, etc.

We had removed gcc/gcc-c++ from the f29-build tags groups, but not the
f29/rawhide tag. I have now done so, so this should hopefully be fixed
in a few here.

Marked it fixed now. Thanks!

kevin

Re: gcc/gcc-c++ removal from buildroot and more

By Mikolaj Izdebski at 07/10/2018 - 06:31

On 07/10/2018 12:14 PM, Vít Ondruch wrote:
It should work with config generated with "koji mock-config" command:

koji mock-config --tag f29-build --arch x86_64 >my.cfg
mock -r ./my.cfg [...]

Re: gcc/gcc-c++ removal from buildroot and more

By Mathieu Bridon at 07/10/2018 - 06:24

Hi,

On Tue, 2018-07-10 at 12:14 +0200, Vít Ondruch wrote:
The groups in Koji are defined differently from the groups in the
Fedora repos.

In the repos, you can find them defined in the comps.xml file, for
example:

<a href="http://dl.fedoraproject.org/pub/fedora/linux/releases/28/Everything/x86" title="http://dl.fedoraproject.org/pub/fedora/linux/releases/28/Everything/x86">http://dl.fedoraproject.org/pub/fedora/linux/releases/28/Everything/x86</a>
_64/os/repodata/b140b2b55e06b7ba6c765d27dd9dbae905745e788a97a390be8b829
80b8c282e-comps-Everything.x86_64.xml

This contains a `buildsys-build` group, which mock installs when
creating a chroot.

In Koji, the groups can be seen with the command:

$ koji list-groups f29-build
[… snip …]
build [f29]
bash: None, mandatory [f29]
bzip2: None, mandatory [f29]
coreutils: None, mandatory [f29]
cpio: None, mandatory [f29]
diffutils: None, mandatory [f29]
fedora-release: None, mandatory [f29]
findutils: None, mandatory [f29]
gawk: None, mandatory [f29]
grep: None, mandatory [f29]
gzip: None, mandatory [f29]
info: None, mandatory [f29]
make: None, mandatory [f29]
patch: None, mandatory [f29]
redhat-rpm-config: None, mandatory [f29]
rpm-build: None, mandatory [f29]
sed: None, mandatory [f29]
shadow-utils: None, mandatory [f29]
tar: None, mandatory [f29]
unzip: None, mandatory [f29]
util-linux: None, mandatory [f29]
which: None, mandatory [f29]
xz: None, mandatory [f29]
[… snip …]
srpm-build [f29]
bash: None, mandatory [f29]
fedora-release: None, mandatory [f29]
fedpkg-minimal: None, mandatory [f29]
gnupg2: None, mandatory [f29]
redhat-rpm-config: None, mandatory [f29]
rpm-build: None, mandatory [f29]
shadow-utils: None, mandatory [f29]

The `srpm-build` group is what gets installed in the chroot to build
the SRPM (see the buildSRPMFromSCM subtask in any build task) and the
`build` group is what gets installed when building binary RPMs (see the
buildArch subtasks in any build task).

I do not know **why** we have this difference between Koji and the main
repos, I just know that we do have it.

This change needs to be reflected in Comps, so that Mock installs the
same thing in its buildroots as Koji.

Re: gcc/gcc-c++ removal from buildroot and more

By =?UTF-8?Q?Tomas... at 07/09/2018 - 20:02

On Tue, 10 Jul 2018 at 00:17, Igor Gnatenko
< ... at fedoraproject dot org> wrote:
[..]
I'm almost 100% sure that it would be possible to save probably more
by remove generate Requires dependencies using {Lib,Requires}.private
out of .pc files (which are for static linking which is not possible
to use on Fedora because only few devel packages provides static
libraries) than generate 1.7k git changes and remove gcc from minimal
set of packages. Not to mention that similar effect would be possible
to reach by add gcc to glibc-devel and gcc-g++ to libbstdc++-devel
requires.

Just one technical question about forming stub Fedora build env
(because I don't know how it is assembled).
How it is done? Just one time by create minimal build image after add
some set of new updates to official repository than snapshot and clone
such image and use it as base on start build all new packages until
next batch of packages will be pushed to repo used by build systems or
every time which comes new build request such build env is assembled
from scratch?
Using for example btrfs and snapshosts would be possible to start
adding all packages listed in BR instantly. Total storage overhead
will be only ~150MB and nothing would be necessary to download to
assemble such base build env. Cleanup all after finished build .. just
remove shanpshot.

kloczek

Re: gcc/gcc-c++ removal from buildroot and more

By Igor Gnatenko at 07/10/2018 - 06:28

On Tue, Jul 10, 2018 at 2:12 AM Tomasz Kłoczko <kloczko. ... at gmail dot com>
wrote:

I don't disagree with you. Moreover, I support this idea, but the problem
here is that you don't have way to say "hey, here is -static subpackage
which owns this .pc file, can you add auto-generated dependencies there?".
Once someone implements this in RPM (I would appreciate this because I have
use-case for Rust packages), we can submit Change Proposal and

Every build it is generating new build environment. I also agree that
creating snapshot would be nice, but this is not implemented. If you would
like to work on this - I would appreciate that. In any case, you need to
talk to Koji <https://pagure.io/koji> developers.

Re: gcc/gcc-c++ removal from buildroot and more

By =?UTF-8?Q?Tomas... at 07/09/2018 - 20:08

On Tue, 10 Jul 2018 at 01:02, Tomasz Kłoczko <kloczko. ... at gmail dot com> wrote:
[..]
Yet another question..
Did above storage used sizes are when all packages are installed with
--excludedocs and --define="%_install_langs en,C"?

kloczek

Re: gcc/gcc-c++ removal from buildroot and more

By Igor Gnatenko at 07/10/2018 - 06:30

On Tue, Jul 10, 2018 at 2:17 AM Tomasz Kłoczko <kloczko. ... at gmail dot com>
wrote:

No. While we can probably assume that no one needs docs, the install_langs
is dangerous. See the
<a href="https://fedoraproject.org/wiki/Changes/Remove_glibc-langpacks-all_from_buildroot" title="https://fedoraproject.org/wiki/Changes/Remove_glibc-langpacks-all_from_buildroot">https://fedoraproject.org/wiki/Changes/Remove_glibc-langpacks-all_from_b...</a>
for existing proposal.