DevHeads.net

RFC: -Wl,--as-needed by default

Hash: SHA256

Hello,

I'm interested why we still don't have this flag in our CFLAGS? It seems that
other distributions like openSUSE enable it by default and it helps in many
cases to avoid over-linking (for example, see thread about poppler).

Are there any reasons not to add it?

Comments

Re: RFC: -Wl,--as-needed by default

By =?UTF-8?Q?Bj=c3... at 11/13/2017 - 05:52

Am Montag, den 13.11.2017, 11:02 +0100 schrieb Igor Gnatenko:

Hello Igor,

that specific flag should be in LDFLAGS, but there are reasons, we do
NOT have it in there, because it will likely break any binaries built
from or containing FORTRAN sources. They will simply SEGFAULT, because
`-Wl,--as-needed` causes some needed runtime libraries NOT being linked
with them, because the linker thinks they are not needed / would over
link.

Cheers,
Björn

Re: RFC: -Wl,--as-needed by default

By =?UTF-8?Q?Tomas... at 11/13/2017 - 17:01

On 13 November 2017 at 10:52, Björn 'besser82' Esser
< ... at fedoraproject dot org> wrote:
[..]
If it is still any issue related to the Fortran it is nothing more
than just plain bug.
However AFAIK only reason of any issues related to use -Wl,--as-needed
is using WRONG list -l<foo> parameters (lack of some -l<foo>) and this
needs to be not treated by apply some workaround but but by apply
necessary fixes in -l<foo> linking parameters.
I've been personally involved in fixing many such problems in more
than last decade and I was one of the many similar people fixing
similar issues across many packages.
If it is still something left it will be IMO not more than few
packages in all Fedora packages.
In some packages using --as-needed will cause reduction of SONAME
dependencies by more than half!!!

In other words -Wl,--as-needed should be used everywhere WITHOUT exceptions.
Easiest way apply this globally in Fedora is add --as-needed in
/usr/lib/rpm/redhat/redhat-hardened-ld spec file by apply patch:

--- /usr/lib/rpm/redhat/redhat-hardened-ld.orig
+++ /usr/lib/rpm/redhat/redhat-hardened-ld
@@ -2,4 +2,4 @@
+ %{!static:%{!shared:%{!r:-pie}}}

*link:
-+ -z now
++ -z now --as-needed

BTW: many packages are fiddling with LDFLAGS adding redundantly
-Wl,-z,now. All those modifications can be now dropped.
As well all packages which already are injecting -Wl,--as-needed can be cleaned.

[tkloczko@domek SPECS.fedora]$ grep LDFLAGS *| grep -c -- --as-needed
150
[tkloczko@domek SPECS.fedora]$ grep LDFLAGS *| grep -c -- -z,now
106

On top of this IMO it would be good to add in binutils ld modification
which will be reporting all dropped during linking -l<foo> as a
warning
Such warnings could be used on keep clean build frameworks.

kloczek

Re: RFC: -Wl,--as-needed by default

By Dridi Boukelmoune at 11/27/2017 - 03:21

On Mon, Nov 13, 2017 at 11:01 PM, Tomasz Kłoczko
<kloczko. ... at gmail dot com> wrote:
Off the top of my head I can see glibc leading to "over" linking: if
upstream is not GNU/Linux-only it may need -lm, -lrt, -lsocket, -lnls
etc for other platforms. For autotools-based projects depending on how
you do the functions detection it may or may not add the -l* flags to
$LIBS if provided by the libc.

Dridi

Re: RFC: -Wl,--as-needed by default

By Jakub Jelinek at 11/27/2017 - 03:46

On Mon, Nov 27, 2017 at 09:21:16AM +0100, Dridi Boukelmoune wrote:
There are several problems with forceful --as-needed:
1) forcing it everywhere is a workaround to broken tools that add -l*
options just in case (like auto*, libtool, pkg-config)
2) until recently, there was no way to enable the option for just one
library and restore it afterwards, so many spots, including, but not limited
to, gcc driver itself, do --as-needed -lgcc_s --no-as-needed, which if
somebody forces --as-needed at the start of the linker command line has the
effect that everything up to the -lgcc_s on the command line behaves
--as-needed, and everything after it doesn't. Since 3 years ago ld
offers the --push-state/--pop-state options, so one can use
--push-state --as-needed -lgcc_s --pop-state instead and that at the end
restores the previous --as-needed behavior. So, before turning it by
default, you'd need to check everywhere where --no-as-needed is used and
see if it is something where --push-state/--pop-state should be used.
3) forcing --as-needed breaks some legitimate uses where some work is done
e.g. in the constructors of some library, but there is no need to use any
symbols, or if you know some library needs to be initialized early but will
be used only from dlopened libraries (e.g. AddressSanitizer and
ThreadSanitizer can be examples of this), or the library extends the binary
and provides necessary symbols for a plugin that will be dlopened.

Jakub

Re: RFC: -Wl,--as-needed by default

By Adam Jackson at 11/27/2017 - 16:30

pkg-config isn't broken here. Individual pc files might be.

It'd be pleasant if ld could warn you about excess linkage.

libtool will throw away your attempts to use --as-needed and --push-
state anyway, because libtool is trash.

- ajax

Re: RFC: -Wl,--as-needed by default

By =?UTF-8?Q?Tomas... at 11/13/2017 - 17:46

On 13 November 2017 at 22:01, Tomasz Kłoczko <kloczko. ... at gmail dot com> wrote:
[..]
Just created pull request with above change:
<a href="https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#request_diff" title="https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#request_diff">https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#requ...</a>

kloczek

Re: RFC: -Wl,--as-needed by default

By Phil K at 11/13/2017 - 21:54

One concern is that -Wl,--as-needed requires greater accuracy with the ordering of objects and 
libraries as you link.   Also, if a package uses a library indirectly, i.e. A uses C via B: A -> B -> C,--as-needed will peel away C and break A unless A explicitly mentions its need for C.  Of course that should be the case already, but you're going to see a number of weaknesses in the variouspackages revealed by adding --as-needed.

On Monday, November 13, 2017 5:48 PM, Tomasz Kłoczko <kloczko. ... at gmail dot com> wrote:

On 13 November 2017 at 22:01, Tomasz Kłoczko <kloczko. ... at gmail dot com> wrote:
[..]
Just created pull request with above change:
<a href="https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#request_diff" title="https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#request_diff">https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3#requ...</a>

kloczek

Re: RFC: -Wl,--as-needed by default

By Florian Weimer at 11/14/2017 - 02:07

On 11/14/2017 03:54 AM, Philip Kovacs wrote:
I think ld no longer links against symbols in indirect dependencies.

#include <openssl/evp.h>
int
main()
{
return (int) &EVP_rc4;
}

/usr/bin/ld: /tmp/ccV4cmYY.o: undefined reference to symbol
'EVP_rc4@@OPENSSL_1_1_0'
//usr/lib64/libcrypto.so.1.1: error adding symbols: DSO missing from
command line
collect2: error: ld returned 1 exit status

Thanks,
Florian

Re: RFC: -Wl,--as-needed by default

By King InuYasha at 11/14/2017 - 06:32

On Tue, Nov 14, 2017 at 2:07 AM, Florian Weimer < ... at redhat dot com> wrote:
I've only seen this with OpenSSL, so I think it's specific to that library.

Re: RFC: -Wl,--as-needed by default

By Panu Matilainen at 11/14/2017 - 06:45

On 11/14/2017 01:32 PM, Neal Gompa wrote:
Implicit linking is not openssl specific at all, this is fairly old news
by now as this change occurred in Fedora 13:

<a href="https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking" title="https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking">https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking</a>
<a href="https://fedoraproject.org/wiki/UnderstandingDSOLinkChange" title="https://fedoraproject.org/wiki/UnderstandingDSOLinkChange">https://fedoraproject.org/wiki/UnderstandingDSOLinkChange</a>

Re: RFC: -Wl,--as-needed by default

By King InuYasha at 11/14/2017 - 06:50

On Tue, Nov 14, 2017 at 6:45 AM, Panu Matilainen < ... at redhat dot com> wrote:
Then something isn't working correctly, because then libcomps builds
should be failing in Fedora. It doesn't. It fails in *every other
Linux distribution* that I've built it for (Mageia, openSUSE,
OpenMandriva, Solus, and others...) unless I patch it to deliberately
link with zlib (it fails with similar issues noted with OpenSSL).

Re: RFC: -Wl,--as-needed by default

By Florian Weimer at 11/14/2017 - 07:08

On 11/14/2017 12:50 PM, Neal Gompa wrote:
That's likely something else entirely. I don't even see anything in
Fedora's libcomps which would need zlib.

Thanks,
Florian

Re: RFC: -Wl,--as-needed by default

By King InuYasha at 11/14/2017 - 07:14

On Tue, Nov 14, 2017 at 7:08 AM, Florian Weimer < ... at redhat dot com> wrote:
It's something related to libxml2, if I remember correctly. I couldn't
work out exactly what was going wrong, but linking zlib directly fixes
the link errors.

Re: RFC: -Wl,--as-needed by default

By Igor Gnatenko at 11/14/2017 - 07:39

Hash: SHA256

On Tue, 2017-11-14 at 07:14 -0500, Neal Gompa wrote:
Then it is problem of libxml2 or other libraries, it has nothing to do with
libcomps. Itself it doesn't use anything zlib-related.

Re: RFC: -Wl,--as-needed by default

By Daniel P. Berrange at 11/13/2017 - 06:06

On Mon, Nov 13, 2017 at 11:52:14AM +0100, Björn 'besser82' Esser wrote:
What % of our distro involves fortran though ? Could this be as simple as
enabling it by default, but having an easy way via an RPM macro to opt-out
of it in the handleful of packages that matter wrt fortran.

Regards,
Daniel

Re: RFC: -Wl,--as-needed by default

By Michael Catanzaro at 11/13/2017 - 07:21

On Mon, Nov 13, 2017 at 5:06 AM, Daniel P. Berrange
< ... at redhat dot com> wrote:
If Debian/Ubuntu/openSUSE (didn't know about openSUSE) can all handle
it, I'm sure Fedora can find a way.

I'll just add: it's pretty annoying that, right now, when I touch
linker flags, there's no easy way to know if my application will link
or not on other distros, because Fedora is more permissive.

Michael

Re: RFC: -Wl,--as-needed by default

By King InuYasha at 11/13/2017 - 07:27

On Mon, Nov 13, 2017 at 7:21 AM, Michael Catanzaro
<mike. ... at gmail dot com> wrote:
In Mageia, we have probably some of the strictest linker flags of all
the distributions (that I know of). Many applications that build or
link fine in Fedora need fixing in Mageia because we don't permit
underlinking[1] or overlinking[2], and we apply several fixers in
configure and brp scripts. We've also designed our flags setup so that
you can set a macro to override as needed, so for things like FORTRAN,
you can change this easily.

I'd like to see these make their way into Fedora.

[1]: <a href="https://wiki.mageia.org/en/Underlinking_issues_in_packaging" title="https://wiki.mageia.org/en/Underlinking_issues_in_packaging">https://wiki.mageia.org/en/Underlinking_issues_in_packaging</a>
[2]: <a href="https://wiki.mageia.org/en/Overlinking_issues_in_packaging" title="https://wiki.mageia.org/en/Overlinking_issues_in_packaging">https://wiki.mageia.org/en/Overlinking_issues_in_packaging</a>