Home

LD Changes To Implicit DSO Linking Update

This is just an update to let maintainers know that the changes to
LD outlined here :

https://fedoraproject.org/wiki/Features/ChangeInImplicitDSOLinking

will be in fedora rawhide pretty soon.

The details behind what this feature will do, along with how to
get failing packages to build can be found here :

https://fedoraproject.org/wiki/UnderstandingDSOLinkChange

Also, packages that have failed to build under these new changes can
be found here :

https://fedoraproject.org/wiki/DSOLinkBugs

Thank-You for your time,

Re: LD Changes To Implicit DSO Linking Update

Hi,

Is it ok to backport changes to F-12 for fixed packages in F-13 for
this DSO feature?

Regards,
Parag.

Re: LD Changes To Implicit DSO Linking Update

It's of course OK to apply those changes to all branches (they won't break
anything for the older ld), but it doesn't make sense to push updates just
for those changes! Please only push updates if you have actual user-visible
changes.

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

Hi,

Thanks all. I just want to know if its possible to apply DSO change in
older active branches. No plans to update those packages in older
branches but to get these changes in upstream first.

Parag.

Re: LD Changes To Implicit DSO Linking Update

The changes to a package's linking lines that you make for this issue are
cleaning up sloppy practice to what was always the right thing to be doing.
So it should always be fine to do that in builds for earlier Fedoras too.

Thanks,
Roland

Re: LD Changes To Implicit DSO Linking Update

It is much more important to find something well integrated acceptable
upstream, in my opinion.

Re: LD Changes To Implicit DSO Linking Update - pthread question

Le 08/02/2010 23:37, Roland Grunberg a écrit :

+ mysql++ should be fixed. now
http://koji.fedoraproject.org/koji/taskinfo?taskID=1981564

But I'm asking about -pthread option (which is detect/use for this package).

According to GCC man page
-pthread
Adds support for multithreading with the pthreads library.
This option sets flags for both the preprocessor and linker.

Ok, this options is also note as an only IA-64, RS/6000, PowerPC and
SPARC option.

Shouldn't this option imply -lpthread on x86 ?
(which seems the case for x86_64, as the DSO bug only affects i686).

Remi

Re: LD Changes To Implicit DSO Linking Update - pthread question

-pthread is the same as -D_REENTRANT at the beginning (which is useless)
and -lpthread at the end. I don't recommend using it at all. Just use
-lpthread instead (at the end of the link line, like all other libraries).

Thanks,
Roland

Re: LD Changes To Implicit DSO Linking Update

Can we get this page updated regularly? It's fairly trivial for a bored
provenpackager to fix these up, but it'd be nice to not waste time on
already-fixed packages.

- ajax

Re: LD Changes To Implicit DSO Linking Update

Yes -- Roland G. and I will do as many rebuilds as we can and update the
list accordingly. We'll also keep an eye on the list so if you've finished
fixing a package we can remove it from the list as well.

Thank you,
-Charley

Re: LD Changes To Implicit DSO Linking Update

Hello Roland,

[...]

I have patched and re-built the ghex package, so it probably won't
belong to the list of failing packages anymore.

See http://koji.fedoraproject.org/koji/taskinfo?taskID=1971701

Cheers.

Dodji

Re: LD Changes To Implicit DSO Linking Update

You can remove avant-window-navigator from the failed list as it builds
OK since updating the version

Here's the successful build log from today

http://koji.fedoraproject.org/koji/taskinfo?taskID=1971277

http://koji.fedoraproject.org/koji/getfile?taskID=1971277&name=build.log

Could you add qbittorrent to the list of failed packages.

http://koji.fedoraproject.org/koji/buildinfo?buildID=155100

Re: LD Changes To Implicit DSO Linking Update

Will there we a switch to give me the old behavior? I might want this for
my own legacy code.

Re: LD Changes To Implicit DSO Linking Update

Not forever. You really should fix your makefiles. It's just cleaning
up usage that was sloppy originally.

Re: LD Changes To Implicit DSO Linking Update

I have not tried it, but apparently --as-needed (or from gcc use
-Wl,--as-needed):

--as-needed
--no-as-needed
This option affects ELF DT_NEEDED tags for dynamic libraries
mentioned on the command line after the --as-needed option.
Normally the linker will add a DT_NEEDED tag for each dynamic
library mentioned on the command line, regardless of whether the
library is actually needed or not. --as-needed causes a DT_NEEDED
tag to only be emitted for a library that satisfies an undefined
symbol reference from a regular object file or, if the library is
not found in the DT_NEEDED lists of other libraries linked up to
that point, an undefined symbol reference from another dynamic
library. --no-as-needed restores the default behaviour.

Rich.

Re: LD Changes To Implicit DSO Linking Update

No, you mean --add-needed.

- ajax

Re: LD Changes To Implicit DSO Linking Update

Fair enough. BTW the man page seems to indicate that --add-needed is
deprecated (replaced by --copy-dt-needed-entries).

Rich.

Re: LD Changes To Implicit DSO Linking Update

Worth clarifying here: Rawhide or (and?) F13?

Re: LD Changes To Implicit DSO Linking Update

They are still the same thing, so both. gcc-4.4.3-5.fc13 is there right now.

Thanks,
Roland

Re: LD Changes To Implicit DSO Linking Update

Hi,

I will say this change is introduced at wrong time, considering we
have only one week left for F13 Alpha freeze. This change should have
been done at time of F12 release where we were already having devel
branches working as F13.
From packager's point of view I can't see anything written at
https://fedoraproject.org/wiki/UnderstandingDSOLinkChange. That page
only describes about DSO Linking change but what am I supposed to do
when one of my package fails to build? Should I ask upstream to
hardcode required DSO names in Makefile or we need to modify CFLAGS in
%build section?

Regards,
Parag.

Re: LD Changes To Implicit DSO Linking Update

2010/2/9 Parag N(पराग़) <<...> at gmail dot com>:

Most of the time upstream (myself included) just forgets to add a
library at the end of the LDADD line, so all I had to do was send a
trivial patch upstream to add "-lm" or something.

See here for an example:
http://bugzilla-attachments.gnome.org/attachment.cgi?id=152993

Richard.

Re: LD Changes To Implicit DSO Linking Update

They will only be so for a fairly short time, and you gave no specific
time frame for landing the change (only 'pretty soon'), so it was not
entirely clear. Thanks.

Re: LD Changes To Implicit DSO Linking Update

Sorry, we meant to be clear that it was "now" as of the posting.

Re: LD Changes To Implicit DSO Linking Update

281 packages? Wov! (That's after the most often occurring problems have
been already resolved, am I right?)

+ let's say 300 "regular" FTBFS bugs -- the F13 mass rebuild will be
really great...

You sure it wouldn't be better to wait for F14 with this? With "no
frozen rawhide" you can submit the changes into rawhide quite soon and
everybody has time enough to work on it.

Regards,
Milos

Re: LD Changes To Implicit DSO Linking Update

To me, this shows that this "feature" is completely broken and should not
make it into Fedora, ever.

+1. Why does this get rushed into F13 the day of the feature freeze?

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

I disagree with that. Previous changes to the build environment - even
upstream GCC changes - have broken way more packages (every GCC .x
release tends to break a lot of things temporarily). Just causing builds
to break doesn't mean the change shouldn't be made...

...but I agree with this, it seems like very bad timing.

Re: LD Changes To Implicit DSO Linking Update

And that's something which really irks me about GCC upstream, they don't
seem to understand what "backwards compatibility" means. That doesn't mean
it's a good idea to break code like this.

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

You are probably looking for bug compatibility, and that isn't something
GCC guarantees, definitely not between major versions.

The C/C++ standards (and standards governing the extensions to the
languages) is what matters, if you write standard conforming code, GCC won't
(intentionally) start rejecting it. But if you have code that happens to
compile because of some GCC bug, eventhough it was invalid, or some C/C++
defect report clarifies some corner case, you need to be prepared to fix up
your code.

Jakub

Re: LD Changes To Implicit DSO Linking Update

And that's one half of what I'm complaining about.

What about those documented extensions that got deprecated and later
removed? That's the second half of what I'm complaining about: even things
which are NOT bugs but documented extensions get deprecated and soon later
removed.

IMHO a compiler should accept code whenever there's a sane interpretation of
it, no matter whether it conforms to some standard or not (in fact, this
used to be a GCC design principle, but sadly no longer is these days), and
code which has been compiling for years definitely has a sane
interpretation.

(If you ever look at TIGCC, you'll also notice that TIGCC's GCC is patched
to re-add things like multi-line string literals, lvalue casts, labels at
the end of compound statements etc. More and more patches I'm applying to
the GCC in TIGCC are to readd things GCC removed. TIGCC also enables -fms-
extensions by default (in C, this allows at least unnamed struct/union
fields). I haven't been updating TIGCC's GCC in a while, but I'm sure that
if I do, there'll be another bunch of such patches to add. And that's only
for C, g++ is much worse.)

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

That sounds to me like you want the GCC team to keep their bugs forever when
those bugs mask bugs in your code, so that you won't have to fix your bugs.
Hopefully you didn't mean something quite so insane.

And what happens the day you need to compile that code with another compiler?
Do you consider vendor lock-in through embrace-and-extend tactics to be a good
thing when a free software project does it?

Björn Persson

Re: LD Changes To Implicit DSO Linking Update

What compiler? A proprietary compiler? The BSD-style-licensed LLVM/clang
which any proprietary vendor can embrace&extend into a non-Free version?
It's not in the interest of the GNU project (which GCC is supposed to be a
part of) to make it easy to compile code with other compilers!

Yes, see above.

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

If that is the case it is extremely short sighted of them.

Re: LD Changes To Implicit DSO Linking Update

That was a statement of fact, not of policy.

The GCC project's policies used to be based on that when it was controlled
more directly by the FSF, but since the EGCS merge, the "follow standards"
camp has basically taken over, which is IMHO quite counterproductive.

A compiler should accept as much code as possible. As long as it UNDERSTANDS
standards-compliant code (real-world standards-compliant code, not some
obscure corner-case designed especially to detect the absence of some
extension which nobody would actually write in a real program), there's no
need to ENFORCE standards compliance. All such enforcement does is:
* make it easy to migrate to a proprietary compiler,
* make it harder to migrate away from a proprietary compiler, as some of
them do accept things like lvalue casts (M$VC supports them, for example),
* gratuitously break backwards compatibility,
* make real programmers (who get errors for obscure standardese reasons they
don't understand when it should be clear what their code is intended to
mean) angry just to make pedantic language lawyers happy.

As a general rule, it's in a compiler's best interest to accept as much code
as possible so it's easy to migrate TO that compiler and hard to migrate
AWAY FROM that compiler. And what's good for GCC is good for Free Software.

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

As a result, you'll be causing dozens of FTBFS bugs just before the feature
freeze. I think this is entirely the wrong time in the release cycle to do
such a change, if it is done at all. (In fact, I still think we're breaking
backwards compatibility for no good reason, and in addition, the combination
of this "feature" and our distrowide packaging policy which dictates
removing .la files causes massive amounts of API incompatibility with
upstream projects. It's sad that FESCo rushed this through within seconds
with only one -1 vote (mine), entirely ignoring my strong objections and
seemingly unaware of the number of affected packages (which surprised even
me, it's even worse than I had expected), and that any attempt to discuss
this further was met with "we've already moved on to the next feature".)

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

I've been fixing upstream projects for weeks to build with
--no-as-needed. The list of projects that fail to build should be much
smaller now, especially for GNOME and Freedesktop stuff.

Richard.

Re: LD Changes To Implicit DSO Linking Update

Well. Even pretty fundamental GNOME stuff like gtk2-devel is still
broken. Look here:

[root@localhost ~]# pkg-config --libs gtk+-2.0
-pthread -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0
-lgdk_pixbuf-2.0 -lpangocairo-1.0 -lcairo -lpango-1.0 -lfreetype
-lfontconfig -lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lrt -lglib-2.0

-lX11 is missing (at least, maybe more). Any gtk app using pkg-config
to figure which libraries are required fails to build now.

I've seen a simliar issue with libpng (-lz missing) and I suspect tons
of other *.pc files are broken too.

cheers,
Gerd

Re: LD Changes To Implicit DSO Linking Update

Gerd Hoffmann <<...> at redhat dot com> writes:

Are all these libraries really required? Putting them into a linker line
causes a huge overlinking adding lots of unneeded direct dependencies to
rpm packages.

That's the problem with the current DSO change :( It causes packagers or
upstream developers to define typical use cases (e.g. when somebody
calls 'mylib_foo(otherlib_struct *)' he calls usually
'otherlib_init(otherlib_struct *)', so that '-lotherlib' is usually
needed).

Such soft decisions are far away from being reliable and to work for
everybody...

I ended up for 'xmlrpc-c' to replace .so symlinks by linker scripts with

| INPUT( AS_NEEDED(+))

commands. The list is created by extracting recursively the
NEEDED libs from .

Not very nice but probably to only way to cope with '--no-add-needed' :(

Enrico

Re: LD Changes To Implicit DSO Linking Update

gtk+-2.0.pc has:

Requires: gdk-${target}-2.0 atk cairo gio-2.0 pangoft2

It seems likely that some, if not all, of the latter four belong in
Requires.private.

Re: LD Changes To Implicit DSO Linking Update

I remember finding also that it added too much unneeded deps, but I had a
look and a random subset of these library were indeed used by gnome
applications and libraries that was installed on my computer at that time.

Re: LD Changes To Implicit DSO Linking Update

No. Why should we put libX11 in the .pc file ?
If you do any direct Xlib calls in your program, you get to put it in
the link line yourself.

Re: LD Changes To Implicit DSO Linking Update

Only if the library you are compiling actually uses any libX11 APIs.
If it only uses APIs from the named libraries, it will link fine.

Jakub

Re: LD Changes To Implicit DSO Linking Update

Sure? For the libpng case I have a build failure which looks like this
isn't the case (bug 565047):


/usr/bin/ld: /usr/lib/gcc/i686-redhat-linux/4.4.3/../../../libpng12.so:
undefined reference to symbol 'crc32'
/usr/bin/ld: note: 'crc32' is defined in DSO /lib/libz.so.1 so try
adding it to the linker command line

Which made me think -lz is needed on the command line even if zlib is
used only indirectly via libpng (and likewise that -lX11 should be there
for gtk2 because gtk2 needs it).

cheers,
Gerd

Re: LD Changes To Implicit DSO Linking Update

That looks suspicious to me. libpng12.so has a DT_NEEDED for libz.so.1,
so its reference should not produce this error. This might be an ld bug.
Are you sure you don't have other references to 'crc32'? It could be
that it is just citing the wrong one for the error. If that's not it,
then AFAICT it's just a bug. Either way, file a binutils bug--but say
whether it's a spurious message or a message misidentifying which reference
has the problem.

Thanks,
Roland

Re: LD Changes To Implicit DSO Linking Update

1. But have those fixes been applied in the Fedora packages? At least
NetworkManager is now failing to build due to this "feature" having been
rushed in just before the freeze, so that's at least one package which
didn't get fixed in Fedora yet.
2. GNOME and freedesktop.org form only a small part of the Fedora package
collection.

I really don't see why this change can't at least wait for F14! Breaking the
build of half of the distro the day of the feature freeze is completely
unacceptable. The change should get reverted for F13 and put into Rawhide
after F13 gets branched.

(I still oppose the feature altogether, but postponing it to F14 would at
least give packagers time to fix their packages.)

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

Hi,

+1. Please revert the changes.

Parag.

Re: LD Changes To Implicit DSO Linking Update

I've filed this proposal for FESCo, it should get considered at the meeting
tonight (20:00 UTC):
https://fedorahosted.org/fesco/ticket/338
"Proposal: postpone ChangeInImplicitDSOLinking feature to F14 and revert its
implementation from pre-branch Rawhide"

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

Sadly, this proposal got rejected. :-( (My own vote was the only one in
favor of my proposal.)

So any complaints about the breakage should be directed at FESCo.

Kevin Kofler

Re: LD Changes To Implicit DSO Linking Update

Kevin, +1 ...

Such big change in day of feature freeze! Maybe we need "layered" freezes -
like do not touch essential components few weeks before freeze and thus let
packagers to do their work while there are finishing it - and they need
something stable under not to fix new issues...

Don't let this as I'm flaming against new features please!

Jaroslav

Re: LD Changes To Implicit DSO Linking Update

Just as a reminder, this change is --no-add-needed, not --no-as-needed.
They have infuriatingly similar names; one of the changes is also to
change the name of --{no-,}add-needed to be more obvious.

--no-as-needed is already the default behaviour, and means "libraries
specified with -lfoo will be emitted into the link output in a DT_NEEDED
entry, regardless of whether any symbols from libfoo are used in the
link output object itself". This may mean your binary or library ends
up with more dependencies than it needs, but is generally harmless.

--no-add-needed is quite different. Your binary a.out uses symbols from
libfoo and libbar. libfoo is linked against libbar. But your link line
only says -lfoo. --add-needed behaviour, the old default, would
implicitly add a "-lbar" as well. --no-add-needed, the new default,
will not, and therefore your link will probably fail.

- ajax

Re: LD Changes To Implicit DSO Linking Update

Not so harmless, IMHO, causing unecessary deps in producing RPM. IIRC
there was some discussion on this in the past. Many of these errors was
caused from using AC_CHECK_LIB autoconf macro and not AC_SEARCH_LIB in
configure.ac

http://www.gnu.org/software/libtool/manual/html_node/Inter_002dlibrary-d...

Nice to know that Fedora will have the same behavior of the AIX libraries in
this area, if I understood correctly the topic under discussion.

Regards
Regards

Re: LD Changes To Implicit DSO Linking Update

This change does not imply AIX linker semantics.

Note that _libraries_ generally do not have a problem building in a
--no-add-needed world. ELF does not require that all references in a
DSO be resolvable at ld time, and this linking change does not change
that. If your library libfoo uses symbols from libbar but does not
itself link against libbar, that's still legal (although probably
impolite).

Executables, however, must be fully resolved at link time. You can ask
for this to be true for libraries as well with the
--no-allow-shlib-undefined option to ld. This plus --no-add-needed is
much closer to the AIX linker behaviour,

Also note that the runtime linker will still do recursive lookups. If
you have a binary that did not link against some needed library, but one
of its dependencies did link against it, the binary will still work.

- ajax