DevHeads.net

undefined symbol: shm_open (ppc64le and aarch64)

One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:

Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':

/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed

Here's the linker invocation:

g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro
-Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-o BiocParallel.so ipcmutex.o -L/usr/lib64/R/lib -lR

Now, google says this might be due to a lack of -lrt, but what's not
clear to me is why it only fails on these two architectures (I've
confirmed that none of the others have -lrt either).

Full build logs are here:

<a href="https://koji.fedoraproject.org/koji/taskinfo?taskID=32581444" title="https://koji.fedoraproject.org/koji/taskinfo?taskID=32581444">https://koji.fedoraproject.org/koji/taskinfo?taskID=32581444</a>

Thanks in advance,

~tom

Comments

Re: undefined symbol: shm_open (ppc64le and aarch64)

By Florian Weimer at 02/07/2019 - 04:29

* Tom Callaway:

Is this an error message from the R interpreter?

On x86-64, the R interpreter already loads librt:

7f0f2635a000-7f0f2635c000 r--p 00000000 fd:01 402963247 /usr/lib64/librt-2.29.so
7f0f2635c000-7f0f26360000 r-xp 00002000 fd:01 402963247 /usr/lib64/librt-2.29.so
7f0f26360000-7f0f26362000 r--p 00006000 fd:01 402963247 /usr/lib64/librt-2.29.so
7f0f26362000-7f0f26363000 r--p 00007000 fd:01 402963247 /usr/lib64/librt-2.29.so
7f0f26363000-7f0f26364000 rw-p 00008000 fd:01 402963247 /usr/lib64/librt-2.29.so

That's because libR loads it:

$ readelf -d /usr/lib64/R/lib/libR.so

0x0000000000000001 (NEEDED) Shared library: [librt.so.1]

The way dynamic linking works is that it searches for symbols in the
caller's namespace, and not just what's referenced by DT_NEEDED.

In general, underlinking can be very problematic because it causes the
base symbol version to be used at run time, even if your program
actually requires a newer symbol version (for example, due to the
headers used at build time). Here it does not matter (today) because we
only have one symbol version for shm_open. But you do get interesting
effects with timer_create.

Thanks,
Florian

Re: undefined symbol: shm_open (ppc64le and aarch64)

By J. Scheurich at 02/07/2019 - 00:06

| One of my packages failed the mass rebuild, but only on ppc64le and
On ububtu (ARM) ...

$ for i in /lib/aarch64-linux-gnu/*.so ; do echo $i;nm -D $i | grep
shm_open; done | more
...
/lib/aarch64-linux-gnu/librt-2.27.so
0000000000003f28 T shm_open

It looks like you have to link -lrt

so long
MUFTI

Re: undefined symbol: shm_open (ppc64le and aarch64)

By TASAKA Mamoru at 02/06/2019 - 21:28

Tom Callaway wrote on 2019/02/07 3:39:
I don't know well about R, however that is probably because R-core (-3.5.3-4.fc30) package already
requires librt.so on x86_64, i686, etc, while on aarch64 and ppc64le, it does not, which probably indicates
that on x86_64, i686, etc R binary is already linked with librt.so , while on aarch64 and ppc64le
it is not.

So probably on x86_64, i686 when dlopen()ing BiocParallel.so symbol for shm_open is already resolved
(because R is linked against librt.so) while on aarch64 and ppc64le it is not.

I guess this is because there is some configuration difference on R.spec on aarch64 and
ppc64le.

Regards,
Mamoru

Re: undefined symbol: shm_open (ppc64le and aarch64)

By Tom "spot" Callaway at 02/07/2019 - 11:49

So, R links with rt if this configure check succeeds:

AC_CHECK_LIB(rt, clock_gettime)

Sure enough, on aarch64 and ppc64le, there is no clock_gettime in
librt.so.1. I'm not sure _why_, but there is probably a good reason.

I now understand why these builds are failing and have several paths to
workaround it.

Thanks!

~tom

Re: undefined symbol: shm_open (ppc64le and aarch64)

By Florian Weimer at 02/07/2019 - 12:34

* Tom Callaway:

Hah!

We moved clock_gettime to libc.so.6 in glibc 2.17, so that it is
available without -lrt as well. But we forgot to turn the legacy
definitions into compatibility symbols, so they are still being picked
up with -lrt. This should be easy enough to fix.

Thanks,
Florian

Re: undefined symbol: shm_open (ppc64le and aarch64)

By Dominik 'Rathan... at 02/07/2019 - 12:41

On Thursday, 07 February 2019 at 17:34, Florian Weimer wrote:
clock_getres(2) confirms:
Link with -lrt (only for glibc versions before 2.17).

Regards,
Dominik

Re: undefined symbol: shm_open (ppc64le and aarch64)

By Dominik 'Rathan... at 02/06/2019 - 17:41

On Wednesday, 06 February 2019 at 19:39, Tom Callaway wrote:
shm_open(3) concurs:
[...]
Link with -lrt.
[...]

So it's rather curious why it's succeeding on all the other arches.

Regards,
Dominik