Fedora, Packaging, Java, and Shrooms


I'm thinking of switching to Fedora 30 Silverblue(once it comes out of beta
anyway) from Arch linux. One of the requirements is to be able to install,
compile from source and easily switch between JDK builds. However, Fedora
fails to meet these requirements so badly I'm fairly certain whoever
packaged and approved the various Java RPMs was on shrooms(partial offense,
sorry but this is nuts).

Confused? Never installed Java in Fedora before? Lets go down the rabbit
hole together!

Firstly, the java version installed by doing:

rpm-ostree install java

gets you Java 8. I understand that Red Hat is providing support until
sometime 2023 however I feel it to be much more appropriate that this
either gives the latest LTS(11 currently) or the newest JDK(12 currently).
Even if still technically support, installing a JRE that old isn't likey to
be advised. Even in Ubuntu 18.10 you get Java 11.

OK, so just specify the specific versions that you actually need and
everything will work as it should. No big deal, right? Nope.

alternatives(see: <a href="" title=""></a>), which is supposed
to allow you to switch between Java versions, flat out doesn't work. You
tell it to list all alternatives like so:

alternatives --display java

and the command executes without printing anything. Odd. Let's just check
what's in /usr/lib/jvm for a sanity check:

ls /usr/lib/jvm



What. I only installed Java 8, Java 11, and Java 12. Installing Java on
either Ubuntu and Arch doesn't duplicate any JRE/JDK like this.

OK, so lets open it in nautilus:

nautilus /usr/lib/jvm

Those aren't duplicates, all but 3 folders are system links? What? This
should be the one and only system location for JRE/JDK(s). What's going on
here? Where do they go?

They go to /etc/alternatives. I guess this is supposed to be how
alternatives finds alternative JRE/JDK installs. Ubuntu doesn't have to do
this for update-alternatives nor does Arch's archlinux-java script but...
OK. This is insanely complicated for no real good reason.

But wait, we aren't done yet because what's being linked to from
/usr/lib/jvm isn't a file, it's... another system link. Back to the 3 non
system links in /usr/lib/jvm which have horrendously long and complex
folder names. Is calling them
java-<version>-<jre/jdk>-<oracle/zulu/openjdk> not enough?

To top this "what" fest off, the JRE/JDK folders in /etc/alternatives
aren't even named properly. That is to say, "jre" is attached to the
front even if what's being linked is a JDK. Yes, a JDK contains a JRE but
it's still horribly confusing for no good reason. Like, imagining if
alternatives did work, does it list duplicate entries for each JRE/JDK? For


which(again) system link to the same JDK install.

What shroom induced insanity is this? Why does alternatives not work?


Re: Fedora, Packaging, Java, and Shrooms

By Colin Walters at 04/12/2019 - 13:16

On Thu, Apr 11, 2019, at 7:41 AM, Ty Young wrote:
Bigger picture, as another person commented the overall design push of Silverblue (and FCOS) is to use containers - I personally do all of my development inside `toolbox` where I install things with dnf (among other tools) and try to keep my host small. (Though I have package layered libvirt, tilix and sway for example).

To be fair this is really rpm-ostree conflicting with alternatives:
<a href="" title=""></a>

Also, more generally anything that tried to "snapshot" system state
would get into a mess with things in /usr referencing parts of /var.
So fixing alternatives to keep state in /etc or /usr solely without
involving /var would benefit everything. For example I bet
alternatives is broken with <a href="" title=""></a>
too (though it might be differently broken).

rpm-ostree breaking alternatives is one of a few examples of why we basically need two systems for a while.

(Though it's interesting of course since once you invest a lot in something like 'toolbox' you start thinking about snapshots of that, i.e. a clean separation of the container image and your state, and that would drive the container image more towards the "ostree style" filesystem layout, i.e. /var/home in your containers too, but that'd bring the alternatives problem back inside the container)

Re: Fedora, Packaging, Java, and Shrooms

By Ty Young at 04/12/2019 - 17:25

On 4/12/19 12:16 PM, Colin Walters wrote:

Which is fine and all. I'm sure toolbox covers some developers needs
nicely. My point is that not everything can be done in containers and
having add-on system software that can be switched between
on-the-fly(for both users or developers) isn't some completely unheard
of idea. I don't think this is a case of Fedora Silverblue not being
capable of being used for mainstream use but rather a pain point that
needs to be hammered out.

Maybe only capture symbolic links in the ostree image and redirect to a
shared directory?

Re: Fedora, Packaging, Java, and Shrooms

By King InuYasha at 04/12/2019 - 14:20

On Fri, Apr 12, 2019 at 1:17 PM Colin Walters < ... at verbum dot org> wrote:
Containers are still pretty bad for complex graphical software
development, though I imagine not many people do that with Java these

openSUSE's transactional-update system handles alternatives fine
because it handles /etc merging and such. So even user settings in
/etc are preserved properly.

Only rpm-ostree has major issues with chunks of the ecosystem because
it doesn't implement some aspects that librpm has, and the filesystem
lockdown is more rigid.

Re: Fedora, Packaging, Java, and Shrooms

By Nicolas Mailhot at 04/12/2019 - 13:39

Le vendredi 12 avril 2019 à 13:16 -0400, Colin Walters a écrit :
Well everyone’s design push nowadays is to use containers. The problem
is not choosing to use containers, the problem is to manage all the
container interfaces and overlays, so they actually give the expected

Java alternatives are a perfect example, how a much simpler tech like
symlinks, can grow into a complex monster, once you've added up all the
desired policy constrains. Simple direct setups are easy. Declining
them to the level of complexity of modern IT without losing ground is

Re: Fedora, Packaging, Java, and Shrooms

By Kevin Kofler at 04/12/2019 - 00:30

Ty Young wrote:
This is probably due to limitations in Silverblue. The Fedora Java packaging
was designed for normal Fedora, at a time where Silverblue did not even
exist as a concept.

No matter how aggressively Silverblue is being marketed at the moment, it is
still pretty experimental and inconvenient to use.

You will probably have a much better experience using a standard, non-atomic
Fedora installation.

The idea is that alternatives should only write to /etc, never to /usr.
Writing to /usr is a bad idea (and incidentally, will likely never work in
Silverblue). So the alternatives in /usr must be symlinks to
/etc/alternatives, and /etc/alternatives contains the user-configurable
symlinks to the real binaries.

The naming allows installing multiple versions of the same major version,
and in particular also 32-bit vs. 64-bit versions.

No. If you just install the java-*-openjdk package, you get only the JRE
part of OpenJDK. The JDK part is in java-*-openjdk-devel.

The way this works is that the -devel package adds:
* files to the versioned directory under /usr/lib/jvm AND
* additional alternatives settings for jdk, javac, etc. (and normally, javac
inherits the setting for jdk which itself inherits the jre setting, but
you can force a mixed version mess if you really want that).

jre_11 is any Java 11 implementation. java-11-openjdk is any minor version
of OpenJDK 11, so more specific. Different applications may want to have
differently stringent requirements. The alternatives are set up to allow
them to specify exactly what they want.

It's not insane, it's complex because there are so many people with so many
different requirements for Java stuff (and it was designed at a time where
the situation was even worse, with Sun Java, IBM Java, and the Free as in
Speech but limited-functionality GCJ stack, which was the best we had in
Free Software before OpenJDK and IcedTea).

Probably a bug or missing feature in Silverblue.

Kevin Kofler

Re: Fedora, Packaging, Java, and Shrooms

By Ty Young at 04/12/2019 - 02:21

On 4/11/19 11:30 PM, Kevin Kofler wrote:

My reason for wanting to switch to Fedora Silverblue is for the safe
system upgrades and easy system recovery. Without either of those, I'd
just stick with Arch. The way software is (sometimes incorrectly,
nvidia-smi in CUDA for example) split into multiple packages in Fedora
isn't a headache worth dealing with for me without something to make it
more appealing... and Fedora Silverblue is very appealing.

Which it does but no alternatives show up even when downloading from
Fedora's repos. Is there no post installation scripts that properly
registers everything? If not, then how are there symbolic links in
/etc/alternatives? What are they even for?

Realistically speaking, is there ever going to be multiple versions from
the same vendor distributed by Fedora? Other Linux distros just do:


for each version and it seems to work fine. If someone compiles from
source or downloads from somewhere they are probably going to have a
different vendor branding or some other custom format like:


or something.

Which isn't clear by the package name. It's ambiguous. Given the shift
from distributing jar programs to modular app bundles one might
reasonable expect any java implementation after Java 9 to include a full
JDK by default which includes a full JRE. It isn't like anything is
going to break by doing this because, again, the JDK is a JRE. Any non
modeler programs will still work.

Is that really a good idea? There may be use cases where one might need
different JRE and JDK of the same version. Java 8 jPackager(deprecated
in newer Java versions) might for example allow a standalone JRE,
reducing some dead weight as opposed to bundling with the full JDK.

I have never personally run into such a situation. As long as
/usr/bin/java links to a valid Java binary it doesn't really seem to
matter. Given that no other Linux distro does this AFAIK, this can't be
common practice.

Fair enough.Given that only Java 8 and newer is available via Fedora's
repos and things have calmed down a bit, is the complexity still worth
it though?

Re: Fedora, Packaging, Java, and Shrooms

By Kevin Kofler at 04/12/2019 - 11:40

Ty Young wrote:
There are such post-installation scripts, but it looks like they are not
working in Silverblue. This is something that needs to be addressed in

32-bit vs. 64-bit is one example (which is why the directory names contain
the CPU architecture, e.g., x86_64).

This is how almost all software in Fedora is packaged: the main package
contains the files required at runtime, the -devel package the files
required to compile software against the package.

And the ambiguous naming is really upstream's fault, because there is no
such thing as "OpenJRE".

Modular Java programs do not need a full JDK either, a JRE is sufficient.
The full JDK is only needed to compile software from source.

This is supported by the alternatives setup. (It is the reason why there are
separate jre and jdk alternatives, though the latter tracks the former by
default, but you complained about exactly that complexity…) But what such
standalone JRE should Fedora ship? Again, there is no OpenJRE. We can only
ship the JRE subset of OpenJDK as the default JRE.

Kevin Kofler

Re: Fedora, Packaging, Java, and Shrooms

By Ty Young at 04/12/2019 - 18:24

On 4/12/19 10:40 AM, Kevin Kofler wrote:

According to Fedora Rawhide doesn't even have a 32-bit JRE/JDK
so i'm not sure why the designation is required. 32-bit has been on the
way out for awhile now. If someone wants to make a 32-bit version they
don't need to follow a distros naming convention.

Fedora also packages nvidia-smi with CUDA libraries and that's wrong
too. On both Windows and in Ubuntu nvidia-smi comes with the driver. The
driver control panel is also included on both as well. nvidia-xconfig
comes with the driver in Ubuntu.

Just because it's the way it has been done doesn't mean it's the right
way. It's just the easier pill to swallow.

Because the JRE is derived from the JDK but there are use cases where
just having a JRE standalone is of benefit. The JRE however is being
killed off. Oracle no longer even distributes a JRE anymore with new
Java versions.

"OpenJDK" is more of a source branding than an indication that it's a
JRE/JDK... but yes it is confusing.

Depending on the program, maybe. If a modular program requires a JDK
module then the JDK is going to need to be used. This isn't immediately
obvious until you run the program and see if it spits out a module not
found error. Granted, no one should ever distribute a modular
application via jar and expect a user to launch via the the complex
command line command. A modular application is generally used in
conjunction with jLink to provide a bundle... which requires a JDK to make.

This specific problem(which branched out of the alternatives one) here
isn't with alternatives but with which the JRE and JDK are separated at
the package level. I'm not even sure how as an end user/developer I'd
even know -dlevel exists on Fedora Silverblue as dnf search doesn't
exist and doesn't bring anything up. Is there an alternative
for Silverblue?

and again, the JRE is being axed. Once Java 8 hits EOL the only way to
get a standalone JRE is (maybe?) by compiling it yourself. Oracle
doesn't distribute it anymore.

Re: Fedora, Packaging, Java, and Shrooms

By Kevin Kofler at 04/12/2019 - 22:50

Ty Young wrote:
While some packages are not being built for 32-bit x86 at all anymore
because upstream dropped support for it entirely (e.g., the Eclipse stack),
this is not the case for OpenJDK, where upstream still supports compiling it
for 32-bit x86 (they just do not provide Oracle Java binaries for 32-bit x86

This is an issue with RPM Fusion or Negativo's repo or wherever you got the
NVidia driver from. (There are unfortunately multiple repositories providing
the driver with different packaging, due to disagreements on how exactly it
should be packaged. The driver is not included in Fedora itself because it
is not Free Software.)

-devel subpackages are how ALL packages that have both a compile-time and a
runtime part are done in Fedora. Only pure compiler packages (e.g., gcc) and
pure runtime-only packages (e.g., all the leaf applications that do not ship
any libraries) are exempt from the split. This is not just a one-time
decision for a single package.

But then why are you complaining about the JRE packages being called
"openjdk"? You explained yourself why they are named that way!

Applications doing dynamic compilation typically only require the
java.compiler module, which is in the main package, not the javac
executable, which is in the -devel package. There are no modules in the
-devel package.

It requires a JDK to make at compile time. (In Fedora, you need the -jmods
subpackage, which depends on -devel. The jmod files that jlink needs are not
installed by default even in -devel.) The end users do not need a JDK to run
it. The point of the main (non-devel) packages is to provide what end users
will need to run Java applications.

This is not specific to Java packaging though. You will also need -devel
packages for C/C++ libraries (or libraries in most other compiled languages)
if you want to compile applications using them.

You can get a prebuilt standalone JRE from several sources. E.g.,
<a href="" title=""></a> lets you choose between a JDK and a JRE download
if you use the "Other platforms" link. (The front page gives you a JDK

Kevin Kofler

Re: Fedora, Packaging, Java, and Shrooms

By Gerald Henriksen at 04/12/2019 - 20:26

According to Koji, Java is indeed still built in a 32bit version,

<a href="" title=""></a>

Fedora doesn't package nvidia-smi - the binary Nvidia driver is not
allowed in Fedora. If you have issues with how it is packaged then
take it up with the 3rd party providers who have packaged it.

But before you do so I suggest you take a look at the packaging
guidelines to see if there might actually be a valid reason for the
way things are done...

<a href="" title=""></a>

What Oracle does or doesn't do isn't relevant, as Fedora is packaging
OpenJDK from sources, and has done so from the beginning of OpenJDK.

If you spent some time learning Fedora, and how Fedora packages
things, or asked questions, you would learn/know that Fedora separates
out everything into a runtime package and a devel package.

Or alternatively explore and see what packages
are created for each source rpm.

As for Silverblue, as mentioned it's a work in progress (and like any
open source project likely welcoming of assistance if that intersts

Re: Fedora, Packaging, Java, and Shrooms

By Nicolas Mailhot at 04/12/2019 - 03:14

Le vendredi 12 avril 2019 à 01:21 -0500, Ty Young a écrit :
You still have OpenJ9 vs OpenJDK, LTS vs non LTS, completely free
software build vs supported proprietary build (Fedora does not ship the
later but some Fedora derivatives do and there is a wish for them to be
able to do it without butchering the Fedora-inherited filesystem

So yes, it is probably possible for someone motivated to invent a
better java alternative setup, but it will be a lot of work, with a lot
of constrains (one of those, being able to upgrade existing setups to
the new system gracefully)

Re: Fedora, Packaging, Java, and Shrooms

By =?UTF-8?Q?Micha... at 04/11/2019 - 08:31


firstly i recommend to use Fedora toolbox [0] for this kind of things on
Silverblue (it's part of Silverblue already).

Secondly, isn't this what modules are meant for? I'm not sure if there
is one for JDK on Fedora.

Michal Konecny

[0] - <a href="" title=""></a>

On 4/11/19 1:40 PM, Ty Young wrote:

Re: Fedora, Packaging, Java, and Shrooms

By Ty Young at 04/11/2019 - 09:08

Not everything can easily be done in a containerized environment. The game
"Minecraft" for example needs a system JRE in order to run as it is written
in Java and is installed via a jar file(old launcher anyway). While not
ideal, I don't think this is against the Fedora Silverblue containerized
idea. That is to say, Java is a semi dynamic extension of the base system
and NOT an application.

Yes, it is possible to bundle a Java program with a JRE/JDK in a Flatpak
and distribute that. The problem is that it both takes up significant disk
space compared to if you were to just use the system JDK and makes it
difficult to switch between JRE/JDK versions if need be.

Java 9 modules you mean? Java 9 modules are a way to create a JDK bundle
with your app with significantly less fat as only what is needed is
included. For distribution of a program written in Java, sure that works as
long as the program is modeler... but not all are.

That's the thing: I need to switch between a "server" build of the JDK(uses
MORE memory, what every distro ships) and a "client" build(uses less
memory, needs to be compiled from source) as well as any other custom
builds I might compile or install from time to time. I'd imagine this could
be done via the build script but that just complicates things and is
specific to the computer the software is being compiled on. It's easier to
just quit the IDE(netbeans in my case), switch versions, and start it up

The only problem is that the current running JRE/JDK is being (potentially)
pulled from under any running Java application that is utilizing the system
JRE/JDK and a newer or older version is replacing it. This, however, is a
problem in Flatpak applications as well and there doesn't seem to be any
solution other than to somehow completely load the application and all of
it's library dependencies(Java, Nvidia libs, etc) in memory. Java is not
system critical just as the Flatpak runtimes are not system critical.

Re: Fedora, Packaging, Java, and Shrooms

By Gerald Henriksen at 04/12/2019 - 20:29

No, Fedora Modules, an alternative to rpms I think.

Re: Fedora, Packaging, Java, and Shrooms

By Nicolas Mailhot at 04/11/2019 - 08:05

Le 2019-04-11 13:40, Ty Young a écrit :


Java is not "just" a system directory, it's a set of coordinated
commands, some of those shared between the jre and the jdk, with the set
varying slightly between Java versions, JDK provider, etc.

Writing good alternative rules given all those can be quite challenging
and brain-damaging (alternative rules must upgrade gracefully to the
next package versions)

I'm sure the maintainers of the Java packages would be delighted to have
someone to review all the alternative rules and fix up all the warts
that crept in since Java 1.3 times (back in my day, once you got
alternatives to work for a new jdk, you tried very hard to forget the
experience). Certainly, all the standardizing on openjdk after SUN sank
should have simplified the mess a little.

Re: Fedora, Packaging, Java, and Shrooms

By Ty Young at 04/11/2019 - 08:30

What you are (primarily) describing is the path export variable to the java
binary which is located in a JRE/JDK install.

In Ubuntu and Arch Linux this is done via a "default" system link in
/usr/lib/jvm that links to the configured system wide JRE/JDK located in
the same directory(/usr/lib/jvm). Surely both you and the Fedora Java
maintainers know this or have at least thought of it?

Again, just system link to the correct folder like every other distro.
alternatives doesn't need to concern itself with specific content. It could
be empty or copy/pasted as far as it *should* care. As long as it follows
the java-* format you can get a list of all JRE/JDK installs regardless of
the name and have "default" system link point to it.

This has nothing to do with how the folders are named. See:

<a href="" title=""></a>

under "managing java".