DevHeads.net

Call for participation: Fedora Flatpaks

I'd like to invite Fedora contributors to start creating Flatpaks of
graphical applications in Fedora. We're still working on putting the final
pieces into place to have a complete story from end to end, but it's
definitely close enough to get started.

If you maintain a graphical application, please try creating a Flatpak of
it. Your experience will vary - some applications are quite easy, but if
your application, for example:

* Uses qt5-qtwebengine
* Uses many KDE libraries
* Uses many Perl or Python packages
* Uses texlive

etc, then you may want to wait - we will eventually be creating shared
builds to make bundling these easier. Also, if your application has a
system service, installs a polkit policy, or otherwise is not
self-contained, then it's not a good candidate for a Flatpak.

Or you can pick one of 280+ applications that have been identfied as easy
to Flatpak:

<a href="https://fedoraproject.org/wiki/Flatpak:Easy" title="https://fedoraproject.org/wiki/Flatpak:Easy">https://fedoraproject.org/wiki/Flatpak:Easy</a>

and assist out the application package maintainer by creating a Flatpak of
that.

An introduction, draft tutorial and other documentation can be found at:

<a href="https://fishsoup.net/misc/fedora-docs-flatpak/flatpak/" title="https://fishsoup.net/misc/fedora-docs-flatpak/flatpak/">https://fishsoup.net/misc/fedora-docs-flatpak/flatpak/</a>

(The plan is to integrate this into docs.fedoraproject.org. For now, the
documentation source
is at: <a href="https://github.com/owtaylor/fedora-docs-flatpak" title="https://github.com/owtaylor/fedora-docs-flatpak">https://github.com/owtaylor/fedora-docs-flatpak</a>)

For help, please ask on #fedora-workstation on Freenode, or mail
<a href="mailto: ... at lists dot fedoraproject.org"> ... at lists dot fedoraproject.org</a>.

Owen

Comments

Re: Call for participation: Fedora Flatpaks

By Bastien Nocera at 09/07/2018 - 09:44

Hey Owen,

As discussed earlier in both mailing-lists and face-to-face, I'd like to know
why this is interesting for either upstream or downstream developers.

Who is the target for this feature, why does it make sense for packagers to
package within Fedora (or eventually CentOS, or RHEL), rather than upstream,
whether in Flathub or an independent repository?

I can expand on what I think are the benefits for Fedora, and its downstreams,
but that would require making guesses at roadmaps that I don't have a view
into.

Cheers

Re: Call for participation: Fedora Flatpaks

By Owen Taylor at 09/07/2018 - 11:25

Hi Bastien,

Here are some of the benefits I see of this effort as compared to simply
telling users to consume Flatpaks from Flathub or independent repositories:

* Benefit to Flaptak users on all distributions: more applications are
available more quickly. Some applications will be much easier to create
Flatpaks of this way because of their build dependencies. For lightly
maintained, older applications, building a Flatpak of an RPM within Fedora
is simple and avoids creating another independent place that someone has to
keep an eye on.
* Benefit to Flatpak users on all distributions: this works towards having
a runtime (whether Fedora or RHEL/CentOS based) that has a long lifetime
and strong security update guarantees
* Benefit to Fedora users: they can get Flatpaks and runtimes from a source
they already have trust in.
* Benefit to Fedora users: this is a repository of Flaptaks we can enable
by default (there are ongoing discussions of splitting up Flathub, but
currently it combines both content that Fedora can point users to, and
content that is problematical from a legal or Free Software point of view,
all mixed together.)
* Benefit to Fedora contributors: they can work within the community and
infrastructure they are already familiar with to fill gaps in the set of
available Flatpaks.
* Benefit to Fedora contributors: they can make their packaging work
available across distributions and distribution versions.
* Benefit to upstream: if they already have a good relationship with Fedora
and their application is well maintained there, they can point users on all
distributions to a Fedora Flatpak.
* Benefit to Red Hat: We build infrastructure technology and content that
we can take into the RHEL context and make runtimes and Flatpaks available
to our customers with the type of guarantees that we are already providing
for RPM content.

LIke many things we do in Fedora, the benefit to RHEL is a big reason that
we've been doing this work, and was an influence in some of decisions about
how things were implemented, but I think the work does stand on its own as
useful to the Fedora and Flatpak communities.

Regards,
Owen

Re: Call for participation: Fedora Flatpaks

By Bastien Nocera at 09/14/2018 - 08:30

Sorry it took a couple of days to get back to you.

If the end-goal is shipping Flatpaks, and that those Flatpaks need to be built
on Fedora infrastructure to be distributable, then we have some other options.

For older, mature or not well-maintained, applications, I would think that
having them available through an upstream Flatpak would be more viable, sharing
maintenance with other distributions.

Having a long lifetime and strong security update guarantees is also a goal
of the Flatpak Freedesktop SDK and runtime.

OK.

Seems that this problem is being worked on then.

Sure, it avoids creating more accounts, but the tooling is so different that
I don't think that it's going to help much.

Most likely duplicating upstream work on getting that same

That doesn't seem to require

In summary, I think that building Flatpaks from Fedora binary RPMs in Fedora
infrastructure is not the right path forward:
- long-term supported runtime and SDK is a good thing, no questions, and that
can probably be generated on Fedora infrastructure, as it shares so much
with the Fedora OS itself
- Building Flatpak from binary RPMs is a bad idea. In Flatpak, you'd want the
app dependencies (the ones that aren't part of the runeimt) to be as closely
configured to what the application needs as necessary. That means that one
applications might disable 99% of a library just to have the one plugin it
needs to run, that wouldn't be possible when building from a binary RPM. That
also means that the application is impacted by changes in those libraries, when
the point of Flatpak is that the runtime is API and ABI stable, and all the
rest is under the application's control. Think of every time you saw a mass
change on the fedora-devel list, that's every time your application might break
even though you didn't make a single change to it.

What I'd rather see would be:
- the tools working on source RPMs, rather than binary RPMs, and would generate
flatpak-builder manifests. Those manifests can be then be used in Flathub,
or by the upstream developers in their own repository, or used in the upstream
project's CI to generate nightlies
- Because it has a global view of library usage, and compilation options, Fedora
can make headway on de-duplicating those particular bits for inclusion in the
runtime, or as shared build modules, similar to <a href="https://github.com/flathub/shared-modules" title="https://github.com/flathub/shared-modules">https://github.com/flathub/shared-modules</a>
- the Fedora infrastructure can then use those upstream manifests, with little
modification, to build against the Fedora SDK, on Fedora infrastructure, with
Fedora signing keys, so that the chain of trust is not broken, whether with
end-users or contributors
- those upstream maintained Flatpak manifests make it easier to share testing
with upstream, and reduce the duplication of effort in packaging.
- Fedora flatpaks don't require so much handholding when bug fixes are necessary
(with the current infrastructure, it would require to build an RPM package, test,
then generate the Flatpak from it, and test again)
- As an upstream maintainer, I can drop my application's RPM, and consider the
Fedora packaging directly when upstream, making it easier to include as part of the CI
because it wouldn't require much more packaging work to be done.

Hope this makes sense.

Cheers

Re: Call for participation: Fedora Flatpaks

By Bastien Nocera at 09/14/2018 - 08:37

Urgh, unfinished trains of thought.

...on getting that same application into end-users hands. What do you think would
happen to the opt-in creation of Fedora Flatpaks if you get none of the benefits
of being able to empower upstream with maintaining that package?

That doesn't seem to require the Flatpaks to be build from binary RPMs, or RPMs
at all. The Fedora/RHEL runtime is part of the OS, so no duplication of work,
but packaging application-supporting libraries would be.

Re: Call for participation: Fedora Flatpaks

By =?ISO-8859-1?Q?... at 09/07/2018 - 02:36

Dne 7.9.2018 v 03:45 Owen Taylor napsal(a):
<irony>
Ah, make bundling easier, right. Finally we can bundle!
</irony>

Honestly, I fail to see how this can be promoted as good for Fedora. It
might be good for upstream but not for Fedora.

Vít

Re: Call for participation: Fedora Flatpaks

By Owen Taylor at 09/07/2018 - 04:24

To be clear, bundling here is *not* the same as simply including the
sources for a library into the application. What bundling means here is
including a particular build of a library into the application Flatpak so
that it is tested, deployed, and upgraded as a unit. But the library is
defined independently in Fedora, and is visible to our tooling.
Applications share libraries (beyond those included in the Flatpak runtime)
either:

- At the source level, by including a reference to a src.fedoraproject.org
branch
- A the binary level by depending on a module that includes a Flatpak
rebuild of the library

We haven't yet created such shared modules - we want to get some experience
first at creating Flatpaks to figure out what makes sense. But clearly they
are useful for dependencies that take a long time to rebuild and are shared
by many applications. (On the other hand, of the 3600 dependencies of
graphical applications in Fedora, almost half are a dependency of only
*one* graphical application.)

Owen

Re: Call for participation: Fedora Flatpaks

By Martin Stransky at 09/07/2018 - 03:23

On 9/7/18 9:36 AM, Vít Ondruch wrote:
As far as I remember we try to use upstream packages with minimal local
changes,
put all our changes to upstream...so what's the problem? Don't you
follow upstream with
your package(s)?

ma.

Re: Call for participation: Fedora Flatpaks

By =?ISO-8859-1?Q?... at 09/07/2018 - 03:33

Dne 7.9.2018 v 10:23 Martin Stransky napsal(a):
This is of course one of the things. If I follow upstream with my
packages, I have quite often submit patch fixing compatibility of
upstream with versions of packages available in Fedora, e.g. quite often
fixing compatibility with newer package available in Fedora, moving
upstream forward. With Flatpacking everything, this will stop.

Other thing is to keep packages compatible in Fedora, this will stop as
well, because everything will be bundled in Flatpak. Ultimately in the
future, it will make things harder to package for Fedora, because there
won't be the required packages in Fedora, because people will rather
bundle them in Flatpak.

At the end, everybody will be "maintaining" their versions of libraries
and forks of the libraries in Flatpacks, keeping security holes unfixed
(because it runs in container, what could happen, right?) etc.

Welcome in Windows world.

Vít

Re: Call for participation: Fedora Flatpaks

By Robin 'cheese' Lee at 09/07/2018 - 01:52

On Fri, Sep 7, 2018 at 9:53 AM Owen Taylor < ... at redhat dot com> wrote:

Re: Call for participation: Fedora Flatpaks

By Owen Taylor at 09/07/2018 - 04:06

On Fri, Sep 7, 2018 at 2:52 AM, Robin Lee < ... at fedoraproject dot org>
wrote:

What is meant here is "a runtime and Flatpaks built out of the Fedora RPMs
on Fedora infrastructure".

Some advantages this has over building and using Flatpaks on Flathub:

- In most cases, it's easier to create a Flatpak from an existing RPM
rather than creating a flatpak-builder manifest from scratch.
- We're able to reuse the Fedora updates infrastructure and automate
rebuilding and releasing Flatpaks and the runtime for security or other bug
fixes
- Applications with complicated build dependencies are easier to handle.
Any RPM in Fedora can be used as a build-time dependency. Only run-time
dependencies that aren't already in the runtime need to be rebuilt and
bundled, and even there it's a mostly automatic process.

(On the other hand, for an upstream application developer who knows nothing
about RPMs and specfiles and so forth, and just wants to create a Flatpak
of their application, flatpak-builder and Flathub is likely more attractive
than creating a Flatpak via Fedora packaging.)

It's not exclusive - you can use Flatpaks from Flathub and from this effort
together - even on a non-Fedora system. And, of course, you can contribute
to both Fedora and Flathub!

Owen

Re: Call for participation: Fedora Flatpaks

By Robin 'cheese' Lee at 09/07/2018 - 04:32

On Fri, Sep 7, 2018 at 5:15 PM Owen Taylor < ... at redhat dot com> wrote:

Re: Call for participation: Fedora Flatpaks

By Martin Kolman at 09/07/2018 - 04:29

On Fri, 2018-09-07 at 05:06 -0400, Owen Taylor wrote:

Re: Call for participation: Fedora Flatpaks

By Owen Taylor at 09/07/2018 - 04:55

Well, sort of. :-) What I've created so far is a Flatpak runtime to run
packages against - there's no corresponding SDK to use with
flatpak-builder. There's an initial definition of packages for such a SDK,
but I haven't actually tried creating the SDK, much less building anything
against it.

Also, the Flatpak runtime is not yet installable and updateable from a
Flatpak remote - we're still working on the last infrastructure pieces to
have that working.

But very soon!

That information is stored in from:

<a href="https://src.fedoraproject.org/modules/flatpak-runtime/blob/master/f/flatpak-runtime.yaml" title="https://src.fedoraproject.org/modules/flatpak-runtime/blob/master/f/flatpak-runtime.yaml">https://src.fedoraproject.org/modules/flatpak-runtime/blob/master/f/flat...</a>

It's hard to read because there are *four* list of packages in thre:
- A base runtime corresponding to org.freedesktop.Platform
- The corresponding SDK
- A full runtime (similar to org.gnome.Platform, but with Qt5 and other
useful additions)
- The corresponding SDK

So packages can be listed 4 times. The contents can also be explored
interactively at:

<a href="https://fishsoup.net/misc/flatpak-runtime-reports/runtime.html" title="https://fishsoup.net/misc/flatpak-runtime-reports/runtime.html">https://fishsoup.net/misc/flatpak-runtime-reports/runtime.html</a>

(The tabs at the top take you to other reports.)

Also, how long will the runtimes be supported by security fixes ? I guess
Exactly - the lifetime of the runtime is determined by the lifetime of the
Fedora release, unless we decide at a Fedora project level to support it
longer. I don't think that's immediately interesting, but maybe eventually.

Yes, you understand correctly.

[There's a wrinkle here in that the plan is to distribute the Fedora
flatpaks via registry.fedoraproject.org as OCI container images, rather
than via an ostree repository, but the support for that is in Flatpak 1.0
on all distros - 'flatpak remote add fedora oci+
https://registry.fedoraproject.org' will be all that is needed, and the
difference is handled by Flatpak behind the scenes.]

Owen

Re: Call for participation: Fedora Flatpaks

By Dominik 'Rathan... at 09/07/2018 - 03:22

On Friday, 07 September 2018 at 03:45, Owen Taylor wrote:
No, thank you. As I repeatedly said in the past, I consider Flatpaks and
similar things against the best interests of a Linux distribution. They
encourage bundling, take up unnecessary disk space and rely on package
management tools that lacked the basic feature of dependency checking
upon removal until very recently[1]. "App store"-like experience is not
what I would like to see in Fedora.

Regards,
Dominik

[1] <a href="https://github.com/flatpak/flatpak/issues/925" title="https://github.com/flatpak/flatpak/issues/925">https://github.com/flatpak/flatpak/issues/925</a>

Re: Call for participation: Fedora Flatpaks

By =?UTF-8?Q?Micha... at 09/07/2018 - 03:03

I thought there will be some automatic building script for flatpaks, so
the maintainers doesn't need to do it by themselves.

Is this automatization now dropped?

mkonecny

On 7.9.2018 03:45, Owen Taylor wrote:

Re: Call for participation: Fedora Flatpaks

By Owen Taylor at 09/07/2018 - 04:33

One level of automation is 'fedmod rpm2flatpak' which automatically figure
out dependencies and creates a module definition file which says what
Fedora packages need to be rebuilt for inclusion in your Flatpak. This
module definition file can often be used without any further editing.
'fedmod rpm2flatpak' also creates a container definition file
(container.yaml) which includes, among other things, the permissions for
the application. This does require manual editing, and, unless the
permission settings can be copied from an existing Flatpak of the
application, usually some testing to get right.

There's really no way to get around that level of human involvement - to
define the permissions for the app and check that the result works.

The other planned type of automation is automated rebuilds. We want to be
to the point where when there is an update for an included package, the
Flatpak is automatically rebuilt, a Bodhi update for the updated Flatpak is
filed, and email is sent to the maintainer requesting that they test the
new Flatpak. This does not exist yet, but the plan is to have it before
Fedora 30. For now, we need to build out the set of Flatpaks so we
understand how the system is working for maintainers and users.

Owen