DevHeads.net

Why retire Python 2 packages and games that still work to end user ?

Hi,

Why we would retire childsplay or gcompris or gdesklets ? IMHO we still
haven't a replacement .

From [1] I strongly disagree with the text, why all python 2 packages
will be removed automatically and why I would have a lot of work if I
want keep one package alive . why not the opposite ? .
Python 3 it isn't even the default why such hurry to drop python 2 at
all , when we still have epel 7 with python 2 ...

My opinion at least postpone this decision one or two releases to
Fedors 33/34 , many things still just work with python 2 .

Best regards,

[1]
<a href="https://fedoraproject.org/wiki/Changes/RetirePython2" title="https://fedoraproject.org/wiki/Changes/RetirePython2">https://fedoraproject.org/wiki/Changes/RetirePython2</a>

Comments

Re: Why retire Python 2 packages and games that still work to en

By Hans de Goede at 08/12/2019 - 06:34

Hi,

On 11-08-19 01:05, Sérgio Basto wrote:
childsplay and gcompris maintainer here.

Childsplay has been in a zombie state upstream for years, some work has
been done upstream but mostly focusing on reusing the code into a memory
trainer app for the elderly, rather then on educational activities.
Childsplay has always had a significant overlap with gcompris, given
the lack of upstream activity and the overlap with gcompris the time has
come to retire childsplay

gcompris has been replaced upstream by gcompris-qt, which is also in
Fedora and which we will keep around.

Both gcompris and childsplay have not seen much if any upstream love for
yeas now and have been kept functional only because of my efforts to keep
them building and working. The lack of any realistic path to get these
ported over to Python 3 is the last straw that breaks the camel's back.

The same goes e.g. for vegastrike which also has seen close to no
activity upstream for I guess 5 years at least...

Regards,

Hans

Re: Why retire Python 2 packages and games that still work to en

By =?ISO-8859-2?Q?... at 08/13/2019 - 09:31

Dne 12. 08. 19 v 13:34 Hans de Goede napsal(a):
Imho gcompris should have been remove long time ago from Fedora. And Gcompris-qt should obsolete-provides gcompris.
It just confuse people. It confused me, it confused Sergio.

Re: Why retire Python 2 packages and games that still work to en

By Hans de Goede at 08/13/2019 - 11:04

Hi,

On 13-08-19 16:31, Miroslav Suchý wrote:
Good idea, I've just filed a bug against gcompris-qt (which I do not maintain)
for this: <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1740801" title="https://bugzilla.redhat.com/show_bug.cgi?id=1740801">https://bugzilla.redhat.com/show_bug.cgi?id=1740801</a>

Regards,

Hans

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/10/2019 - 18:45

Sérgio Basto wrote:
I second that wholeheartedly.

This change is just not implementable as it stands. Way too much upstream
software still depends on Python 2. (In fact, I am not even convinced that
it can be implemented as stated, ever, without dropping huge parts of Fedora
and making it useless for a wide number of users. But what is sure is that
it definitely cannot implemented without huge fallout in time for Fedora
32.)

I also completely fail to see what value the rename from python2 to python27
(yet another one, after the already pointless rename from python to python2)
will bring to our users. But the worst part is the required FESCo exception
approval for every single remaining Python 2 package, along with loads of
bureaucracy that many packages will be unable to comply with (starting from
the plan to move to Python 3, which depends on upstream, if there is even a
live upstream to begin with).

This is absolutely not a reasonable and social way to deal with
compatibility packages in Fedora.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By David Sommerseth at 08/14/2019 - 06:41

On 11/08/2019 01:45, Kevin Kofler wrote:
Like it or not, Python 2 is going to die:
<https://www.python.org/dev/peps/pep-0373/>

Python 2 will not be maintained by upstream after January 1, 2020. Python 2
will go EOL during the lifetime of Fedora 31.

So why are we here? Because Python maintainers have ignored this, or have
hoped this will not be a reality or it will be postponed. But the PEP-373
defining and documenting the EOL of Python 2 was created in November 2008(!).
That is 11 years ago(!).

Life must move on, like or not. Python must move on, like it or not.

And since so many Python package maintainers have ignored this fact, we are
having this discussion now.

By not enforcing Python 2 to really die in Fedora, we will have exactly the
same discussion in the coming decade as well. There has been plenty of time
to get ready for the Python 3 move:
<https://fedoraproject.org/wiki/Changes/Python_3_as_Default> ... This happened
in Fedora 23 - which was released in November 2015. That is close to 4 years ago.

So why are we here? Because Python 2 package maintainers in the Fedora
community have ignored this fact, for almost 4 years. Yes, I know it's not
necessarily an easy task. But 4 years in Fedora land is quite a long time; it
is 8 Fedora releases. If we want to do a move, it is possible to do such a
change in 4 years.

Time has run up. It is time to move on and accept the fate of Python 2
packages not being ready. Those caring so much for unported Python 2 packages
now got a brilliant chance to help moving them forward to Python 3 too.

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/14/2019 - 08:01

David Sommerseth wrote:
So what? Qt 3 had its last release (3.3.8b) in 2008 (and that was basically
just a relicensing of the final 2007 release 3.3.8). Yet, we (mostly me)
still maintain qt3 and backport security fixes where issues are reported to
us, and Qt 3 applications still work fine. I have KSensors (a qt3/kdelibs3
application) sitting in my system tray all the time. It still works (thanks
also to Plasma 5's xembedsniproxy). I also use Quanta Plus to edit HTML. It
also works as expected.

Just because upstream no longer releases something does not mean it has to
disappear from distributions instantly. I see no other distribution being as
aggressive about removing Python 2 as Fedora is.

There is also a fork trying to keep Python 2 alive:
<a href="https://github.com/naftaliharris/tauthon" title="https://github.com/naftaliharris/tauthon">https://github.com/naftaliharris/tauthon</a>
though it is unfortunately unclear whether the most important point
(backporting security issues) will be taken care of reliably:
<a href="https://github.com/naftaliharris/tauthon/issues/109" title="https://github.com/naftaliharris/tauthon/issues/109">https://github.com/naftaliharris/tauthon/issues/109</a>
But it is always possible to do the backports downstream, as we are doing
for the Qt 3 and 4 stacks.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By David Sommerseth at 08/14/2019 - 08:21

On 14/08/2019 15:01, Kevin Kofler wrote:
So what? You chose to do this work to keep this alive. In my personal
opinion, that's wasting of precious time and I would never have done that.
You chose differently, and I respect that.

I honestly don't care much what other distros do. I care about the road
forward for Fedora. In my point of view, putting legacy software which has
reached EOL from upstream behind is really sane. If you want/need it, then
port it forward to the future.

Secondly, this isn't "disappearing instantly", as you say. Fedora started
this migration almost 4 years ago. If this feels "instantly", someone has not
paid attention.

And this is why we should let Python 2 rest in peace once it reaches EOL.
Python 3 contains a lot of improvements over Python 2. The world need to move
forward and not linger in the past.

Who will do this job? And which guarantees do we have it will be done in a
way providing the same quality work the Python community has done so far?

If the result is less secure Python 2 updates, nothing really improves at all
... except keeping code which should be ported forward to Python 3 alive
longer than really needed.

Instead of spending time and resources keeping old stuff alive longer than
needed, rather spend that energy on porting the old Python 2 over to Python 3.
In the long run, this will result in far less work over time.

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/14/2019 - 15:28

David Sommerseth wrote:
It is not practical to get all the legacy Python 2 code ported over to
Python 3. Keeping Python 2 (or something backwards-compatible with it such
as Tauthon) available is actually the much more scalable approach.

Your scorched earth approach will lose a lot of software that has no
replacement and that users rely on.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By Nico Kadel-Garcia at 08/14/2019 - 21:49

On Wed, Aug 14, 2019 at 4:31 PM Kevin Kofler <kevin. ... at chello dot at> wrote:
I've been seeing migrations like this for d decades, with major
releases of many software tools. Preserving legacy versions, forever,
is the precise opposite of "scalable".

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/15/2019 - 07:33

Nico Kadel-Garcia wrote:
What is more work: maintaining one compatibility package, or porting
hundreds of packages (which are not getting ported upstream for whatever
reason) to the new incompatible version?

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By =?UTF-8?B?TWlyb... at 08/15/2019 - 07:55

On 15. 08. 19 14:33, Kevin Kofler wrote:
Once more: The one package you keep talking about stays.

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/16/2019 - 00:20

Miro Hrončok wrote:
The python2 package stays, but we have to jump through completely
unreasonable hoops to be allowed to actually use it.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By Stephen John Smoogen at 08/14/2019 - 15:58

On Wed, 14 Aug 2019 at 16:31, Kevin Kofler <kevin. ... at chello dot at> wrote:
And your daily rants about everything wrong everyone else does loses a
lot of people who want to work on Fedora. The current python
maintainers have said multiple times that they would be happy if
someone took over python2 once it is completely EOL. You can take it
over just like qt3 and no one has to worry about it anymore.

Re: Why retire Python 2 packages and games that still work to en

By =?UTF-8?B?TWlyb... at 08/14/2019 - 15:35

On 14. 08. 19 22:28, Kevin Kofler wrote:
We are keeping the Python 2 interpreter.
We are also keeping PyPy 2.7.
You can also package Tauthon if you want to.

It's the packages with the ecosystem that we want to get rid of.

Re: Why retire Python 2 packages and games that still work to en

By Nico Kadel-Garcia at 08/10/2019 - 19:01

On Sat, Aug 10, 2019 at 7:47 PM Kevin Kofler <kevin. ... at chello dot at> wrote:
Maintaining python 2 requires maintaining a *lot* of infrastructure.

To support multiple pythons, python26, python28 if it ever happens,
python36 python38 when it comes out.

Simply replacing the "python2" line iwth "python27" is not a difficent
edit in most source code.

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/10/2019 - 20:45

Nico Kadel-Garcia wrote:
What kind of infrastructure do you need to maintain a package that is (will
be) no longer updated upstream? This takes almost no work. The only thing to
do is to backport some security fixes from Python 3, and occasionally to fix
FTBFS bugs.

Now if your concern is maintaining all the python2-* packages, then there
ought to be some middle ground between maintaining all of them including
ones that are not used by anything in Fedora anymore, and just dropping all
of them (with or without the planned exception process, whose usefulness
depends entirely on how willing FESCo will be to approve the requests).
Maybe the set of python2-* modules in EL7 or EL8, with or without EPEL,
minus the ones already retired from Fedora by now, would be a reasonable
starting point?

I also think that there ought to be more cooperation from the maintainers of
individual python2-* modules. The approved Fedora 31 Change makes it way too
easy for maintainers to just drop Python 2 support for no reason. If
upstream dropped Python 2 support from the current version and so an old
version has to be packaged specifically for Python 2, that is one good
reason to require somebody else to pick up maintenance, but as it stands, no
such reason is required and Fedora maintainers can arbitrarily drop Python 2
support from their Python modules even if upstream still supports it. This
is just pointless and unhelpful.

We can try to find people to pick up python2 and a bunch of python2-*
modules, but expecting the python2 maintainer(s) to sign up for maintaining
each and every python2-* module is unreasonable and unrealistic. Even if
several of them will also be dead upstream (at least the version that works
with Python 2) and thus require very little maintenance effort.

The whole reason for this discussion is that Python 2.8 will likely never
happen. It definitely will not happen from Python upstream. Otherwise, there
would not be all this talk about dropping Python 2 due to being unsupported
upstream.

I don't see much point in supporting Python 2.6, which is even more ancient
than Python 2.7, and 2.7 is backwards-compatible with 2.6.

And supporting multiple versions is not an argument for not having at least
a python2 symlink and Provides: python2 in python27.

But it is still a completely unnecessary edit.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By =?UTF-8?B?TWlyb... at 08/11/2019 - 03:32

On 11. 08. 19 3:45, Kevin Kofler wrote:

We are still planning to maintain the interpreter. As is documented in the
change. So can we please stop arguing about maintaining the interpreter over and
over? It is staying and our team will maintain it at least until RHEL 7 EOL,
possibly longer.

Yes the concern is about maintaining the python2-* packages.

The difference between EL and Fedora is that package sin Fedora get recent
rebases / updates to newer version. You cannot just package them and call it
"done". And most of the upstreams are coming to a point where you cannot update
the Python 2 package because the new version doe snot support Python 2.

That at the end means you need to go over nonobvious bumps to split the Python 2
package out of the "common" package, in order to update the "common" (now Python
3 only) package. And this problem spreads further with dependencies (even
transitive ones). I don't have the energy or time to split hundreds of packages
and maintain a separate stack of Python 2 packages. If somebody else has, go for it.

The set of python2-* packages to remain is determined by the FESCo exceptions.
It is fairly easy really: We don't have the necessary information to understand
what Python packages are "important" and what are "cruft". Hence if your package
is important, you just need to explain that. Obviously, if you need to maintain
the package as a dependency, for another important package, the exception can be
requested at once, you don't need to request dozens of exception to keep e.g.
chromium around.

When a packager doesn't want to maintain it, that's good enough reason.

You have a right to orphan a package, why you should not have the right to
orphan a half of your package, when he other half works without it?

Requiring others to maintain the packages your packages (or you) need just
because they maintained it until now is not very friendly. Of course, you can
ask nicely, but you cannot say this is their duty.

Arguably, maintaining dead software requires more effort than maintaining live
one. But that kinda depends on the particular package.

I don't need people to maintain **all** Python 2 packages, just mine. But
possibly other maintainers also don't want to maintain theirs. As the snowball
rolls, you need somebody to maintain **almost all** of them.

Correct. We just want to signal that this is a compat package. We will most
likely still provide python2 (so users can discover it more easily). Why is the
package name so important to you?

The point is to support developers who use Fedora as they workstation but need
to support and test Python 2.6 (e.g. on EL6) in their code. Nevertheless, the
python26 package is now orphaned, because one of the dependencies is orphaned
(compat-openssl10).

Correct. Except both of those to happen.

Yes.

Re: Why retire Python 2 packages and games that still work to en

By Peter Robinson at 08/14/2019 - 06:09

On Sun, Aug 11, 2019 at 9:33 AM Miro Hrončok < ... at redhat dot com> wrote:
So why do package maintainers have to do a whole lot of extra work to
keep a package that has already been approved in the distro. There's a
lot of work going into various stacks upstream for python3 work but in
a lot of cases the time available is split and now we're asking the
limited maintainers of packaged in Fedora to do more work
unnecessarily to keep their packages around otherwise they're be
aggressively killed off and they'll have to then do even more work to
get them back.... or probably just go and use Ubuntu or CentOS or
something else instead. This is frankly a ridiculous policy!

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/14/2019 - 07:51

Peter Robinson wrote:
Are you also bringing this one to the Council?

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By Kevin Kofler at 08/11/2019 - 07:28

Miro Hrončok wrote:
Then why do you require all this FESCo exception bureaucracy, including a
Python 3 migration plan, for every single package requiring Python 2, even
if it is only the interpreter (and the shipped standard library)?

And that is a perfectly valid reason to orphan the Python 2 parts. My issue
with the policy you proposed and FESCo approved for F31 is that it does not
require this to be the case, but allows maintainers to drop the Python 2
subpackage just because they don't like Python 2 (or simply want to avoid
going through the bureaucracy you're requiring for F32), which is a quite
antisocial thing to do.

If the partial orphaning policy were limited to packages where upstream
dropped or is in the process of dropping Python 2 support, I would not
object.

That said, those are also the cases where the split out python2-* package
WOULD in fact be "done" and never require getting updated to a newer
version, unless upstream maintains some legacy/LTS branch. So the situation
for whomever ends up stuck maintaining the python2-* package would not be
that different from the RHEL situation.

But why does this have to go through FESCo and a formal approval process?
Why is it not sufficient that the maintainer says "yes, this is still
needed"? If a maintainer wants to keep the package, this clearly means it is
important to SOMEBODY, so why do we need an approval by committee to confirm
this (or worse, veto this against the wishes of the maintainer)?

We do not normally require this level of bureaucracy for things depending on
a compatibility package, and I think this is a very dangerous precedent to
set.

When it takes no extra effort to maintain the Python 2 module because the
current upstream still supports it just fine, what is the rationale for
dropping it, other than "I don't like Python 2 anymore"? As I wrote above,
if upstream stops supporting Python 2, that is a valid reason. If nothing in
Fedora (nor in popular third-party repositories) requires the Python 2 parts
anymore, that is a reason too.

All I am asking for is to not actively drop python2-* subpackages without a
good reason to do so. (It actually requires more effort to go and delete the
specfile portions than to just keep them. ;-) )

My experience maintaining the Qt 3 stack is quite the opposite. Those
packages require a lot less effort to keep going than, say, qt5-qtwebengine.

My idea is that people depending on specific python2-* modules should
maintain them, as usual when a library gets orphaned. But we cannot expect
one person to maintain all of them. Neither you, nor me, nor anybody else.

What I want to see is a small group of python2 maintainers keeping the
interpreter going, and then individual packagers of dependent packages
(reverse dependencies) maintaining the individual python2-* modules they
need for their packages. If nobody can be found for a specific module, then
that module should get retired by the normal retirement process for orphaned
packages. I don't see why a more aggressive removal process is needed here.

OK.

Kevin Kofler

Re: Why retire Python 2 packages and games that still work to en

By Petr Viktorin at 08/12/2019 - 04:46

On 8/11/19 2:28 PM, Kevin Kofler wrote:
Python 2 will not get the attention, fixes, and security updates that
people have been used to in the past decade. That's a big deal, and
unfortunately we know no good way to properly communicate this to all
the affected packagers.
We hope the bureaucracy works for the hundreds of packages with inactive
maintainers; the flip side is that the active maintainers do have to do
some more work, even if it's just filing a ticket and switching a
BuildRequires. Sorry for that.

As for the Python 3 migration plan: we can agree to maintain Python 2
for you to depend on, if there's a feasible plan of moving away from
Python 2. If there's no plan, you'll just run into trouble in a few more
years. If you consider your package important, we really want to know
about the situation.

The Python 3 migration plan is not a requirement, but a very common and
useful piece of information that we want to hear about. Every package is
different; we can't know where each upstream does their planning. As a
packager, you know best where to ask for this info, so we ask you to
find it and paste the link in a Fedora place -- Bugzilla or a FESCo ticket.
If there's another plan instead of a "Python 3 porting plan" (like port
to Rust instead, or to Tauthon, or maintain Python 2 forever), we'd also
like to know about it.

The policy is for those *and their dependencies*.

See, I have a "package where upstream is in the process of dropping
Python 2 support". It's called "python2".
I could just drop it and call it a day, but since people still want it,
I'll maintain it. But only for those who care and who understand the
situation; not for the hundreds of inactive maintainers.
(To be clear: making it possible and straightforward to build Fedora
RPMs with python3 *is* more work than just maintaining the interpreter
for users.)

Sure, but this "python2" one does need ongoing maintenance.

The maintainer needs to say "yes, this is still needed" for all the
(transitive) dependencies as well. There's usually a non-trivial amount
of these.
If you maintain one of the packages that just need the interpreter
itself, then yes, I can see how opening a FESCo ticket is unneeded
bureaucracy. Sorry for that, but please understand that this is not a
common case.
If you maintain *many* packages like that, let us know and we'll do
something to treat them as a batch.

I don't think we really had a compatibility package with that many
dependents, unfortunately.
As for the precedent, yes, I sure hope we're not setting one.

It always takes extra effort to maintain the Python 2 module's
dependencies. It might not be *your* effort, but it is effort.

You are quite lucky, it seems. Congratulations!

I agree with that...

...but not with this. The graph of dependencies is too big and tangled,
and it changes too much over time (if you don't remove leaves, they stop
being leaves). And since many of the packagers involved aren't
responsive, we kind of need them to explicitly say "yes, this is
important and we should keep all the dependencies" rather wait for "no,
drop this".
Hence the whole complicated process for figuring out who should maintain
python2 modules.

For example, at my last count, Chromium had 57 Python2 packages as
transitive (build)dependencies, many of which are probably not best
maintained by Chromium maintainers. Calibre had 104. And those are
programs we know people care about; there are hundreds where we have no
idea how important they are to their packagers or users.

Re: Why retire Python 2 packages and games that still work to en

By Nico Kadel-Garcia at 08/11/2019 - 06:15

It's not merely difficult, it's burning time better spent porting the
python2 packages to python3

I've run into this snowball, quite recently, backporting awscli to
RHEL 6. I finally had to throw in the towel for Samba and give up on
RHEL 6 for current Samba releases with the domain controller enabled.

It's proven helpful with the python3 enabled packages to use
"%{_python3}" and "%{_python2}" consistently, especially when
splitting packages for versions backported to RHEL or publiswhed in
EPEL. Red Hat is due to publish a python3 built right into RHEL 7.7,
so it should be possible to discard python2 more generally for folks
like me that do backports.

Re: Why retire Python 2 packages and games that still work to en

By =?UTF-8?B?TWlyb... at 08/10/2019 - 18:35

On 11. 08. 19 1:05, Sérgio Basto wrote:
If the maintainer wants to, they can request an exception for the package to be
kept.

Opposite? Retire Python 3?

Python 3 is the default.

As I ask repeatedly: Who will maintain the Python 2 ecosystem?

Re: Why retire Python 2 packages and games that still work to en

By =?ISO-8859-1?Q?... at 08/10/2019 - 18:52

On Sun, 2019-08-11 at 01:35 +0200, Miro Hrončok wrote:
yes, but just gives more work to package maintainers .

The opposite, is just removed the authorized packages . If maintainer
not respond or aren't against to .

On Fedora 30 still be Python 2 (Python 2.7.16) and Fedora 31 is not yet
released .

Thanks,