DevHeads.net

Fedora 31 System-Wide Change proposal: Python means Python3

<a href="https://fedoraproject.org/wiki/Changes/Python_means_Python3" title="https://fedoraproject.org/wiki/Changes/Python_means_Python3">https://fedoraproject.org/wiki/Changes/Python_means_Python3</a>

== Summary ==
In package and command names, "Python" will mean "Python 3".

Users installing and running Python or Python packages without
specifying a version will get Python 3.

Running <code>python</code> will run <code>python3</code>. Running
<code>pytest</code> will run the Python 3 version of pytest, and
similarly for <code>pydoc</code>, <code>pylint</code>, etc.

<code>dnf install python</code> will install {{package|python3}}, and
similarly for other <code>python-*</code> provides, e.g. <code>dnf
install python-requests</code> will install
{{package|python3-requests}}.

This is the final preparation for
[https://fedoraproject.org/wiki/Changes/RetirePython2 the retirement
of Python 2 in Fedora 32] done in line with the
[https://github.com/python/peps/pull/989 soon to be finalized]
upstream recommendations.

== Owner ==
* Name: [[User:Churchyard| Miro Hrončok]]
* Name: [[User:Pviktori| Petr Viktorin]]
* Email: <python- ... at redhat dot com>

== Detailed Description ==

=== Motivation ===

The final upstream release of Python 2 is planned for January 2020. No
further fixes will be made upstream. Most of Fedora 31's lifetime is
after that date. Python 2 will be maintained only by its Fedora
maintainers.

In preparation for removing Python 2 from Fedora entirely, we will
make the unqualified name "Python" refer to the fully supported
version.

This is in line with upstream changes to
[https://www.python.org/dev/peps/pep-0394/ PEP 394] recommendations.
(At the time of this writing, these changes are still
[https://github.com/python/peps/pull/989 being finalized], but
recommendations for Linux distributions are accepted.)

This completes the work started with Fedora 23's
[https://fedoraproject.org/wiki/Changes/Python_3_as_Default "Python 3
as Default"] change, with only
[https://fedoraproject.org/wiki/Changes/RetirePython2 removing Python
2] left to do.

=== What is being changed ===

The following changes will be made both in the
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/
Python packaging guidelines] and the actual packages.

==== Package provides ====

'''Before:''' Packages with Python 2 modules used to provide the
unversioned <code>python-</code> name. Users could do <code>dnf
install python-requests</code> and the {{package|python2-requests}}
package would be installed.

'''After:''' Packages with Python 3 modules will provide the
unversioned <code>python-</code> name. Users can do <code>dnf install
python-requests</code> and the {{package|python3-requests}} package
will be installed.

This applies to the {{package|python3}}/{{package|python2}} package as
well as packages with Python modules. <code>dnf install python</code>
will install {{package|python3}}.

The vast majority of packages will be updated by changing
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_the_python_provide_macro
the %python-provide macro], with no changes in the individual spec
files. The change owners will change the macro before the mass
rebuild.

==== Python command ====

'''Before:''' The <code>/usr/bin/python</code> command was a symbolic
link to <code>/usr/bin/python2</code> living in the
{{package|python-unversioned-command}} subpackage of
{{package|python2}}. {{package|python2}} recommended
{{package|python-unversioned-command}}, so most users got the command
by default when {{package|python2}} was installed.

'''After:''' The <code>/usr/bin/python</code> command will be a
symbolic link to <code>/usr/bin/python3</code> living in the
{{package|python-unversioned-command}} subpackage of
{{package|python3}}. {{package|python3}} will recommend
{{package|python-unversioned-command}}, so most users will get the
command by default when {{package|python3}} is installed (and it is
installed by default).

==== Other commands ====

Similarly, the Python-specific commands will switch from Python 2 to
Python 3. This includes at least the following commands:

* pip
* python-config
* wheel
* idle
* pydoc
* pytest
* nosetest
* pycodestyle
* pylint
* epylint
* pyreverse
* symilar
* unit2
* msgfmt.py
* pygettext.py
* flask
* ipython
* f2py
* ipdb
* easy_install
* ...

This list is incomplete, automation is yet to be created to discover
all such commands.

Note that applications like <code>pygmentize</code> or
<code>cython</code>, whose behavior doesn't depend on the Python
version, are not affected. (By current guidelines, they should be
already using Python 3 if possible.)

=== Changes needed by Python package maintainers ===

This section of the change covers action needed from Python package
maintainers. Most of the packages need no change, but there are
several exceptions.

==== Packages with ambiguous names ====

Tracked at: <a href="https://fedora.portingdb.xyz/namingpolicy/" title="https://fedora.portingdb.xyz/namingpolicy/">https://fedora.portingdb.xyz/namingpolicy/</a>

If your package with a Python 2 module or plugin is named with
unversioned <code>python</code> (such as for example
<code>claws-mail-plugins-python</code> or <code>PyQt4</code>), it
needs to be removed or renamed to have <code>python2</code> in the
name (such as, for example, <code>claws-mail-plugins-python2</code> or
<code>python2-PyQt4</code>).

If your package is an application that happens to be written in Python
2 (such as {{package|calibre}}), no renaming is needed (applications
don't need the <code>python-</code>, <code>python2-</code> or
<code>python3-</code> prefix).

==== Packages with ambiguous requires ====

Tracked at: <a href="https://fedora.portingdb.xyz/namingpolicy/" title="https://fedora.portingdb.xyz/namingpolicy/">https://fedora.portingdb.xyz/namingpolicy/</a>

If your package depends on (Requires, BuildRequires, Recommends, etc.)
a Python package with unversioned Python name (such as for example:
systemd-python, python-setuptools, PyQt5, pycairo), it will now get
resolved to the Python 3 version. Such dependencies need to be updated
to specific Python 2 or Python 3 names (such as for example:
pythonX-systemd, pythonX-setuptools, pythonX-qt5, pythonX-cairo where
X is either 2 or 3).

==== Packages with ambiguous provides ====

Most of the backwards compatibility <code>python-*</code> provides are
handled by the %python_provide macro and the change owners will change
the macro behavior, so you don't have to worry about those.
However, sometimes manual provides were added. If your package has
some manual backwards compatibility provides for Python 2 packages,
those need to be moved to the Python 3 packages or removed.

For example, before this change, the {{package|python2-m2crypto}}
package provided and obsoleted <code>m2crypto</code>. Instead, the
{{package|python3-m2crypto}} package should do that after this change.

==== Packages with missing %python_provide ====

The <code>%python_provide</code> macro used to do nothing for Python 3
packages. As such, it was often forgotten and only used with the
Python 2 packages.
Packagers of Python 3 packages should make sure to use the
<code>%python_provide</code> macro according to the guidelines:

%package -n python3-%{srcname}
...
%{?python_provide:%python_provide python3-%{srcname}}

==== Packages with Python versioned commands and tools ====

Some packages have two versions of the provided commands. For example,
the {{package|python2-pytest}} package has
<code>/usr/bin/pytest-2</code> and <code>/usr/bin/pytest-2.7</code>,
the {{package|python3-pytest}} package has
<code>/usr/bin/pytest-3</code> and <code>/usr/bin/pytest-3.X</code>.
The unversioned <code>/usr/bin/pytest</code> command used to be a
symbolic link to <code>/usr/bin/pytest-2</code>. Now it needs to be
changed to <code>/usr/bin/pytest-3</code> and moved to the
{{package|python3-pytest}} package.

For some packages, such duplication makes no sense, because the user
sees no difference. For example, there should be just one
<code>/usr/bin/pygmentize</code> -- the user doesn't care if it runs
on Python 3, Python 2 or if it is written in Rust. This is not a new
rule, but if your package is not following it, now is a good time to
make sure the tool uses Python 3.

==== Packages that need unversioned Python to be Python 2 ====

Since Fedora 29, a
[https://fedoraproject.org/wiki/Changes/Move_usr_bin_python_into_separate_package
specific workaround is needed] to use the <code>python</code> command
as Python 2.
This workaround will now bring Python 3.
If that does not work for your package, you'll need to fix it (patch
it) or retire it from Fedora.

If you only need the <code>python</code> command to mean Python 2
during package build, you can do something like this:

mkdir tmp_path
ln -s %{__python2} tmp_path/python
PATH=$(pwd)/tmp_path:$PATH make ...

However, if your package uses the <code>python</code> command during
runtime, this ugly workaround won't work.

== Benefit to Fedora ==
The name "Python" will not refer to software that will be unmaintained
upstream for most of Fedora 31's lifetime and
[https://fedoraproject.org/wiki/Changes/RetirePython2 retired from
Fedora 32].

== Scope ==

* Proposal owners:
** Changes in {{package|python3}}, {{package|python2}} packages:
*** make <code>/usr/bin/python</code> link to
<code>/usr/bin/python3</code> instead of <code>/usr/bin/python2</code>
(+ the same for other executables there)
*** make {{package|python-unversioned-command}} a subpackage of
{{package|python3}} instead of {{package|python2}}
*** make {{package|python3}} (instead of {{package|python2}})
recommend {{package|python-unversioned-command}}
** Changes in other commands:
*** switch pytest, nosetests, ipython, pip... to python3 (see Detailed
Description for the actual list)
** Changes in {{package|python-rpm-macros}}:
*** make <code>%python_provide</code> provide <code>python-foo</code>
for <code>python3-foo</code> instead of for <code>python2-foo</code>
** Let the mass rebuild change the provides based on the
<code>%python_provide</code> change
** Examine the failures and provide help to package maintainers
** File bugs for remaining packages that provide <code>python-x</code>
or <code>x-python</code> for Python 2

* Other developers (Python package maintainers, see Detailed Description):
** Everybody: Make sure that you require and use python2 or python3
packages explicitly
** If your package provides <code>python-*</code> or
<code>*-python</code> for a Python 2 package by other means than the
<code>%python_provide</code> macro, move that to the Python 3 package
(or remove entirely if not applicable)
** Fix or retire your package if it requires the unversioned
<code>python</code> command to be Python 2

* Release engineering: [https://pagure.io/releng/issue/8482 #8482]

* Policies and guidelines:
** Slightly adapt
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_the_python_provide_macro
the %pyton_provide macro] section of the Python packaging guidelines
and the [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/
Python Appendix].
* Trademark approval: not needed

== Upgrade/compatibility impact ==
Custom scripts with <code>python</code> shebangs will invoke Python 3
by default, whereas previosuly they invoked Python 2 by default. See
the User Experience section for details.

== How To Test ==
For your favorite Python 3 package, check that it can be installed
with the ambiguous Python name. For example, check that <code>dnf
install python-pip</code> installs {{package|pyhon3-pip}} instead of
{{package|pyhon2-pip}}.

For your favorite Python command, check that it invokes the Python 3
version. For example, check that <code>python</code> runs Python 3.

== User Experience ==
Users who run <code>python</code> directly will get Python 3.
Users who run <code>pip</code> directly will get pip for Python 3.
Users who run <code>pytest</code> directly will get Python 3 pytest
for Python 3.
...

Scripts with ambiguous Python shebangs (<code>#!/usr/bin/python</code>
or <code>#!/usr/bin/env python</code>) will be executed by Python 3 by
default.

If users need the <code>python</code> command or the
<code>#!/usr/bin/env python</code> shebang to run Python 2, they can
easily do that by:

ln -s /usr/bin/python2 ~/.local/bin/python

Similarly, sysadmins can do that system-wide:

ln -s /usr/bin/python2 /usr/local/bin/python

If users don't want the python command at all, they can <code>dnf
remove python-unversioned-command</code>.

== Dependencies ==
Most packages that provide Python version-specific functionality will
be affected. However, the Change owners include proven packagers and
they maintain python-rpm-macros, so most will not need packager
attention.

We depend on the Fedora mass rebuild to adjust macro-generated package provides.

(Also, the PEP 394 update could be delayed, or even rejected/changed.
Even if that happens, this Change will not be affected: Fedora would
depart from following the upstream recommendations.)

There are currently (2019-06-25)
[https://fedora.portingdb.xyz/namingpolicy/ 90 packages left with
ambiguous Python requires]. Those need to be adapted. Lot of them
unfortunately Fail to Build From Source and will be retired before the
Fedora 31 branching.

== Contingency Plan ==
* Contingency mechanism: revert the changes, mass rebuild packages
with the original <code>%python_provide</code> macro
* Contingency deadline: beta freeze
* Blocks release? No
* Blocks product? No

== Documentation ==
* Updated Packaging guidelines
* This Change page

== Release Notes ==
TBD. Check User Experience section.

Comments

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/28/2019 - 09:46

On 26. 06. 19 19:57, Ben Cotton wrote:
Based on some questions and proposals that were repeated in the thread (thank
you!), we have edited the proposal to include an "Alternative proposals" section:

<a href="https://fedoraproject.org/wiki/Changes/Python_means_Python3#Alternative_proposals" title="https://fedoraproject.org/wiki/Changes/Python_means_Python3#Alternative_proposals">https://fedoraproject.org/wiki/Changes/Python_means_Python3#Alternative_...</a>

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Zbigniew =?utf-... at 06/27/2019 - 16:09

On Wed, Jun 26, 2019 at 01:57:13PM -0400, Ben Cotton wrote:
What about postponing this change to F32? I'd prefer python2 to be
retired and gone from the distro first, and the symlink and
%python_provide definition only switched then. I think that having
this middle state where python2 is available but python points to
python3 for exactly one release will be more confusing that switching
directly to the final state where python2 is gone and python simply
means python3.

Zbyszek

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By King InuYasha at 06/27/2019 - 17:00

On Thu, Jun 27, 2019 at 5:59 PM Zbigniew Jędrzejewski-Szmek
< ... at in dot waw.pl> wrote:
I think it makes sense to make the switch before we retire, because
then people's expectations are changed ahead of time and they can
adapt to The Future(TM).

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 19:26

On 28. 06. 19 0:00, Neal Gompa wrote:
This is exactly the reason we want this in F31.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Zbigniew =?utf-... at 07/01/2019 - 02:02

I'm now convinced postponing is not appropriate.

In the meantime, I decided to ask some non-developers (biologists actually)
for their opinion, and the majority favours having /usr/bin/python → python3.
The arguments given elsewhere in the thread for always having
*something* as /usr/bin/python are also quite convincing. I'm now
fully on-board with the Change.

Zbyszek

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 07/01/2019 - 02:44

On 01. 07. 19 9:02, Zbigniew Jędrzejewski-Szmek wrote:
Thanks Zbyszek,

if you have some quotes, examples or other relevant arguments, would you mind
putting them on the wiki?

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Zbigniew =?utf-... at 07/01/2019 - 06:51

On Mon, Jul 01, 2019 at 09:44:35AM +0200, Miro Hrončok wrote:
It wasn't really anything new. Just the general sentiment among users that
adapting to python3 is inevitable or already implemented.

Zbyszek

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Martin Kolman at 06/28/2019 - 11:07

On Thu, 2019-06-27 at 18:51 -0400, Stephen John Smoogen wrote:

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Michael Catanzaro at 06/28/2019 - 10:42

On Thu, Jun 27, 2019 at 5:51 PM, Stephen John Smoogen
< ... at gmail dot com> wrote:
It would mean we have no chance of maintaining python scripts that work
for both macOS and Fedora. Fedora's preference doesn't win here: on
macOS /usr/bin/python is python2, and there is no /usr/bin/python2 so
we can't use that in shebangs, and none of that is going to change. So
e.g. WebKit's scripts have to use #!/usr/bin/python or #!/usr/bin/env
python no matter what Fedora wants. Trying to convince Apple to use a
virtualenv for building WebKit isn't going to happen.

(Actually #!/usr/bin/python cannot be used ever, because that doesn't
work on FreeBSD, so we use #!/usr/bin/env python and rely on
downstreams to rewrite it to the #!/usr/bin/python version if required
for packaging. Messy enough yet?)

Anyway, we were able to make WebKit build in Fedora without using
/usr/bin/python by using a combination of (a) changing shebangs for
Linux-specific scripts, and (b) executing the scripts through CMake
otherwise, so the shebang doesn't matter. So it's not an issue that
/usr/bin/python is unavailable during package builds. It's an issue for
developers using Fedora who need to use unpackaged scripts that are
required to work on macOS.

If Fedora has /usr/bin/python, then at least we have a *chance* to make
the scripts we care about work in both python2 and python3 (our current
plan). Whereas without /usr/bin/python, we're really out of options. So
I very much support /usr/bin/python -> /usr/bin/python3. It will cause
some problems for us and we'll have a difficult transition period, but
at least there will exist the possibility of transitioning.

Without /usr/bin/python we'd probably have to tell developers to
manually install as python2 in /usr/local so that scripts using
/usr/bin/env python get python2... way worse.

Michael

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Michael Catanzaro at 07/10/2019 - 10:09

Surprise, apparently python3 is coming to macOS in some form (maybe
only for developers? not sure?) so WebKit should be fine with switching
to pytohn3 regardless.

I don't think it's important to this discussion, just wanted to send a
correction. I wasn't expecting this....

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Chris Murphy at 07/10/2019 - 11:25

On Wed, Jul 10, 2019 at 9:10 AM < ... at gnome dot org> wrote:
My reading of the developer notes is they're dropping the inclusion of
python entirely and developers will just have to install it
themselves, most likely from Homebrew (3rd party repo and package
manager), who only offer python 3.7.4 currently.

<a href="https://developer.apple.com/documentation/macos_release_notes/macos_catalina_10_15_beta_3_release_notes" title="https://developer.apple.com/documentation/macos_release_notes/macos_catalina_10_15_beta_3_release_notes">https://developer.apple.com/documentation/macos_release_notes/macos_cata...</a>

Also, macOS is switching from bash to zsh (?) I don't know why.
<a href="https://support.apple.com/en-ca/HT208050" title="https://support.apple.com/en-ca/HT208050">https://support.apple.com/en-ca/HT208050</a>

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Dridi Boukelmoune at 07/10/2019 - 12:19

Probably because they stuck to an old version of bash to avoid the
GPLv3 re-licensing, or so I've been told. Maybe the version of bash
they use is going EOL as well.

Dridi

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Michael Catanzaro at 06/28/2019 - 10:46

On Fri, Jun 28, 2019 at 10:42 AM, <a href="mailto: ... at gnome dot org"> ... at gnome dot org</a> wrote:
Accidentally hit send too soon. I meant to add: /usr/bin/python ->
/usr/bin/python3 is better than all available alternatives. It's the
only way that /usr/bin/python remains in Fedora pointing to a supported
python interpreter. And that is the only chance that cross-platform
python scripts have to work without pain and suffering (beyond that of
making the script work with both python2 and python3). We're not python
developers and just want to run some python scripts; wrapping up all
our python inside e.g. bash scripts that start a virtualenv would be a
sad end to this tale.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?Q?M=c3=... at 06/28/2019 - 12:23

On 06/28/19 10:46, <a href="mailto: ... at gnome dot org"> ... at gnome dot org</a> wrote:
Thank you. I work on multiple platforms so I make my utility scripts
work on both Python 2 and 3 and only use the standard library. It
would be super annoying if I had to have multiple versions just because
of the shebang line.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Rex Dieter at 06/28/2019 - 09:22

Apologies for not fully reading the proposal to realize this is a new
upstream recommendation. I withdraw any objection.

-- Rex

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 19:44

On 28. 06. 19 0:51, Stephen John Smoogen wrote:
We've been actively forbidding packagers doing that for more than a year.
Most packages that still require /usr/bin/python are either:

* FTBFS since Fedora 28 (and I will make sure we follow the policy this time
and finally kill those)

or

* willingly workarounded by the packagers who tend to ignore all our
recommendations (nothing we can really do here)

Totally that is 10 runtime dependent packages and 64 buildtime.

If we take away /usr/bin/python and "python" provide, those things won't resolve.

If we change it to Python 3, some of them might work, most of them probably
won't. Some of them are broken already (like

$ (repoquery --repo=rawhide-source --whatrequires python; repoquery
--repo=rawhide-source --whatrequires python-unversioned-command; repoquery
--repo=rawhide-source --whatrequires /usr/bin/python) | pkgname | sort | uniq
audit
bibus
bitfrost
blitz
claws-mail
coan
crun
distro-info
distro-info-data
dracut-modules-olpc
dtrx
gcc
gnome-python2-desktop
graphite2
grass
gwebsockets
htop
hyperscan
cherrytree
chocolate-doom
json4s
kcov
libclc
libtaskotron
liquidwar
maxima
mchange-commons
mingw-qt5-qtdeclarative
mingw-wine-gecko
mongo-c-driver
mozc
offlineimap
olpc-contents
olpc-os-builder
perl-Plack
planner
python-rospkg
qtwebkit
qt5-qtdeclarative
sbt
seamonkey
sugar-base
sugar-castle
sugar-deducto
sugar-flip
sugar-jukebox
sugar-kuku
sugar-measure
sugar-pippy
sugar-srilanka
sugar-starchart
sugar-toolkit
sugar-yupana
swift-lang
tarantool
termy-qt
twitter-twemoji-fonts
uboot-tools
udis86
vdsm
vte
wesnoth
wine-mono
0ad

$ (repoquery --repo=rawhide --whatrequires python; repoquery --repo=rawhide
--whatrequires python-unversioned-command; repoquery --repo=rawhide
--whatrequires /usr/bin/python) | pkgname | sort | uniq
gwebsockets
icaro
pyqt-mail-checker
qct
redhat-lsb-languages
resiprocate-turn-server-psql
sugar
sugar-toolkit
vdsm
vdsm-yajsonrpc

Note that there are packages that actually need any /usr/bin/python to build,
such as gcc. But I wouldn't bother with he buildtime deps much - either they
build with py3 or they won't.

The runtime deps bother me, because they need to be fixed or retired either way.
I'll happily retire them all (with big loud warnings before we do).

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Peter Robinson at 07/01/2019 - 09:21

On Fri, Jun 28, 2019 at 2:34 AM Miro Hrončok < ... at redhat dot com> wrote:
I fixed the following up:
uboot-tools
gnome-python2-desktop
gwebsockets
sugar
sugar-base
sugar-castle
sugar-deducto
sugar-flip
sugar-jukebox
sugar-kuku
sugar-measure
sugar-pippy
sugar-srilanka
sugar-starchart
sugar-toolkit
sugar-yupana

The following are FTB due to someone retiring Pyrex out from under
them without notifying me (there's been a number of py2 packages that
have had that happen) so they're going to take a bit longer.
bitfrost
dracut-modules-olpc
olpc-contents
olpc-os-builder

As much as you make some of your points above sound like it's
malicious packagers sometimes it's a lot more simple like "please stop
spamming me with pointless bugzilla updates and provide useful
information for over worked packagers", I thought most of the sugar
stuff had actually been fixed of the python-unversioned-command issues
but clearly some fell through the cracks, it's not intentional.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 07/01/2019 - 14:13

On 01. 07. 19 16:21, Peter Robinson wrote:
Thanks.

I've notified you at least 3 times:

<a href="https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedoraproject.org/thread/4LKL4NAQZOF6ETC62IKEP5KWGSI5PUNA/" title="https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedoraproject.org/thread/4LKL4NAQZOF6ETC62IKEP5KWGSI5PUNA/">https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedor...</a>
<a href="https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedoraproject.org/thread/23I5LPSAYL3OFR5UNEKFLZKPGCRBLVYC/" title="https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedoraproject.org/thread/23I5LPSAYL3OFR5UNEKFLZKPGCRBLVYC/">https://lists.fedoraproject.org/archives/list/devel- ... at lists dot fedor...</a>
<a href="https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject.org/thread/X77IOA2RE3KADMMVPISYSESA3KYJ2UFR/" title="https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject.org/thread/X77IOA2RE3KADMMVPISYSESA3KYJ2UFR/">https://lists.fedoraproject.org/archives/list/ ... at lists dot fedoraproject....</a>

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Peter Robinson at 07/02/2019 - 10:58

On Mon, Jul 1, 2019 at 9:02 PM Miro Hrončok < ... at redhat dot com> wrote:
Do you cc: affected people on those? I was on PTO and traveling in the
window of the messages you link there so they probably got lost in the
million other devel@ and related emails, it's easy enough for them to
get lost if you don't have an explicit notification it's something you
should be paying attention too.

Peter

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 07/02/2019 - 11:44

On 02. 07. 19 17:58, Peter Robinson wrote:
I bcc all the listed people. The lists don't like dozens of recipients.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By catafest at 07/02/2019 - 12:21

I already used Python 3.7.3 in Windows OS and is very good.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By King InuYasha at 06/27/2019 - 18:43

On Thu, Jun 27, 2019 at 7:41 PM Stephen John Smoogen < ... at gmail dot com> wrote:
We've been throwing errors on using /usr/bin/python in packaged code
since Fedora 28, so I don't think it's necessary to wait like that.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Adam Williamson at 06/26/2019 - 13:07

On Wed, 2019-06-26 at 13:57 -0400, Ben Cotton wrote:
Oh, man. I thought we'd decided against this in the past? I'm worried
about the cost/benefit ratio on such a change.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 05:32

On 26. 06. 19 20:07, Adam Williamson wrote:
We did. Circumstances changed.

What worries you do most about "the cost"?

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Adam Williamson at 06/27/2019 - 10:15

On Thu, 2019-06-27 at 12:32 +0200, Miro Hrončok wrote:
Out of interest, what circumstances?

I mean, generalized existential dread? :)

The most obvious is scripts with #!/usr/bin/python . OK, we can try and
find every single one in the distro and patch them (though I'm sure
some will get missed somehow) but there will certainly be ones that
*aren't part of the distro* that get bitten by this.

But yeah, I don't have an itemized list of things that are going to
break or anything. I wish I did!

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 10:33

On 27. 06. 19 17:15, Adam Williamson wrote:
Mostly date.

<a href="https://fedoraproject.org/wiki/Changes/Python_means_Python3#Benefit_to_Fedora" title="https://fedoraproject.org/wiki/Changes/Python_means_Python3#Benefit_to_Fedora">https://fedoraproject.org/wiki/Changes/Python_means_Python3#Benefit_to_F...</a>

"The name 'Python' will not refer to software that will be unmaintained upstream
for most of Fedora 31's lifetime and retired from Fedora 32."

Also, before, we have decided to not do this, as it was against the upstream
recommendation. That recommendation is changing now as Python 2 approaches EOL.

See <a href="https://github.com/python/peps/pull/989" title="https://github.com/python/peps/pull/989">https://github.com/python/peps/pull/989</a>

Yes, we've already done that.
<a href="https://fedoraproject.org/wiki/Changes/Move_usr_bin_python_into_separate_package" title="https://fedoraproject.org/wiki/Changes/Move_usr_bin_python_into_separate_package">https://fedoraproject.org/wiki/Changes/Move_usr_bin_python_into_separate...</a>
<a href="https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error" title="https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error">https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error</a>

There, you are correct. However, would we want "python" to mean Python 2
forever? Or do we want to phase things out slowly and make a designated point in
the future, where this is changed?

Note that users and sysadmins can change the meaning of "python", see:

<a href="https://fedoraproject.org/wiki/Changes/Python_means_Python3#User_Experience" title="https://fedoraproject.org/wiki/Changes/Python_means_Python3#User_Experience">https://fedoraproject.org/wiki/Changes/Python_means_Python3#User_Experience</a>

That would be nice.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Peter Robinson at 06/28/2019 - 03:22

On Thu, Jun 27, 2019 at 5:20 PM Miro Hrončok < ... at redhat dot com> wrote:
I suppose to me the big one with be pypi and what the expectation in
that community is around which version of python points to
/usr/bin/python, have they been running tests against the repositories
there and if it'll break things installed via pip.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/28/2019 - 03:26

On 28. 06. 19 10:22, Peter Robinson wrote:
In my experience pip installed packages generally handle shebangs well (e.g.
they set the shebang to the pip's interpreter). Of course there might be some
packages where this is not true, but the enormous adaptation of using Python
virtual environments IMHO battle tested this for years.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Adam Williamson at 06/27/2019 - 13:13

On Thu, 2019-06-27 at 17:33 +0200, Miro Hrončok wrote:
I don't honestly see the *harm* in this. That is what it has always
meant up until now, after all. That is probably what people expect it
to mean by this point. I always read "python" as "Python 2".

What is the benefit of this change, exactly?

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 13:31

On 27. 06. 19 20:13, Adam Williamson wrote:
The benefit is that people will eventually stop thinking this.

"python" (Python 2) will be unmaintained and dead couple weeks after Fedora 31
is released. As the Python maintainers, we think that "python" being Python 2
brings unfortunate expectations - that it is "the default Python".

We spent years and years trying to mark Python 2 as the "legacy" thing, but as
long as we invoke Python 2 when users run "python" we cannot really expect that
to happen. I believe that we should not run unsupported Python interpreter when
users run "python".

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Vitaly Zaitsev ... at 06/27/2019 - 09:02

On Thu, 27 Jun 2019 12:32:26 +0200

Wouldn't this mean that when python4 is released, there is a replay of
the python2 -> python3 transition experience?

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 09:09

On 27. 06. 19 16:02, stan via devel wrote:
No, we keep everything called python3, we just provide the "python" name.
With python2 -> python3, one of the problems was that everything was just called
"python" before. We are not proposing to start doing that again. All packages
still need to eb called python3-foo, ale package still need to use python3-foo
dependencies and all packages still need to invoke "python3" explicitly.

If Python 4 ever happens, it's going to be a problem one way or another, this
doesn't make it nay more complicated.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Ben Cotton at 06/27/2019 - 10:57

On Thu, Jun 27, 2019 at 10:10 AM Miro Hrončok < ... at redhat dot com> wrote:

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 11:04

On 27. 06. 19 17:57, Ben Cotton wrote:
It's easier for the users, especially beginner Python developers or data scientists.

$ python
bash: python: command not found

Oh right!

$ sudo dnf install python
No match for argument: python
Error: Unable to find a match

Oh well?

Fedora is equipped with fully operational, ready to use and develop on Python
interpreter, why not let people find it?

So I might ask: What's the benefit of not having an unversioned python at all?

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Ben Cotton at 06/27/2019 - 11:49

On Thu, Jun 27, 2019 at 12:06 PM Miro Hrončok < ... at redhat dot com> wrote:

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?UTF-8?B?TWlyb... at 06/27/2019 - 13:00

On 27. 06. 19 18:49, Ben Cotton wrote:
To be fair, upstream allows us to choose. If we decide that we want "python"
command not to exists, it's a valid choice.

As Python maintainers, we want to make it python3. If Fedora decides that
removing it is a better way, we have the ability to do that.

I'd argue that it brings more problems. Such as: Should there also be no "pip",
no "pytest", no "pylint"... command? Or should we switch those to Python 3, but
just have no "python" command? What happens if users do "dnf install python"?
Should they get Python 3 or nothing? etc.

Either way, we really **need** "python" to no longer be Python 2. There are
still people "out there" who call Python 2 the "default Python" because that's
what you get when you type "python".

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Dan Book at 06/27/2019 - 13:50

As an outsider to the Python community, not having any binary or package
that responds to the expected name "python" would be a disaster.

-Dan

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Ben Cotton at 06/27/2019 - 15:09

On Thu, Jun 27, 2019 at 2:52 PM Dan Book < ... at gmail dot com> wrote:

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Gerald Henriksen at 06/27/2019 - 18:48

Because it's not about what some python code expects, it about what
the human being using Fedora expects.

Nobody trying to compile a C program expects to have to use gcc9, they
just expect to type in gcc.

Similarly someone sitting down to use a tutorial to learn Python is
going to expect to type python and get something they can use,
particularly going forward when Python3 really just become Python as
Python2 fades from memory.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?iso-8859-1?q?... at 07/01/2019 - 12:59

Gerald Henriksen wrote:
Despite all its flaws, C is pretty good at backwards-compatibility. A
valid C99 program can be expected to work unchanged as C11. (A correct,
standard-compliant C program, that is. Not one that relies on undefined
behavior or similar craziness.) That is not so with Python. A perfectly
valid Python 2 program is not likely to work in a Python 3 interpreter,
unless it was intentionally written to be a polyglot.

Porting a Python 2 program to Python 3 is not like switching to another
version of GCC. It's more like converting a C program into C++ – and
even then, my gut feeling is that the minimal changes necessary to turn
a typical C program into valid C++ would be fewer changes per line than
the changes necessary to turn Python 2 into Python3.

Developers are supposed to compile C programs with gcc and C++ programs
with g++, just like Python 2 programs need to be interpreted with
python2, and Python 3 programs with python3. Changing /usr/bin/python
to point to /usr/bin/python3 is similar to making /usr/bin/gcc a link
to /usr/bin/g++. The Python folks are trying to completely replace one
programming language with another, and it's no wonder that it's going
slowly.

There may be good arguments for why /usr/bin/python should survive the
removal of Python 2, but this comparison to GCC isn't one.

Björn Persson

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By =?utf-8?q?Jos=C... at 07/01/2019 - 14:42

On Monday, 1 July 2019 18.59.42 WEST Björn Persson wrote:
My gut feeling was the same until I had to port code from python 2 to python
3. Requiring at least python 3.5 and using tools like modernize or futurize to
identify the parts where there could be problems reduced the work a lot.

Of course that good tests are a must since sometimes there are some issues
slipping under the radar. But on the whole the experience was easier than
expected and the resulting code has improved because of that.

Usual disclaimer: it depends on the code being ported and your mileage may
vary. :-)

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Panu Matilainen at 06/28/2019 - 03:59

On 6/28/19 2:48 AM, Gerald Henriksen wrote:
This.

On my home computer I was able to get rid of python-unversioned-command
and symlinked ~/bin/python to /usr/bin/python3, so whenever I need
python I get what I want and not the about-to-retire piece of history,
ah the sanity.

On my work laptop I still run into needing /usr/bin/python -> python2
every now and then, unfortunately. Eagerly waiting for the day when it
starts pointing to python3 instead.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By King InuYasha at 06/27/2019 - 16:34

On Thu, Jun 27, 2019 at 4:58 PM Ben Cotton < ... at redhat dot com> wrote:
The majority of the Python community writes Python 3 code pointing to
an unversioned shebang, even when it's Python 3 only. This is because
in everything *except* Linux distributions, the unversioned name has
already switched over. And OpenMandriva made the switch before us, and
that was not a terribly painful change like it was for Arch many years
earlier.

In fact, I would argue that the only mistake was in RHEL, where RHEL 8
did not ship with the default unversioned names that would point to
the Python 3 variants.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Michael Catanzaro at 06/27/2019 - 17:11

On Thu, Jun 27, 2019 at 4:34 PM, Neal Gompa < ... at gmail dot com> wrote:
Not in macOS.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Elliott Sales d... at 06/27/2019 - 18:03

On Thu, 27 Jun 2019 at 18:59, < ... at gnome dot org> wrote:
System macOS is still Python 2, but new developers are not encouraged
to use it. Any reasonable Python macOS programmer uses conda or
virtualenv (or a wrapper like pyenv, pipenv, etc.), and a Python 3
environment will set python->python3.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Charalampos Str... at 06/27/2019 - 16:24

Depends on what you mean by calling "python". The biggest portion of python code floating around would be compatible with either both python2 and 3, or only with python3, with a few exceptions.

Re: Fedora 31 System-Wide Change proposal: Python means Python3

By Zbigniew =?utf-... at 06/27/2019 - 15:59

On Thu, Jun 27, 2019 at 04:09:58PM -0400, Ben Cotton wrote:
While I'm wary of the proposed change, I think that this particular
avenue of criticism is misguided and we don't need to worry about
"future proofing".

Iiiiiiiif there's ever python4 (which isn't even on the horizon right
now), I think/hope that the way it is introduced will not repeat the
disastrous way that python3 was introduced, and the transition will be
more like the normal python 2.x→2.y and 3.x→3.y transitions, just with
a bigger set of changes. Python has the __future__ mechanism and
deprecation warnings, which can and should be used to phase in
changes. So I think we can safely assume that if python4.0 comes
around, we'll point the python symlink at it when we want it to be the
default, and we'll expect all packages in the distro to support
it. I.e. the python3.z→4.0 switch will not be substantially different
then the current 3.7→3.8 upgrade.

Zbyszek