DevHeads.net

python-django update to Django-1.6

Hey,

recently, I saw a few requests to update python-django to Django-1.6,
the corresponding bug is [1].

As there are quite a few changes, I'd expect this update to be harmful,
at least
- python-django-openstack-auth
- openstack-dashboard

will break, and won't even build any more (because they also execute
sanity checks during build).

So, the current plan is, to fix both packages upstream and then to
update python-django to Django 1.6 in rawhide. I'd expect this to happen
within the next two weeks and I'd update python-django to Django-1.6
around Dec 16th.

Because of bad timing, we won't have Django-1.6 in f20.

Matthias

[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1027766" title="https://bugzilla.redhat.com/show_bug.cgi?id=1027766">https://bugzilla.redhat.com/show_bug.cgi?id=1027766</a>

Comments

Re: python-django update to Django-1.6

By Pierre-Yves at 11/26/2013 - 09:41

On Mon, Nov 25, 2013 at 09:07:55AM +0100, Matthias Runge wrote:
Just an idea, but what about providing Django 1.6 via copr for F{20,19}?
That might also help testing current apps against the new Django.

Pierre

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/20/2014 - 15:19

On 11/26/2013 08:41 AM, Pierre-Yves Chibon wrote:

Just to bring this thread back to life, we're getting to a point where
support for Django 1.6 is becoming more and more necessary. Is there
an ETA on its inclusion in Rawhide or COPR?

Re: python-django update to Django-1.6

By Matthias Runge at 02/21/2014 - 03:48

Whah, thank you!

There's a Django-1.6 build here in copr[1], and I'd like to push an
update to rawhide in about 14 days.

Any feedback is appreciated!

What about pushing Django-1.6 to epel7, too?

[1] <a href="https://copr.fedoraproject.org/coprs/mrunge/Django-1.6/" title="https://copr.fedoraproject.org/coprs/mrunge/Django-1.6/">https://copr.fedoraproject.org/coprs/mrunge/Django-1.6/</a>

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/21/2014 - 08:57

On 02/21/2014 02:48 AM, Matthias Runge wrote:
Fantastic, Matthias!

I'll go about getting Review Board 2.0 beta 3 built against your
Django 1.6 COPR later today so we can test things out. And yes, we
absolutely want Django 1.6 in EPEL 7.

Have we sorted out how exactly we want to build this in Fedora? I'm
still in favor of killing off the python-django package in favor of
python-django16 and python-django15.

Re: python-django update to Django-1.6

By Matthias Runge at 02/21/2014 - 14:37

On Fri, Feb 21, 2014 at 07:57:34AM -0500, Stephen Gallagher wrote:
We haven't sorted this yet. Still I'd prefer a kind of rolling version
(python-django) as the most recent version. That way, at least packages
working with the latest version would get upgrades automatically.

If we have a better plan, i.e if reviewboard requires Django-1.5 instead
of Django-1.4, we need to make sure, Django-1.4 gets replaced by
Django-1.5.

A Django-package should provide e.g Django = %{version}
and a package requiring series 1.4.x should
require Django >= 1.4, Django < 1.5
Then ideally the package providing Django-1.5 would obsolete Django-1.4
to get that replaced, the same for later packages.

Does that sound sane, does it make sense? I'd still would like to test
out, if that would lead to selecting the right Django package, instead
of just installing the latest version.

Currently, Django is named Django in EPEL6, python-djangp in EPEL7 and
in Fedora. It would make sense, to unify all the packages; that
shouldn't be a big problem at all.

Matthias

Re: python-django update to Django-1.6

By John.Florian at 02/21/2014 - 09:40

I too would much prefer this approach. For somebody like me who wants to
maintain company-private packages based on django, this affords more
flexibility. I realize it may always mean more packaging work to keep
several python-djangoXYs in the distro, but it makes for a less rigid
coupling between the OS and what framework version you need to use.

Re: python-django update to Django-1.6

By Matthias Runge at 02/21/2014 - 14:11

On Fri, Feb 21, 2014 at 08:40:52AM -0500, <a href="mailto:John. ... at dart dot biz">John. ... at dart dot biz</a> wrote:

Re: python-django update to Django-1.6

By John.Florian at 02/21/2014 - 15:37

Oh, yes I see that now. I was wishing for a python2/python3 like
environement where both could coexist, but clearly would break all my
imports. I keep seeing more and more reason to adopt the virtualenv
approach, but I'd rather stick with the Fedora packaging guidelines to
avoid bundling, even for private projects. It just seems so much more
sensible, yet it does lead the "jump when we say jump" syndrome. Still,
I'd rather deal with the upgrade conflict you mention than to be pinned to
an older Fedora release. It's always a fragile balancing act choosing
between what needs to be new and what cannot yet be new.

Re: python-django update to Django-1.6

By Simo Sorce at 02/21/2014 - 11:36

On Fri, 2014-02-21 at 08:40 -0500, <a href="mailto:John. ... at dart dot biz">John. ... at dart dot biz</a> wrote:
+1 I still have an application that is slowly moving to 1.5 but not
there yet and it is painful to have to keep an older Fedora Version
running just because of that.

Simo.

Re: python-django update to Django-1.6

By Matthias Runge at 02/21/2014 - 14:14

On Fri, Feb 21, 2014 at 10:36:24AM -0500, Simo Sorce wrote:
Matthias

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/21/2014 - 14:28

On 02/21/2014 01:14 PM, Matthias Runge wrote:

My suggestion would actually be that Fedora releases should ship ONLY
with the latest supported upstream version and should be allowed to
pick up the next one during its supported lifecycle.

So for F21, we'd ship with only Django 1.6 support and would pick up
1.7 when it arrives. The problem with shipping F21 with 1.5 and 1.6
support is that when 1.7 lands (and upstream drops all support for 1.5
at that time), we're stuck with only two choices:

1) Attempt to assume the maintenance burden on the abandoned branch.
2) Retire it from Fedora and strand anyone who has been using it.

Neither of these are good choices.

Upstream Django has a nine-month release cycle, meaning that each
version is only supported for 18 months. This is perfectly acceptable
for Fedora, as long as we don't ship with a version that's already
into its 17th month...

Now, EPEL on the other hand gets even more troubling, since it has a
much longer lifecycle...

One other approach we might consider (though this is not currently an
FPC-approved solution) would be to package Django as a software
collection and all Django apps would depend on the appropriate
collection. Since the 1.5 and 1.6 collections could coexist on the
system, when an app updates to support the new one, it needs only
change its Requires: to use the newer Django collection and it should
Just Work(TM).

Now, that's forbidden by policy at this point, but maybe we could at
least experiment with this in a COPR repository for the time being. It
would be nice to be able to come to the FPC with a working setup and
ask them to bless it for us, rather than presenting them a problem
statement and hoping that they can find a consensus.

Re: python-django update to Django-1.6

By Simo Sorce at 02/21/2014 - 14:58

This may not be possible, as it depends how long after upstream release
fedora is cut. In django case there is a long tail of applications that
needs porting so if you force 'lastest' you would end up breaking a
number of packages that do not support latest yet.

Honestly I do not think we have good choices. Upstream simply moves too
fast and causes all these issues to start with. We can only try to do
damage control.

It would be perfectly acceptable if the whole ecosystem moved at that
speed, but that is not the case, which is why I find django's policy
annoying.

I think django should be moved completely out of the base distro and be
only a collection, keeping it in the distro is painful and never
satisfies everyone.

Sounds like a decent plan :)

Simo.

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/21/2014 - 15:06

On 02/21/2014 01:58 PM, Simo Sorce wrote:

I'm having a parallel conversation about this with Toshio on
#fedora-devel right now. He believes it may be possible to get Django
to be parallel-installable on the base system without SCLs and is
running some tests. If he can make this work, that would make our
lives a lot easier. More to come, stay tuned...

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/21/2014 - 15:41

On 02/21/2014 02:06 PM, Stephen Gallagher wrote:
Ok, so it turns out that Python Eggs are a lot smarter than I gave
them credit for. If you turn your attention to
<a href="http://fedoraproject.org/wiki/Packaging:Python_Eggs" title="http://fedoraproject.org/wiki/Packaging:Python_Eggs">http://fedoraproject.org/wiki/Packaging:Python_Eggs</a>, you will find
that it describes quite well how to modify a Python compat package
(such as python-django14) to be parallel-installable with the newer
package.

Toshio has been testing this implementation with ReviewBoard 1.7.21
(Django 1.4) and ReviewBoard 2.0beta2 (Django 1.5) this afternoon and
so far it appears to work properly, with both python-django14 and
python-django installed on the same system.

We need to do some more testing to be certain, but it seems this may
be the easy way forward. Hooray!

Re: python-django update to Django-1.6

By Stephen Gallagher at 02/21/2014 - 17:31

On 02/21/2014 02:41 PM, Stephen Gallagher wrote:

So, as it turns out, this seems to be working flawlessly!

I had to make two minor changes to the ReviewBoard package to support
the parallel-installable version of python-django14. One I have
submitted back upstream (<a href="https://reviews.reviewboard.org/r/5520/diff/" title="https://reviews.reviewboard.org/r/5520/diff/">https://reviews.reviewboard.org/r/5520/diff/</a>)
and the other was Fedora-specific (I needed to be less total about the
modifications I made to the requires.txt in the ReviewBoard python egg).

I suspect we'll need to make similar modifications to other Django
apps in Fedora to support this, but it's very low-impact, so I'd
suggest that this is clearly the approach we want to take.

Toshio is going to send out the diff he created of the python-django14
package to this list as well.

Re: python-django update to Django-1.6

By Toshio Kuratomi at 02/21/2014 - 17:29

On Fri, Feb 21, 2014 at 02:41:31PM -0500, Stephen Gallagher wrote:
<a href="http://fedoraproject.org/wiki/Packaging:Python_Eggs#Multiple_Versions" title="http://fedoraproject.org/wiki/Packaging:Python_Eggs#Multiple_Versions">http://fedoraproject.org/wiki/Packaging:Python_Eggs#Multiple_Versions</a> has
the information but feel free to ping me if you have questions. In
particular, wsgi scripts seem to need a slight variant of the
__requires__. setuptools is looking for __requires__ in the __main__ module
(the toplevel of the script that's executed). With wsgi, that's not the
.wsgi script, there's another level on top of it. So you have to do
something like this in the wsgi script:

import __main__
__main__.__requires__ = ['Django >= 1.4, < 1.5']
import pkg_resources

Usually you don't have to specify Django specifically in the __requires__
either. If you have a python application/module that specifies the version
of Django that you need, then you can use that instead. For instance,
Reviewboard specifies that it requires Django > 1.4.10, < 1.5 in the
install_requires section of its setup.py. That ends up in the
ReviewBoard-1.7.21-py2.7.egg-info/requires.txt file. So you can do this:
import __main__
__main__.__requires__ = ['ReviewBoard']
import pkg_resources

and pkg_resources will find the versioned Django dependency in the
Reviewboard egg-info and use that to find the correct Django version.

Feel free to ping me on IRC or email me if you need more information or
encounter one of setuptools'/pkg_resources' crazy corner cases.
-Toshio

Re: python-django update to Django-1.6

By Matthias Runge at 02/26/2014 - 08:59

On Fri, Feb 21, 2014 at 01:29:54PM -0800, Toshio Kuratomi wrote:
Thank you, Stephen and Toshio, it's
very much appreciated!

In between, I submitted a request for review[1]
for a python-django15 package. I'd intend to change
the python-django14 package based on Toshios patch
as well, so that we'll have sooner or later
2-3 Django versions installable in parallel.

Best,
Matthias

[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1070230" title="https://bugzilla.redhat.com/show_bug.cgi?id=1070230">https://bugzilla.redhat.com/show_bug.cgi?id=1070230</a>

Update to Django-1.6

By Matthias Runge at 03/18/2014 - 08:11

Hey there,

now that we can have python-django15 in f21, I'll upgrade
python-django to Django-1.6 in a week.

Any interest, to have this on F20, too?

python-django15 is installable in parallel, no penguin will be
harmed, but the upgrade to Django-1.6 might hurt one or the other.

Matthias

Re: Update to Django-1.6

By John.Florian at 03/19/2014 - 08:36

<a href="mailto:devel- ... at lists dot fedoraproject.org">devel- ... at lists dot fedoraproject.org</a> wrote on 03/18/2014 08:11:04:

I'd like to see 1.6 for F20. That would let me remove some hacks (in
private code).

Re: Update to Django-1.6

By Stephen Gallagher at 03/18/2014 - 08:17

On 03/18/2014 08:11 AM, Matthias Runge wrote:
Woohoo!

Yes, absolutely. With Django 1.4 no longer maintained upstream and
Fedora 20 on an extended cycle, we should definitely be supporting
both 1.5 and 1.6 on it.

This is the best part. :)

When this goes to updates-testing for F20, please make a
devel-announce post describing what changes that Django package owners
should make in order to keep their package on Django 1.5 or older if
they need to.

Update to Django-1.6, even for f20

By Matthias Runge at 03/26/2014 - 07:44

On Tue, Mar 18, 2014 at 08:17:36AM -0400, Stephen Gallagher wrote:
Earlier today, I've built Django-1.6 for Rawhide and for F20 as well and
submitted the upgrade to f20-testing via bodhi[1]. Please test
and leave karma there.

If you're a packager and your package requires an older Django version,
please change your package to require python-django15 and add

from pkg_resources import require
require('Django>=1.5,<1.6')

to appropriate location(s) in your package. There is a blog post[2] on
how to consume older versions of Django. Even if the update to
Django-1.6 won't make it to stable, it should be safe to change your
package anyways, as older Django versions will be retired sooner or
later.

If you require help, please contact me off list.

Thanks,
Matthias

[1] <a href="https://admin.fedoraproject.org/updates/python-django-1.6.2-1.fc20" title="https://admin.fedoraproject.org/updates/python-django-1.6.2-1.fc20">https://admin.fedoraproject.org/updates/python-django-1.6.2-1.fc20</a>
[2] <a href="https://www.matthias-runge.de/2014/03/26/fedora-django-1-6/" title="https://www.matthias-runge.de/2014/03/26/fedora-django-1-6/">https://www.matthias-runge.de/2014/03/26/fedora-django-1-6/</a>

Re: python-django update to Django-1.6

By John.Florian at 02/21/2014 - 15:46

I love you guys! It's always a treat to read how smart folks take
something and just make it better like this. I wish I had time to
contribute more, but just wanted to say thanks to all who are always
working on these kind of goals. It's my favorite thing about Fedora and
FOSS in general.

Re: python-django update to Django-1.6

By Matthias Runge at 02/21/2014 - 14:55

On Fri, Feb 21, 2014 at 01:28:54PM -0500, Stephen Gallagher wrote:
Hmm, looking at larger Django applications included in Fedora.

Askbot: still requires exclusively Django-1.4, does not work with later
versions.
OpenStack-Dashboard: supports Django-1.4, 1.5, Django-1.6 support is
in the development branch, release will be in April
ReviewBoard: you already mentioned, Django-1.6 support is in
preparation.

I'm sure, I forgot something important.

Ideally, we'd provide the same versions on EPEL and on Fedora. If we'd
try to prevent breaking older applications, Django packages have to
carry the version in their name. At least, this enables us to pick
new upstream releases, when they come out, without breaking anything.

Re: python-django update to Django-1.6

By Adam Williamson at 02/24/2014 - 19:13

On 2014-02-21 10:55, Matthias Runge wrote:
I wouldn't block too hard on askbot in Fedora. We (me, nirik, and some
other people I've forgotten) were looking at it recently - in the
context of unbundling tinymce, or something - and it's basically DOA so
far as Fedora goes. We strongly suspect the package hasn't worked on
Fedora for years (or, possibly, ever). The current F20 and Rawhide
packages at least certainly can't possibly really work (unless someone's
fixed an awful lot of stuff in the last three weeks or so).

The package exists to back ask.fedoraproject.org , basically (that, of
course, runs on EL6). I doubt there's any other live askbot deployment
using the Fedora/EPEL packages.

Re: python-django update to Django-1.6

By Stephen Gallagher at 11/25/2013 - 12:24

On 11/25/2013 03:07 AM, Matthias Runge wrote:
This is kind of why I keep coming back to: "Why do we have
python-django at all?" I don't really see any reason why we shouldn't
kill off the python-django package and just carry 'python-django15'
and 'python-django16' packages with a conflict.

The number of incompatibilities between releases is such that I don't
think we really want to be forcing upgrades on other packages at all.
We should just be carrying whichever two versions are supported by
upstream at any given time. Upstream is very good about maintaining
bugfixes and security fixes in both supported streams.

Re: python-django update to Django-1.6

By Simo Sorce at 11/25/2013 - 13:51

+1 by changing version the current way, the only ting we can guarantee
is a lot of broken packages all the time.

Simo.

Re: python-django update to Django-1.6

By Matthias Runge at 11/26/2013 - 07:02

On 11/25/2013 06:51 PM, Simo Sorce wrote:

From my experience, it was just a pain to have python-django14 and
python-django[1]. Introducing one or two other packages python-django15
and python-django16 will make it more difficult for users to update
django. How should packages require Django? Just require python-django?
Sadly, yum can not handle that properly[1].

When dropping python-django as provides/requires, we'd have the
situation packages will require a specific version. That's rather
unfortunate, because combination of packages requiring some other
python-django-foo package might require a different django version.

At least for OpenStack Horizon I can say, we're up to fix compatibility
issues with Django-1.6 upstream.

Matthias

[1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=978647" title="https://bugzilla.redhat.com/show_bug.cgi?id=978647">https://bugzilla.redhat.com/show_bug.cgi?id=978647</a>

Re: python-django update to Django-1.6

By Simo Sorce at 11/26/2013 - 09:34

On Tue, 2013-11-26 at 12:02 +0100, Matthias Runge wrote:
Packages should require the latest version they work with.
If some package is really awesome and supports multiple versions I guess
it could support a generic python-django.

It's ok if 2 packages become incompatible this way, they wouldn't work
anyway with the wrong version of django.

Simo.

Re: python-django update to Django-1.6

By Stephen Gallagher at 11/26/2013 - 09:59

On 11/26/2013 08:34 AM, Simo Sorce wrote:

I think Simo has the right idea here. We should drop the standard
"python-django" package at this point and instead have python-django15
and python-django16. Each of those packages should add a virtual
Provides: and Obsoletes: for python-django.

Existing packages with a non-strict version will then default to
upgrading to the absolute latest version (python-django16). If that's
not acceptable to their project, they'll need to release a new update
with 'Requires: python-django15' and things should go back to normal.
In the future, if they update so they work with both
currently-available versions, they can go back to 'Requires:
python-django' and will then work with whichever version the user has
on the system (such as for another project).

Yes, it slightly increases the packager work, but it should give a
better experience for the user... to a point.

Since Django 1.5 and 1.6 cannot presently co-exist on the system,
they'll need to have an explicit Conflicts:. This does mean that users
will have an issue if they end up pulling Django 1.6 as part of an
upgrade and then try to install a package that Requires:
python-django15. We can't automatically remove python-django16, so the
user will have to know to do this manually.

Re: python-django update to Django-1.6

By Simo Sorce at 11/26/2013 - 10:31

One issue to resolve is how to upgrade to the next version if you have
python-django15 and F22 has python-django16 and python-django17, perhaps
some clever way of making python-django16 obsolete (instead of conflict)
python-django15 once 1.5 is pushed out of the new distro version ?

Simo.

Re: python-django update to Django-1.6

By Stephen Gallagher at 11/26/2013 - 13:57

On 11/26/2013 09:31 AM, Simo Sorce wrote:

Obsoletes: wouldn't really be appropriate here, because that implies
that it's a complete replacement of the old version. That said, it
still might be the best of a bad situation...

I still think that the safest approach is to require that packagers of
Django apps keep their dependencies set appropriately.

So if they know they support 1.5 and 1.6:

Requires: python-django >= 1.5
Conflicts: python-django >= 1.7

And if they only support one, they should
Requires: python-django15