DevHeads.net

Plans for Node.js 6.x

OK folks, it's Bad Decision Time.

Today, Node.js 6.0 was released. This is a significant ABI-breaking release,
which means there is no guarantee that existing modules will work with it at all.[1]

Better still, Node.js 5.x is only going to be supported until sometime this
summer, because they're aiming for the 6.x branch to become the new LTS in
October[2]. This puts us in quite a bind.

Fedora disallows major ABI changes in a stable release, so if Fedora 24 Final
ships with Node.js 5.x, we will be stuck with maintaining it until Fedora 24 EOL
(which is probably going to mean until approximately June 2017, or almost a full
year after upstream drops support). This means manually backporting any security
issues that come up without the benefit of a new version to rebase to and with
an increasing likelihood of the patches diverging from upstream.

On the flip side, the major ABI break is hitting us post-Beta Freeze for Fedora
24, which is a really terrible time to be switching to a new major version of a
language runtime (particularly since we don't actually know if any of the other
packages on the system will work with it at all).

We don't have any particularly good options here. A downgrade back to 4.x (which
will be supported until at least April 2018, well past F24 EOL) would be very
difficult at this point, since node modules may have updated to newer,
incompatible versions.

Furthermore, this came at a time where I was planning to write to the nodejs
list and let people know that due to my changing work responsibilities, I'm not
going to be able to be the primary maintainer of the main package any longer.
(I'd be able to swing the occasional minor- or point-release update, but
wrangling a major release won't be possible.)

I realize this is inopportune, but it's best if we figure out *immediately* how
we're going to handle this.

Options:
1) Downgrade back to 4.x, downgrading or dropping any modules in the collection
that don't run on that LTS version.
2) Stick with 5.x for the life of Fedora 24, handling security backports
ourselves once it hits EOL this summer.
3) Upgrade to 6.x, fixing or dropping any modules in the collection that don't
run on it yet.

I'll stick around to help with this major effort (since it's partly my fault
we're in this mess; I didn't realize that the release schedule for 5.x was so
poorly aligned with Fedora 24), but after that I'm going to need someone else to
step up and handle the primary maintenance.

I don't like any of the choices, but I'm going to suggest that option 3) may be
our best choice if we can manage it; we will want to be doing the same work for
Fedora 25/Rawhide anyway, so it's not wasted effort. Also, a lot of the node
modules in Fedora are only there because originally we needed them as
dependencies for npm. Since we recently switched to carrying the embedded npm
(and its rat's-nest of dependencies) inside the main nodejs package, we can
probably get away with breakage in a lot of the modules in the collection.

We may only need to focus in the short term on those packages that are required
by other parts of the Fedora Project (like nodejs-less, which is consumed in the
build process for many web apps).

Option 2) is the path of least resistance in the immediate short-term, but once
we run up against the upstream EOL, the maintenance burden could be prohibitive.
In theory, we could petition FESCo for permission to break ABI in the stable
release, but I wouldn't recommend it (and would probably be voting against it
were it to come from anywhere else; I'd abstain in this case due to conflict of
interest). Given that we know about the potential risk in advance, I doubt we'd
see much sympathy either. So we should realistically assume that if we choose
this option, someone is going to need to maintain the security backports (and it
will not be me, sorry).

As for Option 1)? I think someone with more knowledge of the individual modules
in Fedora (Tom Hughes? Jared Smith?) would need to figure out how many modules
would be broken if we downgraded. If it's sufficiently small, I suppose we could
epoch-bump nodejs and its virtual npm Provides: and go that route. I don't love
that we will effectively been playing yo-yo with the version in F24, but it
would be a solution...

Thoughts, other ideas? Please skip the rotten fruit; I'm on a diet.

[1]
<a href="https://github.com/nodejs/node/blob/master/CHANGELOG.md#2016-04-26-version-600-current-jasnell" title="https://github.com/nodejs/node/blob/master/CHANGELOG.md#2016-04-26-version-600-current-jasnell">https://github.com/nodejs/node/blob/master/CHANGELOG.md#2016-04-26-versi...</a>
[2] <a href="https://github.com/nodejs/LTS#lts_schedule" title="https://github.com/nodejs/LTS#lts_schedule">https://github.com/nodejs/LTS#lts_schedule</a>

Comments

Re: Plans for Node.js 6.x

By drago01 at 05/01/2016 - 11:59

On Wed, Apr 27, 2016 at 4:00 AM, Stephen Gallagher < ... at redhat dot com> wrote:
The whole thread is based on this wrong assumption. Quoting from the
update policy [1]:

"If upstream does not provide security fixes for a particular release,
and if backporting the fix would be impractical, then a package may be
rebased onto a version that upstream supports. The definition of
practicality is left to the judgment of FESCO and the packager. "

And from the discussion that seems to be the case here so I'd opt for

4) Leave 5.x in and once it becomes EOL and a security issues comes up
update to 6.x mid release. It would have had been tested in rawhide
(and possible COPR) in the meantime which would minimize the impact.

1: <a href="https://fedoraproject.org/wiki/Updates_Policy#Security_fixes" title="https://fedoraproject.org/wiki/Updates_Policy#Security_fixes">https://fedoraproject.org/wiki/Updates_Policy#Security_fixes</a>

Re: Plans for Node.js 6.x

By Stephen John Smoogen at 04/27/2016 - 09:54

On 26 April 2016 at 22:00, Stephen Gallagher < ... at redhat dot com> wrote:
4) Drop NodeJS from Fedora 24 altogether. If there isn't one already,
have a nodejs team built of people who are interested in it and are
committed to doing things like side builds and similar requirements.
They can then have a plan on what nodejs work should be done and what
plans they will align on. This would be similar to the perl/python and
other groups.. and makes sure that when someone bows out it doesn't
kill the entire stack until someone comes in to build the work again.

Re: Plans for Node.js 6.x

By Josh Boyer at 04/27/2016 - 10:00

On Wed, Apr 27, 2016 at 9:54 AM, Stephen John Smoogen < ... at gmail dot com> wrote:
That would be a pretty big regression considering it has been in
Fedora for a while. The user experience of needing nodejs and then
having to hunt for it after upgrade seems poor.

I don't disagree having a nodejs team would be a good idea, but I
think you're being a bit unfair to Stephen. He hasn't bowed out yet
and even a well intentioned nodejs team could have made the same
choices that led to this situation.

josh

Re: Plans for Node.js 6.x

By Stephen John Smoogen at 04/27/2016 - 10:16

On 27 April 2016 at 10:00, Josh Boyer < ... at fedoraproject dot org> wrote:
I apologize that I came out as being unfair to Stephen. I was hoping
that I was being even more fair as I don't see this as his fault or
problem to have solve by himself.

I was seeing a lot of people with solutions that required him to do
even more work and larger herculean tasks without anyone stepping up
with a "Hey what can I do to help you here" but a lot of "How about
you do this.."

To me Fedora should always be <a href="https://en.wikipedia.org/wiki/Stone_Soup" title="https://en.wikipedia.org/wiki/Stone_Soup">https://en.wikipedia.org/wiki/Stone_Soup</a>
and not <a href="https://en.wikipedia.org/wiki/The_Little_Red_Hen" title="https://en.wikipedia.org/wiki/The_Little_Red_Hen">https://en.wikipedia.org/wiki/The_Little_Red_Hen</a> . The
conversation to me was tending towards the latter and I wanted to make
sure Stephen had the freedom to say "ok that is enough." without
feeling like he has to burn himself out to 'save' a release.

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 10:15

On 04/27/2016 10:00 AM, Josh Boyer wrote:
We have that already. The Node.js SIG exists, but I've been acting as
coordinator for the base pieces. I'm going to be stepping down from that soon
(which is why I asked for someone to step up there), but there *are* people
doing plenty of other work (notably Tom Hughes and Jared Smith).

Yeah, not acceptable in my opinion.

For the record, we *did* have a team that made these choices. When I suggested
moving to Node 5 to avoid an issue we introduced with a version discrepancy
between upstream and Fedora's npm delivery, we failed to notice the maintenance
schedule incompatibility. So now we're trying to figure out how best to resolve it.

Re: Plans for Node.js 6.x

By Stephen John Smoogen at 04/27/2016 - 10:18

On 27 April 2016 at 10:15, Stephen Gallagher < ... at redhat dot com> wrote:
My apologies extend to Tom and Jared also. I did not do enough
research before posting and I came across in a way I did not intend at
all to any of you.

Re: Plans for Node.js 6.x

By King InuYasha at 04/27/2016 - 09:59

On Wed, Apr 27, 2016 at 9:54 AM, Stephen John Smoogen < ... at gmail dot com> wrote:
There's apparently a Node.js SIG[0], so I guess it'd be their job.
That said, Stephen was part of that SIG and it sounds like he's bowing
out. There's still a few other folks in there.

My main concern is with getting stuff like electron-based programs in
Fedora. Electron depends on Node.js to work, so having a reasonably
up-to-date stack is important if we want to be able to support those
applications.

[0]: <a href="https://fedoraproject.org/wiki/SIGs/Node.js" title="https://fedoraproject.org/wiki/SIGs/Node.js">https://fedoraproject.org/wiki/SIGs/Node.js</a>

Re: Plans for Node.js 6.x

By Denise Dumas at 04/27/2016 - 07:38

Sounds like a job for rhscl :-)
Denise

Re: Plans for Node.js 6.x

By Josh Boyer at 04/27/2016 - 08:54

On Wed, Apr 27, 2016 at 7:38 AM, Denise Dumas < ... at redhat dot com> wrote:
Maybe?

Having nodejs in an SCL (or eventually module) would certainly help
with versioning issues going forward. However, given Fedora has
nodejs in the base repository today, Stephen is still left with a hard
choice here. If his repoquery magic is correct then I think reverting
to 4.x at the base level for F24 is likely the right idea.

I do like the idea of having 5.x and 6.x in an SCL or module or COPR
(all somewhat variations on a theme) though.

josh

Re: Plans for Node.js 6.x

By Peter Robinson at 04/27/2016 - 09:10

On Wed, Apr 27, 2016 at 1:54 PM, Josh Boyer < ... at fedoraproject dot org> wrote:
Versioning maybe but not the "we're going to be shipping a major
version of nodejs that will be EOL half way through the F-24 cycle"
problem. Whether it's SCL format or standard package format it would
still be built against F-24 deps and we'd still have the issue here.
Sure there could be 4.x and 6.x along side each other but we'd still
have a EOL potentially vuln release hanging around.

Peter

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 09:20

On 04/27/2016 09:10 AM, Peter Robinson wrote:
I'm not sure that's actually a solvable problem. The only thing we can
realistically do is offer them the newer version so that they can migrate. If
they're still using an EOL release after we've announced that it's dead, that's
kind of their problem.

Yes, I know that differs with how we've done things in the distro in the past,
but that was (in many ways) due to the technical limitation of not having
migration mechanisms available in many cases.

Re: Plans for Node.js 6.x

By King InuYasha at 04/27/2016 - 08:59

On Wed, Apr 27, 2016 at 8:54 AM, Josh Boyer < ... at fedoraproject dot org> wrote:
Would it be possible to try a nodejs 6.x build of everything in a
side-tag or something? My understanding (based on the changelog) is
that things should generally work, as while the ABI broke, most of the
API remained the same.

Personally, I'm not a fan of the idea of using SCLs to support newer
environments in Fedora. I'd prefer if SCLs were used to support older
ones, with newer ones being the default.

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 09:09

On 04/27/2016 08:59 AM, Neal Gompa wrote:
Well, Rawhide will be moving to Node.js 6.x relatively soon and I think it's
probably safe to assume that a COPR will appear for running it on F24 if we
decide to do the downgrade (or stay on 5.x).

Well, I think the idea behind modularization is that we would build a module for
each of the versions and they would follow their own lifecycle and not
necessarily be tightly dependent on the base Fedora release. Thus there wouldn't
really be a "default" for anything that wasn't a component of the foundational
module (aka the "base" module).

Re: Plans for Node.js 6.x

By Tom Hughes at 04/27/2016 - 09:04

The actual set of packages that need to be rebuilt is actually very
small - just a dozen or so modules that have binary components.

Most modules are pure javascript, which don't need rebuilding and where
doing so will often prove nothing as we don't always have tests.

Well the question is, will the non-LTS node branch ever work for us
given then need to provide 13 months of support - how long is the
lifetime of the non-LTS branches?

Tom

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 09:18

On 04/27/2016 09:04 AM, Tom Hughes wrote:
From <a href="https://github.com/nodejs/LTS#lts_schedule" title="https://github.com/nodejs/LTS#lts_schedule">https://github.com/nodejs/LTS#lts_schedule</a> it looks like non-LTS branches
are maintained for about 9 months, while LTS releases are maintained for 30 months.

So you're probably right; going forward we should probably align Fedora with
whichever branch is (or will become) the LTS branch for its entire lifetime.

This still leaves us with a choice, because the 4.x branch is officially LTS and
the 6.x branch will become LTS in October. (In the early phase of the branch, it
won't break compatibility, but the set of features may grow; that gets locked in
when it goes LTS).

Re: Plans for Node.js 6.x

By Matthew Miller at 04/27/2016 - 14:20

On Wed, Apr 27, 2016 at 09:18:44AM -0400, Stephen Gallagher wrote:

And it looks like a) LTS releases will be even-numbered and b) come out
every October. That's kind of cutting it close with Fedora's late-October
release target, but maybe doable assuming they have prerelease/betas we
could put into _our_ beta, a la GNOME.

I suggest putting a comment explaining "please only use LTS releases"
(plus some of this explanation) right into the nodejs spec file.

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 15:08

On 04/27/2016 02:20 PM, Matthew Miller wrote:
Technically, the release up to the point where they become LTS is stable but not
LTS (I don't know what exactly that means...). For Fedora, this basically means
that we should be able to land the newest version in April and it will be
API-compatible for that whole time and locked down in October.

So that should be as safe as we can realistically make it.

Not a bad idea.

As it is, I think I've come to the conclusion that Fedora will drop back to the
4.x LTS for Fedora 24. Unless I hear some really compelling new information by
tomorrow morning, I'll work on that tomorrow.

Re: Plans for Node.js 6.x

By King InuYasha at 04/27/2016 - 09:25

On Wed, Apr 27, 2016 at 9:18 AM, Stephen Gallagher < ... at redhat dot com> wrote:
If the actual package set required to rebuild with nodejs 6 is really
only 14 packages, I would suggest we at least try to do it. Offering
nodejs 6 would be a rather nice feature in our cap, and when nodejs 8
is cut to be LTS, we can transition to that with the corresponding
Fedora release. It looks like Nodejs LTS releases will likely line up
with spring Fedora releases, so in the future, we could plan for that.

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 09:47

On 04/27/2016 09:25 AM, Neal Gompa wrote:
14 packages is the set that would be required to rebuild for Node.js 4.x. I have
no idea how much breakage would be involved with 6.x. You say "My understanding
(based on the changelog) is
that things should generally work, as while the ABI broke, most of the API
remained the same.", but I see dozens of "SEMVER-MAJOR" commits, many of which
are removing some API (a deprecated one in some cases, but I have no way of
knowing which of our packages would be affected).

As Tom mentioned, many of the npm modules in Fedora don't have working test
suites either, so a rebuild succeeding might not mean anything at all about
whether it would actually work.

At this point, I'm probably going to fall on the side of downgrading to 4.x
(which right now is scheduled to still be in maintenance mode for the entire
life of F24). I don't think there's enough time left in the Fedora schedule to
jump to 6.x without significant risk of breakage.

Re: Plans for Node.js 6.x

By Tom Hughes at 04/27/2016 - 04:19

Off the top of my head I'm not aware of anything that requires 5.x and
for the most part I think people try to support at least 4.x and 5.x at
the moment, and often earlier versions as well.

Tom

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 08:37

On 04/27/2016 04:19 AM, Tom Hughes wrote:
OK, I did some repoquery magic just now and came up with (unique-only):

nodejs(engine)
nodejs(engine) >= 0.1
nodejs(engine) >= 0.10
nodejs(engine) >= 0.10.0
nodejs(engine) >= 0.10.12
nodejs(engine) >= 0.10.15
nodejs(engine) >= 0.10.3
nodejs(engine) >= 0.10.36
nodejs(engine) >= 0.1.103
nodejs(engine) >= 0.12.0
nodejs(engine) >= 0.1.90
nodejs(engine) > 0.1.90
nodejs(engine) >= 0.2.0
nodejs(engine) >= 0.2.0-0
nodejs(engine) >= 0.2.4
nodejs(engine) >= 0.2.5
nodejs(engine) >= 0.3.0
nodejs(engine) >= 0.3.1
nodejs(engine) >= 0.3.6
nodejs(engine) >= 0.4
nodejs(engine) >= 0.4.
nodejs(engine) >= 0.4.0
nodejs(engine) >= 0.4.1
nodejs(engine) >= 0.4.2
nodejs(engine) >= 0.4.7
nodejs(engine) >= 0.4.9
nodejs(engine) >= 0.6
nodejs(engine) >= 0.6.0
nodejs(engine) >= 0.6.19
nodejs(engine) >= 0.6.3
nodejs(engine) >= 0.6.6
nodejs(engine) >= 0.8
nodejs(engine) >= 0.8.
nodejs(engine) >= 0.8.0
nodejs(engine) >= 0.8.19
nodejs(engine) >= 0.9.0
nodejs(engine) >= 4
nodejs(engine) >= 4.0.0

So according to this, we have nothing in the package collection that is known to
require only 5.x or later. So that's a point in favor of the 4.x downgrade approach.

I don't love the idea of regressing the versions post-Beta, but it's starting to
look like the least-risky approach. We really have no idea what is going to be
broken by 6.0 and I don't want to stick some poor volunteer with maintaining
backports of a dead upstream release.

Re: Plans for Node.js 6.x

By Peter Robinson at 04/27/2016 - 09:07

When you went from 4.x -> 5.x the only stuff that appeared to need to
be remixed was the binary archful rpms.

P

Re: Plans for Node.js 6.x

By Stephen Gallagher at 04/27/2016 - 09:11

On 04/27/2016 09:07 AM, Peter Robinson wrote:
Yeah, but as that was some time ago, there was certainly an opportunity for new
versions that were 5.x+ specific to have landed in the repository. Last night I
was too tired to think of the repoquery check, else I probably would have done
it before sending the first mail here.

If we downgrade, we'll need to respin the archful ones, but those are very few
(Bodhi tells me it was 14 packages) and they should be trivial rebuilds.

Re: Plans for Node.js 6.x

By Chris Murphy at 04/27/2016 - 00:14

On Tue, Apr 26, 2016 at 8:00 PM, Stephen Gallagher < ... at redhat dot com> wrote:
Ok well they all sound like a shade of terrible for one reason or
another, so I'll suggest one maybe more terrible so that in comparison
these three sound better.

Ship 5.x and basically plan on abandoning it when upstream does, i.e.
no plan at all, in advance, to do security fixes beyond upstream's
date. If they happen, it's a bonus. Kinda reminds me of this Red
October quote from Cpt Ramius: "When he reached the New World, Cortez
burned his ships. As a result his men were well motivated."

Meanwhile 6.x goes into copr now and can "ship" voluntarily at anytime
in the life of Fedora 24.

Include the plan in the release notes and common bugs, and ask
volunteers to point this out in particular on upstream's EOL day for
5.x as a friendly reminder on the usual forums and such.

Re: Plans for Node.js 6.x

By James Hogarth at 04/27/2016 - 04:03

On 27 Apr 2016 05:15, "Chris Murphy" < ... at colorremedies dot com> wrote:
It's an interesting proposition, though I dislike the idea of abandoning a
tech in Fedora during the supported lifespan of the distro. It seems like
it'd set a bad precedent.

It's certainly something we should not entertain without FESCO approval
though at any rate I'd suggest.

Gut feels like the 6.X build attempt, except excessively late in the cycle,
would seem the best bad decision if we didn't get FESCO clearance in the
breakage of a node 6 release during the Fedora 24 lifecycle.

Re: Plans for Node.js 6.x

By King InuYasha at 04/26/2016 - 22:36

On Tue, Apr 26, 2016 at 10:00 PM, Stephen Gallagher < ... at redhat dot com> wrote:
Oh dear... When it starts like this, this is *not* going to be an easy
decision...

That said, here's my thoughts on the options.

I don't particularly like this idea, especially as it would
essentially regress the Nodejs stack. I don't exactly know how many
modules depend on nodejs 5+ functionality, but I know that there are
plenty of interesting applications that do require it that I imagine
people would like to package in Fedora. Generally speaking, I'd like
to see Fedora be *more* attractive to developers of all stripes,
including Nodejs application developers.

This sounds... obnoxious for us, especially for a stack that is in the
early stages of support in Fedora. I can understand doing something
like this with Python, Ruby, or Go, but Nodejs upstream doesn't make
this particular option very easy, and I don't think anyone relishes in
the idea of cherrypicking the Nodejs codebase for fixes to backport.
Heaven forbid we'd need to modify the patches to actually backport
them to Nodejs 5.x. We're not Red Hat, and I imagine most of us simply
don't have the necessary expertise to pull that off.

This option is the least awful in my view, as I would expect that
backward compatibility in Nodejs should be good overall, even if there
would be minor breakages here and there on an API level. Obviously the
whole Nodejs stack would need rebuilding, which would probably
necessitate an f24-nodejs6 tag where everything gets built, and then
finally merged in (either via override or via bodhi). My major concern
with this is that I'm not sure how the process would actually *work*
for doing this. I can't recall a time we've ever done it before, but I
honestly don't see a better way to deal with this in such a way that
would not leave an unpleasant taste to potential new Fedora Nodejs
developers.

A plus on our side is that the effort wouldn't be wasted, since we're
doing it for F25/Rawhide anyway, as you pointed out. We just happen to
be doing it much earlier than we planned.