DevHeads.net

plan to update brotli to 1.0.4 in Rawhide

I'm supposed to notify you a week in advanced, which for me puts it
at 2018-04-21. the following packages will need to be rebuilt once that
happens.

golang-github-dsnet-compress
httpd
php-pecl-http
python-httpbin
webkit2gtk3
woff2

So I just announce that I'm updating the package on the day I update it
right?

Comments

Re: plan to update brotli to 1.0.4 in Rawhide

By Dominik 'Rathan... at 04/14/2018 - 17:48

On Saturday, 14 April 2018 at 03:30, <a href="mailto: ... at pouar dot net"> ... at pouar dot net</a> wrote:
Have you rebuilt all dependent packages successfully beforehand?
If not, and there are API and/or ABI changes that require porting
to the new API, you'll be creating unexpected work for the maintainers
of the above packages. Please do test rebuilds in COPR if you haven't
already and try to fix any issues before actually doing the update.

Regards,
Dominik

Re: plan to update brotli to 1.0.4 in Rawhide

By Milan Crha at 04/16/2018 - 05:00

On Sun, 2018-04-15 at 00:48 +0200, Dominik 'Rathann' Mierzejewski
wrote:
Hi,
yes, you did, Travis. Thanks for it.

Well, I'd not call it unexpected. Once you announce API/ABI change it
is expected that at least a rebuild, or also a code change(s), will be
needed. The maintainer of the package which does that API/ABI change
can help with porting to the changes, either with a patch or at least
pointing to upstream guide with the port instructions to the new API,
of course, and it's highly welcome, but it's not always possible.

I'm not sure it's fair to expect everyone doing this. First of all,
Travis doesn't seem to have commit rights to any of the referenced
packages, according to [1], neither he's part of provenpackagers group,
thus his best effort starts with this announcement and can be followed
by a pull-request or bug filling with a proposed change for the other
packages, but even that is sometimes not doable. Second, most/many
package maintainers are volunteers, kind of proxy for upstream work
they may not have any influence on, they just provide something useful
in Fedora for others, to make it easier to use for them. It doesn't
matter how large the package is or how many dependencies it has, from
my point of view. Compare it with something huge, like gcc updates,
mass rebuild(s) due to it and all the work involved around that. That
couldn't be done that smoothly with a group of volunteers "over the
weekend".

I do not know Travis and his position in that what I imply above, I
only try to give my point of view on this. It's always nice to see the
help from the package maintainer, but I do not think you can force it
in general. Not talking that the dependencies can have available
upstream fix already and the other maintainer can be aware of it, thus
the work you want from Travis might be useless and duplicated. Maybe.
It's all about coordination and it starts with these announcements.

Bye,
Milan

[1] https://src.fedoraproject.org/rpms/$RPM

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/17/2018 - 14:00

So is someone else is going to do the checking if the reverse dependencies
work with the new version or do I have to do that?

Re: plan to update brotli to 1.0.4 in Rawhide

By Zbigniew =?utf-... at 04/18/2018 - 01:21

On Tue, Apr 17, 2018 at 02:00:42PM -0500, <a href="mailto: ... at pouar dot net"> ... at pouar dot net</a> wrote:
*Somebody* has to do it. If *you* can at least start doing that, that
would be great and will probably make the whole process smoother.
Milan's point is that we cannot say that you *have* to do it, because
you're just a volunteer.

If you have the time, I'd suggest building the updated brotli version
locally, installing it locally (including the -devel package), and
rebuilding all the dependent packages locally. All using 'fedpkg
local'.

If that works, that's great, and then you can tell other maintainers
that a simple rebuild is enough. If it does not work and changes to
those packages required, then maybe you want to delay the update of
brotli a bit and coordinate with the maintainers of dependent packages.

Zbyszek

Re: plan to update brotli to 1.0.4 in Rawhide

By Adam Williamson at 04/18/2018 - 14:28

On Wed, 2018-04-18 at 06:21 +0000, Zbigniew Jędrzejewski-Szmek wrote:
And I'm gonna keep pointing out, since everyone seems to be skipping
over it, that the update *may not bump the soname at all*. In which
case no rebuilds are needed. You should check that before checking
anything else.

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/18/2018 - 15:45

On Wed, 18 Apr 2018 12:28:48 -0700

Well the minor version and the soname of the library itself changed but the
major version and the sonames of the symlinks pointing to the library
remained the same. Also 2 fields in BrotliDistanceParams changed positions,
but I'm not sure whether this is exposed to other packages or not.

Re: plan to update brotli to 1.0.4 in Rawhide

By Michael Catanzaro at 04/18/2018 - 17:14

On Wed, Apr 18, 2018 at 3:45 PM, <a href="mailto: ... at pouar dot net"> ... at pouar dot net</a> wrote:
The first digit in the soname is what matters. E.g. I have on my
machine libbrotlidec.so.0.6.0 and libbrotlienc.so.0.6.0. If the first 0
changes to 1, then that signals an ABI break, and libraries that depend
on it need to be rebuilt. We call that a soname bump. If only the last
two digits change, then the update is supposed to be ABI-compatible,
and you don't need to worry about it.

Sometimes upstreams mess this up, though, so be careful.

Is the definition exposed in a public header? If so, then that would be
an ABI break, and a soname bump would be required. Looks like
BrotliDistanceParams is an implementation detail that's only declared
in a private header, so it's not part of the API and you probably don't
need to worry about it.

Re: plan to update brotli to 1.0.4 in Rawhide

By Adam Williamson at 04/18/2018 - 17:25

On Wed, 2018-04-18 at 17:14 -0500, <a href="mailto: ... at gnome dot org"> ... at gnome dot org</a> wrote:
This is the most common convention, but it's not invariably true. (Some
upstreams *intentionally* do not follow this versioning convention for
their sonames, note). What the package dependencies ultimately go by is
the soname embedded in the library file itself, which you can find with
objdump:

[adamw@adam systemd (f28 %)]$ objdump -p
/usr/lib64/libbrotlienc.so.1.0.1 | grep SONAME
SONAME libbrotlienc.so.1.0.1

as you can see, in the brotli package I have installed currently,
things are *not* as you suggest: the actual soname really is
'libbrotlienc.so.1.0.1'. It's not 'libbrotlienc.so.1'. We see this
reflected in the package's Provides:

[adamw@adam systemd (f28 %)]$ rpm -q --provides brotli
brotli = 1.0.1-3.fc28
brotli(x86-64) = 1.0.1-3.fc28
libbrotlicommon.so.1.0.1()(64bit)
libbrotlidec.so.1.0.1()(64bit)
libbrotlienc.so.1.0.1()(64bit)

those last three are the auto-generated soname-based provides, and we
can see it in the corresponding auto-generated requires:

[adamw@adam systemd (f28 %)]$ sudo dnf repoquery --whatrequires
"libbrotlicommon.so.1.0.1()(64bit)"
Last metadata expiration check: 0:00:00 ago on Wed 18 Apr 2018 03:20:44 PM PDT.
brotli-devel-0:1.0.1-3.fc28.x86_64
php-pecl-http-0:3.2.0~RC1-1.fc28.x86_64

Note however that that's with brotli-1.0.1-3.fc28.x86_64 . In Rawhide,
in brotli-1.0.3-1.fc29 , the sonames were changed to be just .so.1 ,
not .so.1.0.1 or .so.1.0.3 . So if the new 1.0.4 retains the '.so.1'
soname, it should not require any rebuilds.

The other part of this is that it's ultimately the upstream project's
responsibility to change the soname when it needs changing - i.e. when
the ABI has changed - and *not* change it when it *doesn't* need
changing. Our packaging system assumes that upstreams do this; if they
don't, it will give weird results, but if they don't (and downstream
packagers don't catch this and adjust for it somehow), weirdness can
result (like updates that change the ABI but not the soname, so
dependencies are satisfied but the apps suddenly don't actually work
any more, or updates that change the soname unnecessarily and require
us to do rebuilds that weren't really needed).

AFAICS it's in a private header, so I think the soname properly
remained the same and no rebuilds should be needed.

Re: plan to update brotli to 1.0.4 in Rawhide

By Michael Catanzaro at 04/18/2018 - 23:24

On Wed, Apr 18, 2018 at 5:25 PM, Adam Williamson
< ... at fedoraproject dot org> wrote:
Very interesting, thanks for the correction!

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/18/2018 - 17:43

the soname the reverse deps link to doesn't seemed to have changed as they
all link to the .so.1 soname, but I'm not sure how to check for "weirdness"
in these packages

Re: plan to update brotli to 1.0.4 in Rawhide

By Adam Williamson at 04/18/2018 - 17:53

On Wed, 2018-04-18 at 17:43 -0500, <a href="mailto: ... at pouar dot net"> ... at pouar dot net</a> wrote:
From what you've posted I don't think there's any problem, basically.
Just go ahead and send the new package to Rawhide and everything should
keep working. I promise not to yell if it doesn't. :P

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/19/2018 - 10:29

On Wed, 18 Apr 2018 15:53:06 -0700

Just did after I read this email, though I didn't say anything until now as
I didn't think to respond to this until now

Re: plan to update brotli to 1.0.4 in Rawhide

By Dominik 'Rathan... at 04/18/2018 - 16:06

On Wednesday, 18 April 2018 at 22:45, <a href="mailto: ... at pouar dot net"> ... at pouar dot net</a> wrote:
You can use rpmsodiff and the tools in libabigail package to check the ABI
and API differences.

Regards,
Dominik

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/18/2018 - 16:27

On Wed, 18 Apr 2018 23:06:57 +0200

I was using this
<a href="https://github.com/lvc/abi-compliance-checker" title="https://github.com/lvc/abi-compliance-checker">https://github.com/lvc/abi-compliance-checker</a>

Re: plan to update brotli to 1.0.4 in Rawhide

By Pouar at 04/15/2018 - 18:23

On Sun, 15 Apr 2018 00:48:23 +0200

Is there documentation or a wiki page on how to do that? This is my first
time going through this process.