DevHeads.net

Sphinx and xindy

I have been working on an update for coq 8.9.0 in Rawhide. A mock
build fails when building the documentation:

Latexmk: applying rule 'makeindex CoqRefMan.idx'...

The dependency on xindy comes from Sphinx. Take a look at
%{python3_sitelib}/sphinx/texinputs/latexmkrc_t, which is used to
create the latexmkrc for coq that refers to xindy.

As I understand it from looking through bugzilla, the texlive-xindy
package was dropped because it frequently failed to build.

So what is the right thing to do here? Try to debug the xindy
problems so the texlive-xindy package can be resurrected? Disable
xindy support in python-sphinx (if possible)?

Comments

Re: Sphinx and xindy

By =?UTF-8?B?TWlyb... at 03/14/2019 - 06:13

On 13. 03. 19 17:20, Jerry James wrote:
There should be a configuration option that disbales xindly. Does the
documentation build with it?

latex_use_xindy = False # put it in doc/conf.py or similar

Re: Sphinx and xindy

By Jerry James at 03/14/2019 - 22:45

On Thu, Mar 14, 2019 at 4:13 AM Miro Hrončok < ... at redhat dot com> wrote:
I've got to figure out where to put that option. Coq has its own
documentation builder, coqdoc, so this option will have to be inserted
into it somewhere, I guess. I'll try to figure it out tomorrow.

Re: Sphinx and xindy

By Jason L Tibbitts III at 03/14/2019 - 11:17

MH> There should be a configuration option that disbales xindly. Does
MH> the documentation build with it?

Since xindy isn't really something that can be relied upon, is it
possible (or reasonable) to do this globally in our sphinx packages?
Even when it was enabled, it was architecture-limited which would
force that limitation to propagate down to potentially anything that
used sphinx. (xindy is written in LISP and builds with clisp. There's
a certain irony in that Jerry is also a maintainer of clisp.)

I do think that the disabling of xindy was supposed to be only temporary
so I think it could be re-enabled, but having it off probably makes
texlive maintenance easier. The proper solution, I guess, is to fix
clisp to work on s390x and then fix whatever issues prevent xindy from
being enabled all the time. I think it would help to remove it from
texlive-base entirely and let it stand on its own. That way issues with
it wouldn't prevent texlive-base from building.

- J<

Re: Sphinx and xindy

By Jerry James at 03/14/2019 - 22:49

On Thu, Mar 14, 2019 at 9:18 AM Jason L Tibbitts III < ... at math dot uh.edu> wrote:
Yes, I am, and nobody ever mentioned xindy problems to me. There were
no bugs filed against clisp, no email messages to me, nothing. If I'd
known about these problems, I would have helped look into them.

The clisp issue with s390x has been on my TODO list for a long time.
I've been using most of my Fedora time to hunt down bugs that keep
other packages from building at all. I seem to be reaching the end of
that, at last, so I'll try to figure this out next.

Re: Sphinx and xindy

By Jerry James at 03/17/2019 - 23:54

On Thu, Mar 14, 2019 at 8:49 PM Jerry James < ... at gmail dot com> wrote:
This appears to be caused by the clisp code incorrectly pairing
register and volatile. Still, I find gcc's behavior surprising, which
is why I filed <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1689769" title="https://bugzilla.redhat.com/show_bug.cgi?id=1689769">https://bugzilla.redhat.com/show_bug.cgi?id=1689769</a>.
That may very well be closed as NOTABUG, but I would like the gcc
maintainers' input.

I'll work up a patch for clisp, and then we can try building xindy again.

Re: Sphinx and xindy

By Jerry James at 04/01/2019 - 23:36

On Sun, Mar 17, 2019 at 9:54 PM Jerry James < ... at gmail dot com> wrote:
That was, in fact, an s390x-specific gcc bug, now fixed in Rawhide.
Sadly, the clisp build is still failing in Rawhide, with failures in
the socket tests:

<a href="https://koji.fedoraproject.org/koji/taskinfo?taskID=33879684" title="https://koji.fedoraproject.org/koji/taskinfo?taskID=33879684">https://koji.fedoraproject.org/koji/taskinfo?taskID=33879684</a>

An obvious endianness problem, right? Except that the build
*succeeds* when I run it in mock on actual s390x hardware (thanks to
Dan for the account). This has been happening for a couple of days,
across a Rawhide compose.

So now I've got two Lisp implementations, gcl and clisp, that build
successfully in mock and fail to build in koji (for completely
different reasons). I'm traveling for a few days, so figuring this
out will have to wait until I get back. The lack of visibility into
koji is pretty frustrating, though.

Re: Sphinx and xindy

By Jerry James at 04/08/2019 - 10:57

On Mon, Apr 1, 2019 at 9:36 PM Jerry James < ... at gmail dot com> wrote:
The failing test attempts to bind to localhost:9090. Apparently
something on the s390x koji builders binds that port. The test is
arguably broken; it should iterate over successive port numbers until
if finds one that is available, rather than picking a single port
number and failing if that port is unavailable. Anyway, I "fixed" it
for now by choosing a different port number. I'll ask upstream to
really fix it.

So clisp has been built for all architectures in Rawhide now. I
cannot build it for F30 yet, because the version of gcc with the s390x
fix has not yet been submitted as an update. Regardless, can somebody
who knows how to build xindy try it in Rawhide and let me know if
there are problems doing so? Or give me instructions on how to build
it myself? Thanks,

Re: Sphinx and xindy

By Jerry James at 04/11/2019 - 23:04

On Mon, Apr 8, 2019 at 8:57 AM Jerry James < ... at gmail dot com> wrote:
Somebody out there has been involved with past attempts to build
xindy. Please, I would like to make progress on this so I can get
back to building the coq stack's new versions. Which package used to
contain xindy? How does one attempt to build it? I would like to see
if there are still build issues with the new clisp.

Thank you,

Re: Sphinx and xindy

By Jason L Tibbitts III at 04/17/2019 - 14:18

JJ> Somebody out there has been involved with past attempts to build
JJ> xindy. Please, I would like to make progress on this so I can get
JJ> back to building the coq stack's new versions. Which package used
JJ> to contain xindy?

It is/was part of texlive-base. Look at line 20 of the texlive-base
spec:

# Not ppc64, not s390x, not aarch64 due to lack of clisp
# code SIGSEGV's on armv7hl
%global xindy_arches empty

JJ> How does one attempt to build it?

Just set that define appropriately. It was disabled in
ac0adc86c6ee1f2559e4313f210b828778388999 because I guess there's some
sort of circular build dependency and I guess it never got turned back
on again. Before that it was set to:

%global xindy_arches %{ix86} x86_64

And before acf14dee9c90d6f8b775adc0c1c5151d7df28642 that included arm:

%global xindy_arches %{arm} %{ix86} x86_64

To go back further you have to go back to the texlive package before
-base was split out.

My suggestion? Package it as an entirely separate package to avoid any
kind of circular build dependency. Call it xindy or texlive-xindy; I've
no preference.

- J<

Re: Sphinx and xindy

By Jerry James at 04/15/2019 - 22:41

On Thu, Apr 11, 2019 at 9:04 PM Jerry James < ... at gmail dot com> wrote:
Anybody? Anybody? Bueller?

Re: Sphinx and xindy

By Dridi Boukelmoune at 04/10/2019 - 01:46

On Mon, Apr 8, 2019 at 4:58 PM Jerry James < ... at gmail dot com> wrote:
No, for that you should bind port zero instead to get a random one in
a race-free manner (e.g. parallel execution of multiple tests for
example).

Dridi

Re: Sphinx and xindy

By Jerry James at 04/11/2019 - 23:02

On Tue, Apr 9, 2019 at 11:47 PM Dridi Boukelmoune
<dridi. ... at gmail dot com> wrote:
Ah, thanks. I didn't know about the port zero trick. I'll suggest
that to upstream.

Re: Sphinx and xindy

By Brian C. Lane at 04/09/2019 - 18:35

On Mon, Apr 08, 2019 at 08:57:22AM -0600, Jerry James wrote:
FYI port 9090 is used by cockpit.

Re: Sphinx and xindy

By =?UTF-8?B?TWlyb... at 03/14/2019 - 11:47

On 14. 03. 19 16:17, Jason L Tibbitts III wrote:
I don't think that we build much TeX documentation in our packages.

Re: Sphinx and xindy

By Jerry James at 03/14/2019 - 22:51

On Thu, Mar 14, 2019 at 9:48 AM Miro Hrončok < ... at redhat dot com> wrote:
I think you mean that the packages you maintain don't build much TeX
documentation. :-) I tend a LOT of packages that use (La)TeX, either
directly or indirectly. I've got two packages, coq and gap, that have
extensive documentation systems that use TeX under the hood, so that
propagates down to every package that depends on them. In my world,
TeX is essential and extensively used.

Re: Sphinx and xindy

By =?UTF-8?B?TWlyb... at 03/15/2019 - 02:47

On 15. 03. 19 3:51, Jerry James wrote:
Well I meant TeX docs with Sphinx.
But obviously even that could be wrong, that's why I've used the words "think"
and "much".

If you think exploring disabling xindy globally in our sphinx package is worth
it, let me know.

Re: Sphinx and xindy

By Jerry James at 03/17/2019 - 23:55

On Fri, Mar 15, 2019 at 12:48 AM Miro Hrončok < ... at redhat dot com> wrote:
Ah, sorry for the misread. No, I think you are correct. Sphinx seems
to be used mostly to build HTML documentation.

Let's first see if we can fix it to be reliable.