DevHeads.net

Modularity tooling intro?

Hi everyone,

I have a bunch of packages in a local repo built for various versions
of Fedora and CentOS, and am looking to build some of them for EL-8.
Clearly the way to go for an EL-8 add-on repo is to build modules, so
that it what I'd like to do. However, I want to do it on my own
infrastructure and hence use the lower-level tooling such as
mbs-manager/mock and local git repos rather than fedpkg/koji/dist-git.

There's quite a bit of documentation around about building modules but
what I've found seems to be either out of date (e.g. uses "mbs-build",
which no longer exists) or is based on fedpkg, which abstracts away the
lower-level operation and is geared towards Fedora infrastructure.

Is there some documentation somewhere that covers getting started with
the tools like mbs-manager?

I thought this workshop looked like a good starting point:
<a href="https://github.com/fedora-modularity/workshop" title="https://github.com/fedora-modularity/workshop">https://github.com/fedora-modularity/workshop</a>

When I discovered that "mbs-build" no longer exists, I tried using
"mbs-manager" but clearly I'm missing something:

$ mbs-manager build_module_locally --file test-module.yaml -s platform:f30
2019-05-30 09:51:04,712 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
2019-05-30 09:51:04,863 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2019-05-30 09:51:05,577 - MainThread - MBS.utils.submit - DEBUG - Submitted normal module build for test-module:f30:20190530085105
Traceback (most recent call last):
File "/usr/bin/mbs-manager", line 11, in <module>
load_entry_point('module-build-service==2.19.1', 'console_scripts', 'mbs-manager')()
File "/usr/lib/python3.7/site-packages/module_build_service/manage.py", line 251, in manager_wrapper
manager.run()
File "/usr/lib/python3.7/site-packages/flask_script/__init__.py", line 417, in run
result = self.handle(argv[0], argv[1:])
File "/usr/lib/python3.7/site-packages/flask_script/__init__.py", line 386, in handle
res = handle(*args, **config)
File "/usr/lib/python3.7/site-packages/flask_script/commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/module_build_service/manage.py", line 167, in build_module_locally
username, handle, params, stream=str(stream), skiptests=skiptests)
File "/usr/lib/python3.7/site-packages/module_build_service/utils/submit.py", line 503, in submit_module_build_from_yaml
return submit_module_build(username, mmd, params)
File "/usr/lib/python3.7/site-packages/module_build_service/utils/submit.py", line 633, in submit_module_build
mmds = generate_expanded_mmds(db.session, mmd, raise_if_stream_ambigous, default_streams)
File "/usr/lib/python3.7/site-packages/module_build_service/utils/mse.py", line 417, in generate_expanded_mmds
current_mmd = Modulemd.Module.new_from_string(mmd.dumps())
TypeError: <flask_script.commands.Command object at 0x7f89f21409e8>:
Argument 0 does not allow None as a value

Any pointers anyone?

Paul.

Comments

Re: Modularity tooling intro?

By =?ISO-8859-2?Q?... at 05/30/2019 - 07:30

Dne 30. 05. 19 v 11:16 Paul Howarth napsal(a):
<a href="http://frostyx.cz/posts/how-to-build-modules-in-copr" title="http://frostyx.cz/posts/how-to-build-modules-in-copr">http://frostyx.cz/posts/how-to-build-modules-in-copr</a>

Re: Modularity tooling intro?

By Paul Howarth at 05/31/2019 - 09:57

On Thu, 30 May 2019 13:30:10 +0200

Thanks, but copr is on Fedora infrastructure and I want to use my own.

Surely it can't be that hard to use module-build-service locally with a
suitable module.yaml and a pointer to git repos with rpm specs etc. in
them?

Paul.

Re: Modularity tooling intro?

By King InuYasha at 05/31/2019 - 09:58

On Fri, May 31, 2019 at 9:57 AM Paul Howarth <paul@city-fan.org> wrote:
I haven't managed it so far. It's also trickier if you have local
infrastructure that works differently than Fedora's, as there's no
real room for supporting alternative systems yet. :(

Re: Modularity tooling intro?

By Jun Aruga at 06/01/2019 - 07:56

Sorry I missed your above message.

Maybe you can open the ticket at below repository's issue page.
<a href="https://pagure.io/fm-orchestrator" title="https://pagure.io/fm-orchestrator">https://pagure.io/fm-orchestrator</a>

fm-orchestrator repository is including module-build-service, and mbs-manager.

I remember "mbs-build local" command worked for the local build in late 2017.
But I could not find the command now.

Re: Modularity tooling intro?

By Adam Samalik at 06/03/2019 - 04:03

The local module builds we have documented at the moment [1] should work if
you have an access to the Fedora infrastructure (==internet connection) and
your packages are in the Fedora dist-git.

I know that Merlin (merlinm) is working on tooling that allow you to do
local module builds without relying on any external infrastructure,
basically the way mock works with standalone packages today. He's submitted
a talk to Flock [2] about it. I know that not everyone can attend, but he's
probably a good person to talk to, and if his proposal gets accepted, there
will be a recording. And the docs should get updated, too, when it's ready!
So.. it's coming

[1]
<a href="https://docs.fedoraproject.org/en-US/modularity/making-modules/building-modules-locally/" title="https://docs.fedoraproject.org/en-US/modularity/making-modules/building-modules-locally/">https://docs.fedoraproject.org/en-US/modularity/making-modules/building-...</a>
[2] <a href="https://pagure.io/flock/issue/145" title="https://pagure.io/flock/issue/145">https://pagure.io/flock/issue/145</a>

Re: Modularity tooling intro?

By =?ISO-8859-1?Q?... at 06/03/2019 - 09:18

I wish the initiative started with something simple like "get
module-build-macros SRPM out of modulemd YAML" [1].

Vít

[1] <a href="https://pagure.io/fm-orchestrator/issue/1217" title="https://pagure.io/fm-orchestrator/issue/1217">https://pagure.io/fm-orchestrator/issue/1217</a>

Dne 03. 06. 19 v 10:03 Adam Samalik napsal(a):

Re: Modularity tooling intro?

By Jun Aruga at 05/30/2019 - 06:44

Maybe this is the document.
<a href="https://docs.fedoraproject.org/en-US/modularity/making-modules/" title="https://docs.fedoraproject.org/en-US/modularity/making-modules/">https://docs.fedoraproject.org/en-US/modularity/making-modules/</a>

In my understanding below command executes "mbs-manager" internally.
I do not run "mbs-manager" command directly.

```
$ fedpkg module-build
```

On Thu, May 30, 2019 at 11:17 AM Paul Howarth <paul@city-fan.org> wrote:

Re: Modularity tooling intro?

By Paul Howarth at 05/31/2019 - 09:58

On Thu, 30 May 2019 12:44:20 +0200

Indeed, but that goes off and builds a module on Fedora infrastructure
and I want to build modules for a local repo instead.

Paul.