building nfs-ganesha for ceph

Hi Niels and Kaleb,

I'm wondering how we could best build nfs-ganesha's ceph FSALs in CBS.

Right now, since we'll likely support multiple versions of nfs-ganesha
simultaneously for different versions of Ceph, I'm leaning towards
using a custom "dist tag" like thing in the Release field so we don't
collide with what Kaleb's doing at
<a href="" title=""></a>

Eventually it would be nice to have a unified nfs-ganesha build shared
between Ceph and Gluster. If we built it in storage7-common-candidate,
we would have to tag gluster and ceph into that, which sounds kinda
messy, because you only get to choose one version of each. It'd be
more manageable to have a separate set of tags and targets just for
nfs-ganesha versions. Like:

... etc

But that still leaves the problem of shipping a nfs-ganesha-gluster
sub-package that will not have its runtime deps fulfilled in
<a href="" title=""></a> . Vice
versa, we'd have a nfs-ganesha-ceph sub-package that lacks runtime
deps in <a href="" title=""></a>

What do you think? Just have separate nfs-ganesha builds for now?

- Ken


Re: building nfs-ganesha for ceph

By Kaleb S. KEITHLEY at 12/04/2018 - 19:08

On 12/4/18 5:38 PM, Ken Dreyer wrote:
I too would like to get to a place where we have a single ganesha package.

Gluster's GFAPI guarantees its ABI by using versioned symbols. A Gluster
FSAL built against glusterfs-4.1 (or earlier) will work with
glusterfs-5's And glusterfs-6's. And so on.

So we don't build separate ganesha-2.7 packages for both glusterfs-4.1
and glusterfs-5. We build a single ganesha-2.7 package with
glusterfs-4.1 and it is tagged into both the glusterfs-4.1 and
glusterfs-5 releases.

I think it should be achievable. I try and build ganesha with as many
FSALs enabled as possible. It's been a while though since I looked at
which version of Ceph was in CBS for the Storage SIG.

If Ceph doesn't guarantee its ABIs though that's going to make things

Re: building nfs-ganesha for ceph

By Ken Dreyer at 12/05/2018 - 12:11

On Tue, Dec 4, 2018 at 4:09 PM Kaleb S. KEITHLEY < ... at redhat dot com> wrote:
I confirmed with Matt Benjamin on ceph-devel today that this ok. We'll
be able to build nfs-ganesha-2.7 against Luminous and then cross-tag
that into our newer releases (eg mimic and nautilus). So we can follow
the same model that you're following with building nfs-ganesha in the
"oldest supported" build target.

In terms of sharing one single nfs-ganesha build across ceph and
gluster, I think that will require more work. We'd probably want:

1) A custom nfs-ganesha "-build" tag/target that contains the "oldest
supported" versions of both ceph and gluster,

2) cross-tag the nfs-ganesha builds from there into both the ceph and gluster
-testing tags,

3) Update the mash configuration to exclude the irrelevant sub-packages
when building ceph and gluster's "testing" and "release" repositories.

Unfortunately that last bit looks really tricky. I'm not sure there is
a way to tell mash itself to exclude a particular sub-package, and!cbs-tools.git/master/scripts!
uses a basic mash config template for all CBS tags.

- Ken

Re: building nfs-ganesha for ceph

By Niels de Vos at 12/05/2018 - 12:53

On Wed, Dec 05, 2018 at 09:11:58AM -0700, Ken Dreyer wrote:
This is something that I'm very much looking forward to see happening.
The 3rd point is indeed something that (from my understanding) is not
possible. I have thought of an alternative, but that is still a little
ugly. Not sure if we want to go that route.

- add a new repository with all nfs-ganesha-x.y packages (direct
dependencies and sub-packages), without ceph/gluster

- add a new repository definition in centos-release-gluster and
centos-release-ceph packages. This entry in the .repo file should use
the 'exclude=' directive to skip the Ceph sub-package for Gluster, and
the Gluster sub-package for Ceph.

For additional projects (like Samba) this would work in a similar way.


Re: building nfs-ganesha for ceph

By Ken Dreyer at 12/05/2018 - 13:47

On Wed, Dec 5, 2018 at 9:53 AM Niels de Vos < ... at redhat dot com> wrote:
The biggest reason I want to exclude the package server side is that
will allow repoclosure to pass for the repo. repoclosure provides me a
sanity-check that the entire "product" is installable and I'm not
missing any packages. With the way we're growing more and more
packages, I want to make this as automated as possible. Ideally
repoclosure would automatically gate a package going from -candidate
to -testing.

One problem I see with configuring this exclusion client side is that
I'm not sure I can rely on everyone using the exact same .repo file. I
eventually want to ship packaged mock configs for Ceph, so we'd have
to duplicate that there as well :(

I'd like to know if the CentOS admins plan to stay on mash long-term,
and if so, let's look into filtering at that level, like Pungi can do.
What do you think?

- Ken

Re: building nfs-ganesha for ceph

By Niels de Vos at 12/05/2018 - 14:14

On Wed, Dec 05, 2018 at 10:47:53AM -0700, Ken Dreyer wrote:
Yes, I understand that. This is an item on my wishlist for the Gluster
repositories for a looong time already.

Ah, yes. I have my mock-config files from the 'cbs' tool, but it is not
friendly for others to use. Packaging them would be really nice too.

This definitely would have my preference as well!