DevHeads.net

Linking commits to builds on dist-git

Good Morning Everyone,

Since koji 1.15 released last December, koji has a dedicated field for each build
storing the entire source URL with git hash, regardless of how the build was
started.
This allows us to know for each build the exact git hash that was used.

We've implemented a small service listens to fedmsg messages from koji, retrieve
the git hash from that field, and flag the corresponding commit in dist-git with
the outcome of the build.
This way, just while browsing the commits in dist-git you will be able to
directly access the corresponding build made in koji (whether it succeeded or
failed).

Few examples:
<a href="https://src.fedoraproject.org/rpms/rust-exa/c/56591280ba0c1e178105bb4dc59f2c60ea7ff32b" title="https://src.fedoraproject.org/rpms/rust-exa/c/56591280ba0c1e178105bb4dc59f2c60ea7ff32b">https://src.fedoraproject.org/rpms/rust-exa/c/56591280ba0c1e178105bb4dc5...</a>
<a href="https://src.fedoraproject.org/rpms/python-pygit2/c/a5d6031a6682e68d154cfccda51a3549b1aa86df" title="https://src.fedoraproject.org/rpms/python-pygit2/c/a5d6031a6682e68d154cfccda51a3549b1aa86df">https://src.fedoraproject.org/rpms/python-pygit2/c/a5d6031a6682e68d154cf...</a>

Hoping this is helpful,
Best regards,
Pierre for the Fedora Infra team

Comments

Re: Linking commits to builds on dist-git

By Daniel P. Berrange at 08/10/2018 - 05:16

On Fri, Aug 10, 2018 at 11:00:46AM +0200, Pierre-Yves Chibon wrote:
IIUC this is just a UI addition, not actually using git tags ?

ie, I'd love to be able todo "git show libvirt-4.5.0-1.fc28" from the cli
to view a tag / commit from which the NEVR was made.

I get that there'd be security consequences to allowing some service the
ability to write to git, but there are a variety of ways to deal with that.

For example, at cost of extra storage, the build system could have its own
clone of the primary dist-git repos to which it adds the tags. The Fedora
packagers could add that build system repo as a 2nd git remote to get the
tags into their local checkout.

Regards,
Daniel

Re: Linking commits to builds on dist-git

By Ken Dreyer at 08/10/2018 - 18:12

On Fri, Aug 10, 2018 at 3:16 AM, Daniel P. Berrangé < ... at redhat dot com> wrote:
It would be cool to have Git tags in dist-git.

There is something very close that I use in the meantime, "fedpkg
gitbuildhash". It queries Koji and prints the dist-git commit sha1 of
a build NVR.

For example:

$ fedpkg gitbuildhash remctl-3.14-2.fc29
6c633d840e4221831f5502c36822a73b0860e5fc

- Ken

Re: Linking commits to builds on dist-git

By Pierre-Yves at 08/10/2018 - 05:27

On Fri, Aug 10, 2018 at 10:16:13AM +0100, Daniel P. Berrangé wrote:
I'm pretty sure we used to do this at one point but one of the issue is that
tags are no immutable, packagers can change them even if we block force push.
I believe this is why we no longer do this :)

That would be one way, it would pretty much double our storage cost indeed :(

Pierre

Re: Linking commits to builds on dist-git

By Daniel P. Berrange at 08/10/2018 - 05:32

On Fri, Aug 10, 2018 at 11:27:43AM +0200, Pierre-Yves Chibon wrote:
A git commit "update" hook can be used to block deletion or modification
of any existing tags.

Regards,
Daniel

Re: Linking commits to builds on dist-git

By Todd Zullinger at 08/10/2018 - 13:01

Daniel P. Berrangé wrote:
Indeed. The default update hook provides exactly such a
capability (as well as others to prevent deletion of tags
and pushing lightweight tags). The tag can be found in the
git source:

<a href="https://git.kernel.org/pub/scm/git/git.git/tree/templates/hooks--update.sample" title="https://git.kernel.org/pub/scm/git/git.git/tree/templates/hooks--update.sample">https://git.kernel.org/pub/scm/git/git.git/tree/templates/hooks--update....</a>

and in the git-core package:

/usr/share/git-core/templates/hooks/update.sample

Similarly, a hook could be used to disallow the tagging
service from writing to anything outside of refs/tags to
help allay the concerns about a service having write access
to the git repositories.

Also, many thanks to Pingou and everyone who helped to add
this feature!

Re: Linking commits to builds on dist-git

By =?UTF-8?B?TWlyb... at 08/10/2018 - 05:05

On 10.8.2018 11:00, Pierre-Yves Chibon wrote:
It is, Thank You!

Can we also see it in the commit list? E.g.
<a href="https://src.fedoraproject.org/rpms/python-pygit2/commits" title="https://src.fedoraproject.org/rpms/python-pygit2/commits">https://src.fedoraproject.org/rpms/python-pygit2/commits</a>

Re: Linking commits to builds on dist-git

By Pierre-Yves at 08/10/2018 - 05:17

On Fri, Aug 10, 2018 at 11:05:52AM +0200, Miro Hrončok wrote:
I seem to remember Slavek working on something like this. I'll have to check
again.

Pierre

Re: Linking commits to builds on dist-git

By =?UTF-8?B?TWlyb... at 08/12/2018 - 16:25

On 10.8.2018 11:17, Pierre-Yves Chibon wrote:
Yes I've seen this feature on one of the Pagure instance where Slávek is
involved.