DevHeads.net

fedora-img-dl: a tool for downloading Fedora iso's and images

Hi, I made a small cli tool for downloading Fedora iso's etc.

It can download rawhide, branched, beta, and released isos (eg Workstation
Live etc), and even WS Live respins (support for spins coming later).

You can try it now from <
https://copr.fedorainfracloud.org/coprs/petersen/fedora-img-dl/>.

Feedback is welcome.

Jens

ps I am not entirely in love with the name, so if you have suggestions for
a better one, let me know - note it may also be extended to more OSes in
the future perhaps.

Comments

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Adam Williamson at 04/15/2019 - 12:57

On Mon, 2019-04-15 at 17:22 +0800, Jens-Ulrik Petersen wrote:
I already basically wrote this:

<a href="https://pagure.io/fedora-qa/fedfind" title="https://pagure.io/fedora-qa/fedfind">https://pagure.io/fedora-qa/fedfind</a>

it has a lot more capabilities, and is used quite heavily in various
tools and processes the QA team uses.

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Richard W.M. Jones at 04/16/2019 - 16:55

On Mon, Apr 15, 2019 at 09:57:26AM -0700, Adam Williamson wrote:
Well I can go one better here and say there's a way to *not* download
the ISOs :-)

<a href="https://rwmj.wordpress.com/2019/04/13/virt-install-nbdkit-live-install/#content" title="https://rwmj.wordpress.com/2019/04/13/virt-install-nbdkit-live-install/#content">https://rwmj.wordpress.com/2019/04/13/virt-install-nbdkit-live-install/#...</a>

Rich.

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Adam Williamson at 04/16/2019 - 17:05

On Tue, 2019-04-16 at 21:55 +0100, Richard W.M. Jones wrote:
But you still need to know where the ISO you want to use *is*, which
fedfind can help you with ;)

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Richard W.M. Jones at 04/17/2019 - 03:56

On Tue, Apr 16, 2019 at 02:05:40PM -0700, Adam Williamson wrote:
Absolutely :-) There's also the question Chris Murphy asked on that
posting about whether in fact it actually downloads the whole ISO
anyway (because of the way our ISOs contain a squashfs containing an
ext4 image[1]).

If we did go for lazy downloading this would be an opportunity to use
a format which is more attuned to this purpose. And it could also
contain much more content, since it would only be downloaded on
demand.

There are some sites which let you PXE boot Linux distros over the
internet (<a href="https://netboot.xyz/" title="https://netboot.xyz/">https://netboot.xyz/</a> being the most notable). I have a
vision that we could do this officially for Fedora one day, so you
could entirely try out Fedora without any download or local install
(until you decided you want to perform the local install).

Of course I have already written the nbdkit plugin for this too ...

<a href="https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/#content" title="https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/#content">https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/#content</a>

Rich.

[1] <a href="https://rwmj.wordpress.com/2009/07/15/unpack-the-russian-doll-of-a-f11-live-cd/" title="https://rwmj.wordpress.com/2009/07/15/unpack-the-russian-doll-of-a-f11-live-cd/">https://rwmj.wordpress.com/2009/07/15/unpack-the-russian-doll-of-a-f11-l...</a>

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Adam Williamson at 04/17/2019 - 11:21

On Wed, 2019-04-17 at 08:56 +0100, Richard W.M. Jones wrote:
Surely the most notable for Fedora is <a href="https://boot.fedoraproject.org/" title="https://boot.fedoraproject.org/">https://boot.fedoraproject.org/</a> ?
:)

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By =?ISO-8859-1?Q?... at 04/18/2019 - 05:38

Dne 17. 04. 19 v 17:21 Adam Williamson napsal(a):

Last time I checked it didn't work on UEFI systems :/

<a href="https://pagure.io/fedora-infrastructure/issue/4437" title="https://pagure.io/fedora-infrastructure/issue/4437">https://pagure.io/fedora-infrastructure/issue/4437</a>

I did not have a chance to test it recently.

Vít

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Jens Petersen at 04/16/2019 - 09:55

On Tue, Apr 16, 2019 at 12:58 AM Adam Williamson < ... at fedoraproject dot org>
wrote:

Wow, very cool, looks comprehensive, thanks.
I see it even downloads composed images from Koji.

Jens

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Adam Williamson at 04/16/2019 - 10:47

On Tue, 2019-04-16 at 21:55 +0800, Jens-Ulrik Petersen wrote:
That's actually what it's mainly used for: it started when we (QA)
wanted to be running tests on both nightly image builds and TCs/RCs in
various places, and found ourselves writing big ugly lumps of
conditionals for where various bits turned up under different
circumstances into all these different places. I hated that, so I made
fedfind to be a sort of 'hey where the hell do I get these bits'
helper.

That was before Pungi 4, when there wasn't really a proper 'compose'
concept, so to find nightly lives it had to run Koji queries and all
sorts of nastiness like that. Since Pungi 4 things are better, but it
turns out fedfind is still useful for...various reasons (it provides a
bunch of helper features, it flattens image metadata into a more usable
form, it provides a sort of 'compatibility' implementation so you can
treat stable releases as if they were Pungi 4 composes even though they
actually either weren't built by Pungi 4 or have had their metadata
stripped and bits of themselves moved around...stuff like that).

fedfind does not actually implement downloading (you'd have to just
pipe the URLs it can give you to curl, or whatever) and doesn't
actually have a concept of 'the current rawhide' or 'the current
branched' exactly - that is, it can't find these for you:

<a href="https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/" title="https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/">https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/</a>
<a href="https://dl.fedoraproject.org/pub/fedora/linux/development/30/" title="https://dl.fedoraproject.org/pub/fedora/linux/development/30/">https://dl.fedoraproject.org/pub/fedora/linux/development/30/</a>

if you ask it for just 'branched' or 'rawhide' without any version
information, it will find the latest successful pre-mirroring compose
location for you, e.g. right now it would find:

<a href="https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20190416.n.0/" title="https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20190416.n.0/">https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-201904...</a>
<a href="https://kojipkgs.fedoraproject.org/compose/branched/Fedora-30-20190416.n.0/" title="https://kojipkgs.fedoraproject.org/compose/branched/Fedora-30-20190416.n.0/">https://kojipkgs.fedoraproject.org/compose/branched/Fedora-30-20190416.n.0/</a>

it's quite a subtle difference - those things will usually be *the
same thing*, just in different places - but if it's important to you,
we could probably write that support into fedfind somehow.

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Dan =?utf-8?B?x... at 04/15/2019 - 05:54

Hi Jens,

cool project! I see a certain overlap with fedora-mediawriter though, as
that one can download ISOs too.

I've skimmed the sources (is this upstream:
<a href="https://github.com/juhp/fedora-img-dl" title="https://github.com/juhp/fedora-img-dl">https://github.com/juhp/fedora-img-dl</a>?) to take a look whether you
verify the GPG signatures of the downloaded images, but couldn't find
anything like that (although I can barely read Haskell). If your tool
would do that, I'd consider that a killer feature.

Cheers,

Dan

Jens-Ulrik Petersen < ... at redhat dot com> writes:

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By Jens Petersen at 04/17/2019 - 06:07

On Mon, Apr 15, 2019 at 5:55 PM Dan Čermák <dan.cermak@cgc-instruments.com>
wrote:

Okay, thanks, my own use-case is more downloading Live images for local
testing.

I've skimmed the sources (is this upstream:

Okay, I implemented it yesterday in version 0.3 along with support for
Spins.
Available now in
<a href="https://copr.fedorainfracloud.org/coprs/petersen/fedora-img-dl/" title="https://copr.fedorainfracloud.org/coprs/petersen/fedora-img-dl/">https://copr.fedorainfracloud.org/coprs/petersen/fedora-img-dl/</a>.

Cheers, Jens

Re: fedora-img-dl: a tool for downloading Fedora iso's and image

By =?UTF-8?Q?Micha... at 04/15/2019 - 05:43

Will this be used by Fedora Media Writer [0]?

[0] -
<a href="https://docs.fedoraproject.org/en-US/quick-docs/creating-and-using-a-live-installation-image/" title="https://docs.fedoraproject.org/en-US/quick-docs/creating-and-using-a-live-installation-image/">https://docs.fedoraproject.org/en-US/quick-docs/creating-and-using-a-liv...</a>

On 4/15/19 11:22 AM, Jens-Ulrik Petersen wrote: