DevHeads.net

I would like to propose that we turn on XFS Reflink in Fedora 29 by default

We are adding some features to container projects for User Namespace
support that can take advantage of XFS Reflink.  I have talked to some
of the XFS Reflink kernel engineers in Red Hat and they have informed me
that they believe it is ready to be turned on by default.

I am not sure who in Red Hat I should talk to about this? Whether we
should turn it on in the installer or in the mkfs.xfs command?

Who should I be talking to?  To make this happen.

Comments

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Peter Robinson at 04/28/2018 - 08:55

On Sat, Apr 28, 2018 at 11:09 AM, Daniel Walsh < ... at redhat dot com> wrote:
I would speak to Eric Sandeen I believe he's the Red Hat maintainer
(or one of them) of XFS.

Peter

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Steven Whitehouse at 04/28/2018 - 09:21

Hi,

On 28/04/18 14:55, Peter Robinson wrote:
Steve.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 04/28/2018 - 16:13

On 4/28/18 9:21 AM, Steven Whitehouse wrote:
So, for context, I am the upstream maintainer of xfsprogs as well as for
Fedora xfsprogs.

Historically, new features in XFS have gone from "Experimental" (i.e. under
development), then dropped Experimental (development is ~done) but still optional,
and eventually default. We do this very conservatively, to give bugs a chance
to shake out, which is one of the reasons XFS has a good reputation for /not/
eating your data.

Reflink on XFS only recently dropped "Experimental" and is not yet default upstream;
it won't be default upstream for some time to come - think on the order of months.

However, we do want to give reflink more exposure, and so jumping the gun a bit and
turning it on for rawhide / Fedora 29 is probably a good idea.

I'm mostly ok with patching it on by default in mkfs.xfs; it does confuse things a bit
when "our" version behaves fundamentally differently from upstream, but it's probably
the right thing to do here. I'll make sure that none of the other xfs developers have
strong objections, and if not, I'll patch it in for fedora 29.

Unless this should be a full blown Feature? If so, I'm ok with following that path
as well.

thanks,
-Eric

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Stephen Gallagher at 04/29/2018 - 05:47

XFS is the default filesystem on Fedora Server Edition, so yes: I think we
should really have a System-Wide Change Proposal submitted for this,
primarily to ensure that the information is spread widely (Change Proposals
like this are picked up by Fedora Marketing and tech news, so it’ll be more
widely dispersed than just on the fedora-devel list).

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Colin Walters at 04/29/2018 - 15:59

And we use Server partitioning now for Atomic Host. But for the
vast majority of times people say "Fedora" they're talking about
as a desktop. And Workstation uses the Anaconda defaults which are
ext4 today. See also
<a href="https://pagure.io/pungi-fedora/pull-request/257" title="https://pagure.io/pungi-fedora/pull-request/257">https://pagure.io/pungi-fedora/pull-request/257</a>
I'd say it makes sense to revisit the default here globally in Anaconda.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Jason L Tibbitts III at 04/30/2018 - 13:13

CW> I'd say it makes sense to revisit the default here globally in
CW> Anaconda.

Maybe. Have the issues which made XFS less suitable for use on laptops
been resolved? The primary one I recall was that each mounted
filesystem would have a corresponding kernel thread doing about 20
wakeups per second. This was not really good for battery life and power
consumption in general.

Last I checked (which was 2016 or so) those wakeups were slated to be
around for a while longer.
<a href="https://www.spinics.net/lists/xfs/msg40210.html" title="https://www.spinics.net/lists/xfs/msg40210.html">https://www.spinics.net/lists/xfs/msg40210.html</a>

- J<

Re: I would like to propose that we turn on XFS Reflink in Fedor

By King InuYasha at 04/30/2018 - 13:16

On Mon, Apr 30, 2018 at 2:14 PM Jason L Tibbitts III < ... at math dot uh.edu>
wrote:

And there's still the fun restriction of XFS not being able to shrink. It's
not particularly important in the server case, but in the desktop/laptop
case, it happens enough in my experience that I'm not sure I'd want a
default filesystem that can't shrink.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Martin Kolman at 05/02/2018 - 08:07

On Mon, 2018-04-30 at 18:16 +0000, Neal Gompa wrote:
Of course as with any thin-provisioning you are also giving up things,
such as easily finding out how much free space is actually available to
a given storage volume.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Chris Murphy at 05/02/2018 - 16:21

On Wed, May 2, 2018 at 7:07 AM, Martin Kolman < ... at redhat dot com> wrote:
There's one option that meets the discussion requirements without the
fragile out of space, harder to fix, obscure errors and warnings, and
byzantine interface of LVM (thinp).

And that option has had stable reflink copies, snapshots, and online
resize (shrink and grow) for over 6 years; and is actually being used
by thousands of ordinary desktop Linux users; and without the
handholding that LVM requires or confusion it induces.

First order of business for making LVM thinp a default for
Workstation: a one cycle release of the code of conduct for the Fedora
Community as it pertains to the change, so they can feel free to ask
things like "what kind of drugs were you guys on when making this
decision?"

And I say this as a user of LVM thinp for VM's and my fifth backup
system (sorta funny, do I trust it? Or don't I? I trust it but hope to
god I don't have to? Yes.)

Giving up things like sanity and time as well. I'm not trying to be
mean, I'm completely serious. I've volunteered for the experience,
it's bad ass in many ways, but it is not something I'd foist on the
community by default. I'm vaguely open to the idea of Server using it,
but mostly skeptical.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Chris Murphy at 05/24/2018 - 10:11

Late and off-topic by now, but in retrospect my response is closer to
stupid than the intended hyperbolic+funny.

On Wed, May 2, 2018 at 3:21 PM, Chris Murphy < ... at colorremedies dot com> wrote:

Criticism of negative user experience is not at all inhibited by the
code of conduct. And couching such criticism in hyperbole confuses the
argument I was trying to make, while also unintentionally suggests a
recommendation intended to result in better experience comes from
substance abuse. That's just gross and disappointing. And I apologize
for it.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Marius Vollmer at 05/02/2018 - 04:35

Neal Gompa < ... at gmail dot com> writes:

But note that even ext4 can't shrink while being mounted.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Roberto Ragusa at 05/25/2018 - 08:12

On 05/02/2018 11:35 AM, Marius Vollmer wrote:
I consider that an annoying limitation of ext4, and I would not expect
a replacement filesystem to remove features instead of adding them.

We should have got online shrinking as standard nowadays, IMHO.

Regards.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By King InuYasha at 05/02/2018 - 07:15

On Wed, May 2, 2018 at 5:36 AM Marius Vollmer <marius. ... at redhat dot com>
wrote:

But it can shrink when it's not. This is incredibly important for being
able to deal with resizing both / and /home at the same time, or even
trying to make space for multi-booting (typically with Windows but some
people do other OSes too).

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 05/02/2018 - 08:25

On 5/2/18 7:15 AM, Neal Gompa wrote:
I've always seen the need for shrink as an indicator that someone had
poor planning along the way, or insufficient tools for provisioning to
start with. Sure, there are exceptions, but in general who needs shrink
on a regular basis?

Shrink is actually pretty damaging to the filesystem; it takes all the
locality that the allocator tried to provide, and scatters it to the
wind. The result is a stitched-together mess.

Not only that, but wouldn't any sane administrator with important data
to take care of make a backup before an invasive action like shrink?

And if you have a backup, you're halfway to mkfs & restore, which will
leave you in a much better place.

So yes, you can shrink ext4, but it really should be seen as a last resort
IMHO. I know it can be expedient at times, but I'm not sure people really
consider the downsides of the action. On the surface, "yay it's smaller
now!" but a bit more investigation shows that it's a de-optimizing,
potentially dangerous administrative action. Just my $0.02.

-Eric

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Chris Murphy at 05/02/2018 - 17:49

On Wed, May 2, 2018 at 7:25 AM, Eric Sandeen < ... at redhat dot com> wrote:
Even ancient NTFS does online shrink. It's not because it's regularly
needed on a per user basis, it's because when needed it'd be a huge
PITA to not have it. I don't know the origin story why NTFS got
shrink capability. HFS and HFS+ did not originally have it, it
happened with a bunch of other upgrades including journaling, soon
thereafter was the switch to Intel CPUs, and "boot camp" support for
dual boot. I'm pretty sure shrink support anticipated dual booting.

I think there's a general expectation on Linux desktop systems to be
able to make room for some other OS.

Btrfs excluded. It's either the same (the supers get new size
information and that's the only change), or block group migration
moves extents closer together and on a spinning disk toward the faster
sections. A shrink is essentially a partial balance, and so operation
ends up being more efficient.

It's completely unnecessary on a file system designed for shrinking.

And sure, I get you're talking about file systems not really designed
for it. NTFS shink reliability probably has more to do with how
ancient it is, tons of users, and a lot of weird ancient junk it runs
on - they've had a lot of opportunities to catch the edge cases -
rather than design. But I haven't heard any Windows admin consider it
dangerous or invasive. I've never had NTFS shrink blow up.

I did once have JHFS+ on Core Storage LV blow up on me, although I was
being kind of a saboteur: shrink grow shrink grow shrink grow shrink
KABOOM.

:-P

This thread reminds me of a Start Trek IV scene:

McCoy: My God, man! Drilling holes in his head isn't the answer! Now
put away your butcher knives and let me save this patient before it's
too late!

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Stephen John Smoogen at 05/02/2018 - 19:07

On 2 May 2018 at 18:49, Chris Murphy < ... at colorremedies dot com> wrote:
My understanding was that NTFS shrink was to get a sales check mark
versus a supported feature. You could shrink a FAT file system so
customers wanted NTFS to be shrinkable too so enough was implemented
to make it happen but it was not recommended to be done unless last
resort. [The other story was that Unix file systems could not shrink
and it would be a 'this is better than Unix' checkmark to show off in
a sales demo. ] In practice, shrinking NT 3.51/4.0 was a nightmare
with blowups happening for many odd reasons. I got tired of walking
customers through recovery steps and blaming Linux on destroying their
Windows system by the time I left Red Hat support in 2000.

When I did help desk at a large facility, Windows 2000 was better but
would blowup occasionally . We found that Windows XP was pretty
reliable to shrink with no blowups during shrinkage. Instead what
happened was many unexplained problems afterwords. Higher number of
BSoD, drivers being there on disk but the OS saying 'nope' sometimes,
file metadata gone versus corrupt. [My windows admins friends said 7
was better than XP, and that they have had much better luck with 10,
but that people aren't dual booting or shrinking file systems as much
as they used to.]

The server admins never reported crashes during shrinking but the
help-desk admins I worked with, it was a different story. I expect it
has to do with the fact that servers usually have more control of
'best practices' being done, while the desktop/laptop can have all
manner of things running which can affect file integrity in some way.
If it wasn't a server, the usual response from Microsoft was that if
it happened after a shrink, reinstall. If it happened again after that
open a ticket through the contract support.

For a lot of us, working out all the steps to shrink a disk properly
and deal with possible hangups is second nature. We can say 'of course
we should be able to shrink disks'. We also know when not to try and
shrink a disk unless we have done some steps before hand. However, a
lot of the people who are going to try and shrink a disk are just
following whatever stackoverload voted the highest that day. It isn't
going to say 'well you know if you are at 90% on that version of
BTRFS/EXT/JFS you don't want to shrink it until you have updated XYZ.'
or some other thing you know to look out for. Instead as you say they
are going to expect that it will work all the time just like they
expect seat belts to keep them alive at 100 mph crash.

In any case, I think we have completely lost any link to the original topic.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Nico Kadel-Garcia at 05/02/2018 - 16:57

On Wed, May 2, 2018 at 9:25 AM, Eric Sandeen < ... at redhat dot com> wrote:
I've used it several times in the last year, usually to re-arrange
storage so that a cache is on a separate filesystem and less likely to
screw up the whole working environment.

Locality is *much* less critical with modern flash drives.

Yes, they would. So what? While it's common to stop a filesystem,
backup a directory, and re-assign a new partition mounted locally to
restore the data to, this can be a *very* painful operation.

In theory, yes, which is why I personally prefer to use such tools.
But doing this to "/" when someone has used default partitioning and
you need to clean it up later with minimal downtime is very expensive
in manpower.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Nicolas Mailhot at 05/02/2018 - 09:09

Le 2018-05-02 15:25, Eric Sandeen a écrit :

Desktops because they have no planning (poor or otherwise) pretty much
by definition.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Ian Pilcher at 05/02/2018 - 08:42

On 05/02/2018 08:25 AM, Eric Sandeen wrote:
The point isn't so much that you need it on a regular basis, it's that
when you need it, you *really* need it.

I'll buy the poor planning argument on a server that does pretty much
the same thing for the entirety of its life/deployment, but the case of
a laptop/desktop that goes years without being reinstalled, and then
unexpectedly needs tens of gigabytes of space to bisect a kernel bug
is very different.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 05/02/2018 - 08:56

On 5/2/18 8:42 AM, Ian Pilcher wrote:
Given that it is exception activity, dump/mks/restore is also a less
convenient but more robust solution to the problem.

I mostly want to point out that shrink permanently de-optimizes your
filesystem, and carries some risk as well, especially if you have no
backups.

If you're putting your years-old root or home filesystem at risk to bisect
a bug I'd humbly suggest that an external or additional disk might be more
suited to the task.

I don't have any real horse in this race - if Fedora feels that shrink
capability trumps features like reflink, that's fine. Just offering my
thoughts on the matter, and trying to point out that shrink has its
downsides.

-Eric

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Ian Pilcher at 05/02/2018 - 09:51

On 05/02/2018 08:56 AM, Eric Sandeen wrote:
I'm sitting in a hotel room with a laptop. What do I backup *to*?

See above.

Ack.

I personally think that it's fine that XFS can't shrink. It's just
important to be clear about the use cases for which it's intended. I
also get uncomfortable with just dismissing use cases like this without
considering that there may be legitimate reasons for them.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 05/01/2018 - 07:39

On 4/30/18 1:16 PM, Neal Gompa wrote:
As we discussed on IRC, I think idle filesystems won't get these wakeups;
if this is still a concern, investigation of the actual effects on battery
life (if any) are recommended.

XFS realistically is not going to get shrink. Fedora will need to decide if
the other capabilities & features outweigh the lack of shrink capability.

-Eric

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Dusty Mabe at 04/30/2018 - 15:31

Yes! I've been wanting that for a long long time!

Dusty

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Peter Robinson at 04/29/2018 - 06:23

Assuming that the plan is to leave it enabled in F-29 on branching and
have it ship enabled in F-29 I agree, if the intention is to leave it
enabled in rawhide and disable it on branching then the Change
Proposal mechanism isn't the way to ensure wider knowledge.

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 04/29/2018 - 11:45

On 4/29/18 6:23 AM, Peter Robinson wrote:
I'd be happy to have it left on for F29, it /might/ even be default upstream
by the time F29 ships. I'll need to re-familiarize myself w/ the
"System-Wide Change Proposal" process, I guess.

FWIW, turning it on has very little effect until it's actually used, so it
really should not be destabilizing for most, even if bugs lurk. ;)

-Eric

Re: I would like to propose that we turn on XFS Reflink in Fedor

By Eric Sandeen at 05/03/2018 - 07:53

On 4/29/18 11:45 AM, Eric Sandeen wrote:
Getting back on topic (back from the Shrink Wars), I'd like to get this
turned on. I'd prefer to do this via a soon-to-land patch which enables
config files for mkfs, so that distros can set their own defaults without
having to patch source. I'm going to give it a bit of time to see if we
can get that upstream, and then will see about simply shipping that
patch along w/ a config file that enables reflink for F29.

Thanks,
-Eric