Hard drive spec change

By Mike Chambers at 03/10/2010 - 17:22

Hadn't seen this discussed yet (not really a big hardware geek), and
just saw an article about this today. Are we (linux as a whole) ready
for this or getting ready, or already using it? And If we bought a new
hd, via sata in my case within past year or newer computer in past year,
are we using the new 4kb sectors now or just able to handle it?

<a href="http://www.pcmag.com/article2/0,2817,2361156,00.asp" title="http://www.pcmag.com/article2/0,2817,2361156,00.asp">http://www.pcmag.com/article2/0,2817,2361156,00.asp</a>

Development

46 comments

We've been working on install and boot support for this for a while now.

There has been a lot of work upstream on 4k sector support, and in general
yes, we are ready.

-Eric

Is F12 ready or will only F13 be? E.g. if I want to have a RAID with LVM
and dmcrypt, will this all align properly?

I just bought a WD20EARS and tested on F12. fdisk has an option to set
the sector size to 4096 byte, but it will still use sector 63 by default
for a new partition. Shouldn't it then default to sector 16, which is
sector 64 with 512 byte sector size?

And pvcreate, mdadm and cryptsetup seem not to accept a sector size
parameter. pvcreate and cryptsetup at least accept an alignment
parameter, but this will leave it to me to make the math instead of
doing it automatically. :-( I am not sure how well mdadm will behave,
but it seems not to have any parameter to tweak the alignment or the
sector size.

Regards
Till

The default pseudo-geometry will still be 63 sectors/track unless you
change it, and by default a partition's start-of-data is forced to the
beginning of a track. Making the sectors larger doesn't change that.

Warning: this question is asked without any knowledge about the subject.
Does it really make sense that the number of sectors/track is
independent of the size of a sector?

That parameter is already totally unrelated to the physical layout of
data on the disk. It is no more invalid for a 4K sector size than it
is for a 512-byte sector size. Really, the only thing that is affected
is the amount of space that fdisk (in the default DOS Compatibility
mode) leaves between a primary or secondary partition table and the
start-of-data for the partition.

The sectors/track are afaik totally non interesting nowadays and the F13
fdisk behaves quite different if the non dos compatibility mode is used.
Then it tries to be Windows compatible afaics, e.g. forcing the
beginning of the first partition at a 1MiB offset from the beginning of
the drive. For pure (modern) linux systems the best tool seems to be for
me bdisk, which uses modern GPT partition tables, that do not care about
tracks or cylinders.

Regards
Till

^^^^^
*sigh* This is meant to be gdisk.

Regards
Till

I mean sector 8 here. So I just gave an example why the tools should do
the math for me. ;-)

Regards
Till

ons 2010-03-10 klockan 15:57 -0600 skrev Eric Sandeen:

Problems can probably be expected in case the drive does not report its
real block size to the software, though, like my WD15EARS (I think) or
VMware's emulated SCSI drives.

There's only so much we can do in the face of bad hardware ;)

-Eric

How about making it possible to overwrite the wrong reported values,
e.g. by making /sys/block/sdb/queue/[physical_block_size
writable,minimum_io_size} writable.

Regards
Till

I think that would be a bad idea - if you know what you want to change, why not
just invoke the tool with those specific parameters?

It would be very confusing to have us probe the device, get the real information
from the storage device and then let a user update that information to something
"false" :-)

ric

It would not be that annoying, if it was only one tool. But there are
e.g. at least fdisk, mdadm, lvm, cryptsetup and mkfs.*/mkswap that all
have to properly align the data.

So add some interface to query whether the information was probed or set
by a user. Then it should not be that confusing. :-)

Regards
Till

We are currently working to verify that storage devices work properly & report
the information that they want us to use (doing this with several storage
providers and have also raised this with EMC/VMware).

Specifically for vmware, I think that our default will be good for them when a
device fails to report. Specifically, their own tech report suggests using a
non-default alignment (sector 128 iirc?). We can certainly work to verify this
with them.

If we see real world issues during this testing, we can start thinking about how
to fix it.

thanks!

ric

This is good, but it already failed with Western Digital afaik. I
somewhere read that they did it correctly with test drives but now the
released drives do not report their sizes properly, so there are already
real world issues. It was mentioned on this thread I and have a WD20EARS
that shows this problem, too.

Regards
Till

I should have asked - do you have the details captured in bugzilla? If so, that
will be useful to help kick off the discussion with them.

Thanks!

Ric

It seemed to be common knowledge already, but I just created a bug
report:
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=575143" title="https://bugzilla.redhat.com/show_bug.cgi?id=575143">https://bugzilla.redhat.com/show_bug.cgi?id=575143</a>

Regards
Till

We should follow up with them and see if we can help them fix their devices...

ric

It got some discussion when first 4KiB drives shipped to customers
last year. Also, LWN has excellent (as usual) writeup on the matter:
<a href="http://lwn.net/Articles/377895/" title="http://lwn.net/Articles/377895/">http://lwn.net/Articles/377895/</a> . If you need access to this article,
please drop me an email.

On 2010/03/10 15:22 (GMT-0600) Mike Chambers composed:

The change is for the benefit of manufacturers, not users. Readiness is only
spotty. The discussion has been extensive and ongoing on the linux-ide
mailing list.

Users do benefit as well - more capacity per platter specifically is one
obvious win.

ric

On 2010/03/10 17:09 (GMT-0500) Ric Wheeler composed:

Some users, yes, those with unfathomable storage capacity requirements,
saving every byte ever encountered without regard for its utility. For those
with more common needs, the compatibility cost outweighs the purported
benefits. Most users don't even need 1/10 of .2TiB, much less the 2TiB at
which larger than 512 byte sectors might start looking sane in cost/benefit
analysis.

Famous last words!

Note also that the access time will be slightly faster.

And power consumption will go down as you won't need as many platters :-)

ric

On 2010/03/10 20:19 (GMT-0500) Ric Wheeler composed:

As if an average normal person could tell.

Not materially for those whose needs are already down to less than one
platter. MultiGHz, Multicore CPUs consume magnitudes more power than HDs.

There are also a lot of low-power systems that use a HD, e.g. tv
recorders or home entertainment systems. Also if you use some of the
recently available ARM base plug computers, the power consumption of one
HD makes a lot of difference compared to the remaining system. E.g. a
SheevaPlug[0] consumes 2.3W in idle mode or 7W with full CPU according
to wikipedia and running Fedora on it is supported within the secondary
archs project.

Regards
Till

[0] <a href="http://en.wikipedia.org/wiki/SheevaPlug" title="http://en.wikipedia.org/wiki/SheevaPlug">http://en.wikipedia.org/wiki/SheevaPlug</a>

Not always. A typical 3.5" harddrive consumes about (max):
0.65A * 5V = 3.25W
0.50A * 12V = 6.00W
which totals 9.25 Watts, and less when not transferring data.
I am composing this message on a system with a 2.5GHz, two-core
processor that consumes 45 Watts maximum, and less when "idle".
So in this case the ratio is closer to 5:1, not 10:1.

That doesn't compare to figures that I have seen - a lot of 7200rpm hard
disks will draw up to a couple of amps on the 12v line (at least, during
startup) giving you peak power consumption in the 20-30W range. The
lastest data sheets I can find for Seagate's Baracuda 7200 drives for
e.g. quote a maximum draw of 2.8A (33.6W).

Admittedly that's a peak value and not an average but so is the 45W
processor figure; I think claiming that modern CPUs consume "magnitues
more power" than modern hard disks is not justifiable.

Regards,
Bryn.

On 2010/03/10 19:11 (GMT-0800) John Reiser composed:

RAM, GPUs & fans are also nonzero power consumers, so any reduction in HD
power consumption in typical desktops and laptops is minimal in the grand scheme.

OTOH, loss of backward compatibility means accelerated additions to landfills
of otherwise perfectly useful hardware.

It should be fine to have an option for larger sector sizes, but it shouldn't
mean extinction in the foreseeable future of a happily working standard for
those who only want to replace a HD when the HD dies, and not commit whole
systems to landfills for want of a generic component.

Sounds to me like the HD manufacturers want to make 512 go the way of PATA,
accelerating obsolescence to drive up profitability of the whole computer
hardware industry. People shouldn't have to buy whole new systems for want of
replacing a dead 20GiB 512 byte sector HD.

For a data center, this matters. Maybe not for your laptop, but not all
the world's storage is on small systems.

linux will be backward compatible with 512-sector drives just fine; nothing
will stop that ...

.... it doesn't mean extinction, your old drives will continue to work just fine,
and if you -have- to buy a 4k sector drive, linux is ready for you. (for that
matter, I expect 512 sector drives will be available for a good long time, too).

I really don't understand why this bothers you so much; there is new hardware
on the way, and Linux works with it. If you want to keep your 20G drives until
they die, nobody is stopping you. These things will likely coexist for a long time.
Nobody is forcing your hand.

I suppose this now-off-topic discussion has gone on long enough, though, so this'll
be my last word on the subject here.

-Eric

On 2010/03/10 22:41 (GMT-0600) Eric Sandeen composed:

Have you tried to buy a replacement PATA disk lately, particularly one no
larger than the 2^28 ATA-5 addressing limit? Buying a replacement 20G HD, or
one compatible with it even if 10X or more the storage size actually needed,
has become a difficult task, particularly if a new product and a warranty
longer than 90 days is desired. The bother is that it looks like HD makers
will have their way with 512 as they have with PATA, forcing a lot of
otherwise useful hardware into landfills prematurely.

No. I haven't tried buying a replacement 386 lately, either.

Recycling vs. landfills is an entirely different discussion, but I have
been happy to get rid of PATA drives. My only complaint about SATA
drives is that they are still too slow, too small, too expensive, and
too power-hungry.

But I'm still missing the main point. Should Fedora avoid supporting
4KB sectors just because some users might not notice any improvement?

You know you can buy a PCI SATA controller card for about $10 in any PC
junk store, right?

On 2010/03/11 12:10 (GMT-0800) Adam Williamson composed:

PC BIOS treat those as SCSI cards, which do not play nice with boot device
order control by the PC BIOS, if not OS device names. Even when neither is a
problem, it's no given that a PCI slot is available, or the junk store has
enough cards for all machines with dead HDs on the LAN.

What exactly are you trying to say?

"Do not support newer hardware to give vendors a reaons to sell old hardware" ?

That does make zero sense to me ...

On 2010/03/11 23:23 (GMT+0100) drago01 composed:

Not even. Just don't use new technology as excuse to accelerate abandonment
of old hardware. New stuff does not instantly convert old stuff into bad
stuff. We don't force old BMWs into salvage yards just because new ones use
different sized tires. Tire manufacturers don't need to throw away the molds
for the old tire sizes, just as HD manufacturers don't need to throw away all
tooling for PATA HD interfaces. Old machines work fine with new tires/HDs,
regardless how much better newer machines with more evolved tires/HDs (might)
work.

Once upon a time, Felix Miata < ... at earthlink dot net> said:

A car analogy isn't exactly appropriate here; car lifetime is probably
an order of magnitude longer than the typical hard drive. Also, tires
are a consumable, while for a large percentage of computer users, hard
drives are not (they are inside the magic box somebody got at Wal-Mart).

In any case, I don't know what your problem is. I went to newegg.com
and still see over a dozen PATA drives listed, in sizes from 80G to
500G.

If you really need more PATA drives and want a wider selection, buy a
SATA-PATA adapter and use a SATA drive.

Hard drive manufacturers operate on razor-thin margins and can only make
a certain number of drives. If the vast majority of new drives bought
are SATA, they are going to stop making PATA drives (or at least raise
the prices significantly). They are not in business to cater to the
small (and rapidly shrinking) market.

Then why are you saying it here? To the very best of my knowledge, the
Fedora project does not manufacture any hard disks.

I think that you might have a more convincing argument by stating that
most people, if we do our job right with the kernel & tools, won't be
able to tell the difference.

For the storage industry, this is equivalent to things like a process
change in the CPU market. Pack more stuff in the same space, reduce
power and improve performance.

If you personally don't care or don't know enough to notice these
improvements, we just need to make sure it is as painless as possible.
You probably won't notice the difference.

For anyone serious about storage (performance, reliability and power
consumption) this will be a positive step.

Ric

On 2010/03/10 21:28 (GMT-0500) Ric Wheeler composed:

Not everyone. Users of larger numbers of small files and small numbers of
large files already lose a heap of space to slack even with 1024k blocksize,
which will at least quadruple if forced to 4k sectors.
<a href="http://en.wikipedia.org/wiki/Internal_fragmentation#Internal_fragmentation" title="http://en.wikipedia.org/wiki/Internal_fragmentation#Internal_fragmentation">http://en.wikipedia.org/wiki/Internal_fragmentation#Internal_fragmentation</a>

Second on my list of annoying replies is a pointer to wikipedia (trumped
only by replies with random URLS !).

If you really want to store lots of really tiny files (< 1KB), you
probably want to look at wants to store them in more efficient ways (tar
them up, use a light weight DB, etc). Having been in the business of
making storage appliances that stored lots of small files, it is a
challenge.

Also note that the overhead of creating a file/directory entry/inode in
most modern file systems can easily consume more than a tiny file. If
you want to test this, just take your favourite file system and make a
brand new, empty FS. Fill it with zero length files and then see what
your per file overhead is.

In any case, you could use a file system (like reiserfs) that does tail
packing.

Ric

Another thing to keep in mind is that most file systems will hit severe
performance issues with file count before you end up filling the disk.

Assume that you take a 2TB disk and try to pack it with 1KB files - you
need to be able to store, sort and fsck a file system with close to 2
billion files. How do you index them? How many files/directory? How deep
is your directory tree?

For most any user, you would be lucky to get up to the point of
utilization where fragmentation would start to be a concern...

ric

Spoken like someone who has never used usenet.

- ajax

And yet they snap up said drives at under $200 a pop....

I'm not sure this a fight you will win. :)

-Eric

Topic	Started
Unresolved Fedora 14 Beta Blocker Bugs	5 hours 10 min ago
spot's firefox4 breaks fedpkg upload	5 hours 17 min ago
Outage: Updates - 2010-09-10 00:00->02:00 UTC	14 hours 1 min ago
sip-4.11.x (and PyQt stack) coming to rawhide	14 hours 7 min ago
F-14 Branched report: 20100908 changes	18 hours 2 min ago
all-versions Koji repository (or a build-id database)	18 hours 47 min ago
rawhide report: 20100908 changes	21 hours 23 min ago
[Test-Announce] Fedora 14 Beta Test Compose Validation Test ( 09/09 - 09/15 )	22 hours 42 min ago
intent to orphan caribou	1 day 34 min ago

Categories

Archives

Hard drive spec change

46 comments

Similar Topics