XFS
XFS and our servers
This page serves as a description of how we format our xfs partitions and why.
How they're formatted
root@db1047:/a/sqldata# xfs_info /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=109108672 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=436434688, imaxpct=5
= sunit=64 swidth=512 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=213120, version=2
= sectsz=512 sunit=64 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
How they get formatted
fenari:/home/midom/xfsfix is some python that gets the right device and UUID names, then spits out executable bash code that looks something like this:
root@db1047:/a/sqldata# python /root/xfsfix umount /dev/sda6 mkfs.xfs -f -d sunit=512,swidth=4096 -L data /dev/sda6 xfs_admin -U f1363f7d-8a44-4abe-9e38-bf2171e265c8 /dev/sda6 mount /dev/sda6
You'll notice that the sunit and swidth numbers put out by the script don't match what xfs_info prints out. Domas guesses that the script's numbers are too large, but states that the numbers printed by xfs_info are acceptable as is.
Why they're formatted this way
[11:55 AM] <domas> 70U [11:56 AM] <maplebed> so domas any thoughts on that pastebin? [11:57 AM] <domas> hmmmm [11:57 AM] <domas> *shrug* [11:57 AM] • domas looks some more [11:57 AM] <maplebed> Jeff_Green notices that the sunit=64 and swidth=512 is also present on db26 [11:57 AM] <domas> not on other machines? [11:57 AM] <maplebed> (in the quest to see what's "right" that seems like a good place to start) [11:57 AM] <domas> db26 is LVM [11:57 AM] <maplebed> I haven't looked at other machines yet. [11:58 AM] <maplebed> db42 is the same... [11:59 AM] <domas> I guess I just have too high numbers there [11:59 AM] <domas> it is not in bytes but in 512b sectors [12:00 PM] <maplebed> not blocks? (which are set to 4096)? [12:01 PM] <domas> pain oh pain [12:01 PM] <Jeff_Green> ha [12:01 PM] <domas> 'sectors' is usually in 512 in linux [12:02 PM] <domas> 512*512 is 256k alignment [12:02 PM] <maplebed> at any rate, I've got to run; if you think the current settings are fine I'll update the docs. [12:02 PM] <domas> they are good enough [12:02 PM] <domas> 32k alignment is good enough too [12:02 PM] <domas> the major thing is not to have 16k partitioned [12:02 PM] <domas> meh, we're talking 10% perf here [12:03 PM] <domas> and we're not overloading i/o anyway [12:03 PM] <Jeff_Green> domas: could you email/wiki/something us some notes on your tweaks? [12:03 PM] <domas> jeff_green: there're not that many! [12:03 PM] <domas> but I can try! [12:03 PM] <Jeff_Green> i saw we tweak raid-related stripe stuff only? [12:04 PM] <Jeff_Green> at CL we ended up tweaking only agcount (to 32) and the usual mount options, curious what/why you tweak [12:04 PM] <domas> jeff_green a/g doesn't matter much, we have just one file that is big enough =) [12:05 PM] <domas> jeff_green: stripe alignment is to avoid multiple reads for one block [12:08 PM] <Jeff_Green> how does that interact with striping in hardware RAID? [12:10 PM] <domas> well [12:10 PM] <domas> if you don't align files on stripe boundaries [12:10 PM] <domas> if a file is made out of 16k blocks [12:10 PM] <domas> and you have 64k stripe [12:10 PM] <domas> and it is not aligned [12:11 PM] <domas> so, 25% of blocks will need two I/Os instead of one [12:11 PM] <domas> because the block will reside on two disks [12:11 PM] <domas> now, if you align them all on stripe boundary, all blocks are residing just on one disk [12:11 PM] <domas> (I'm not counting mirrors) [12:12 PM] <domas> back in the day it was much more painful, as we had to align partitions too [12:12 PM] <domas> (which is what xfsfix was mostly for) [12:12 PM] <domas> we were editing partitiontable with xfsfix before [12:12 PM] <Jeff_Green> ok. I'm going to apply this to db1040 as a comprehension exercise