In some ways I wish I had considered ZFS more when I was building my file server.  I wanted a 12 drive card so I was limited in my choices and might not have found a 12 port card supported by Solaris.  Also not knowing Solaris I was unsure about having all of my files on a file system that only Solaris based distros can support reliably.  If ZFS didn’t work out I would have to move back to Linux, and that would be hard with as much data as I was storing.

The thing that is a real bummer is that I am now finding myself needing to expand the array and wanting to slim the setup a little bit.  What I mean by that is that I want to fill the card so that I am maxed out at the 12 drives, I want to buy the battery for the RAID card and I want to turn off journaling and just go with ext2.  So in the end I am really having to reevaluate how I want my file server to be.

The other option, if I can get the card to work with Solaris/OpenSolaris is ditching the hardware raid funtionality of the card and setup ZFS instead.  Right now, after tuning, I am only getting abut 350mb/s according to hdparm.  With each drive by itself getting 109mb/s on average, 8 of them should be a bit higher than 350.  I feel that having the card handle a lot of things is causing this problem and bypassing that might speed things up.  Again this is all banking on if I can get Solaris to run with that card, which I don’t think is going to happen.  I am going to research some cards again and see what 12 port non-raid cards are available, but I don’t think I want to spend all that money again.

You can’t beat this though:

zpool create raid2z c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0 c0t6d0 c0t7d0 c0t8d0

That just bound all 8 disks in RAID6 and created the file system. No more fdisk and format and mkfs.ext3, etc. Simply beautiful!