I’ve been playing around with various flavors of Solaris, OpenSolaris and the Solaris Community Express Edition for a while now in a professional capacity, trying to see exactly how to position the new ZFS and storage management options for use in large virtualization environments.
From Enterprise to Home
The other aspect I’ve been looking at is how this kind of technology can be effectively used in today’s digital home where the volume of digital media is growing constantly. More details on my current environment in the next article, but suffice it to say that I have about a terabyte of media that I don’t want to lose, plus all of the ancillary data arround it. Scalability is a two way street - you should be able to scale up to the multi-petabyte environments as well as scale down to the small home office.
Build it myself? I’ve been looking at the various NAS options out there for a while but have a really really hard time justifying the cost for the limited functionality and expandability. The closest thing to what I wanted is the latest LaCie 5 disk NAS/RAID box which is really nice but horrendously overpriced IMHO. The Drobo was my other option, but again overpriced and while I have immense respect for the work that Drobo has put into automating RAID management it still lacks a lot of the other intelligence of the ZFS stack like snapshots, iSCSI and data healing. (BTW, there’s nothing that exotic about the Drobo’s capabilities, it’s just some RAID operations that they’ve automated very nicely in their controller).
Building it myself seemed to be the appropriate solution. Which leads me to…
The state of the DIY PC market
Good god, this is insane.
I have been spending an inordinate amount of time browsing the mass of options concerning all of these components. I have yet to find a site with a filtering engine to let me find a motherboard adapted to my requirements by specifying my base requirements like form factor, gigabit ethernet, at least 4 onboard SATA ports, whether or not the SATA ports are port multiplier compatible (in fact, that tidbit of information is pretty much impossible to come by on any of the specifications pages), etc. I spent hours browsing the various stores trying to understand the variations, often with missing information (LAN port is not enough detail - I need to know if it’s 100Mbps or 1000Mbps, and jumbo frame support would be nice to know as well). So from a board that I found on a store site, it’s off to the mfg’s site for more detail, then onto the various review sites to better understand the idiosyncracies of the model, repeat ad naseaum …
Then there’s the state of the processor market. Do you have any idea just how many processors models that Intel and AMD make? Again, insane. A huge learning phase is required in order to understand the different performance profiles, socket connections, power consumption so that I can make some kind of correspondence between the prices and what I’m buying and why.
Ditto for the cases, and then the massive confusion surrounding the different clock speeds of the various components, front side bus, CPU, memory and which combinations are compatible and what the performance side effects are of combining things badly.
On the pre-built side of the house from major manufacturers like DELL and HP, you either buy a POS entry level PC which would have been passable since I could transplant it into another case or a workstation class machine (way overpriced compared to a build your own). In both cases you can’t find any real detail regarding the capabilities of the motherboards. How many internal SATA ports? Power supply wattage? (sometimes), case configuration, available disk bays, what kind of PCI slots?
I finally ended up buying a PC on eBay for 350€ with the following configuration:
Case: Antec P182. A good fit for my uses as it’s sufficiently large to hold a pile of disks, fairly well rated in terms of quietness.
Motherboard: ASUS P5K. A nice board with the essentials for my project, 5 onboard SATA ports (I would have preferred 6, but hey), 4 slots for memory to allow for future expansion, enough PCI slots to be able to add in more SATA cards later, and onboard gigabit ethernet.*
Memory: 2Gb on two slots, so I can upgrade without throwing away the initial investment.
Processor: Intel Core 2 Duo 2.33Ghz. A decent enough multicore processor - I don’t need the fastest thing on the planet but it’s definitely useful to have the multicore design as OpenSolaris and ZFS are fully capable of taking advantage of its multithreading abilities.
Graphics card: nVidia GeForce 8800GT… Massive overkill for a server, but I can always hope that Sun will implement OpenCL down the line in response to Apple implementing ZFS so it will end up being useful for something.
I re-used one 1Tb drive I had in the house and bought three new ones at 89€ each for a total investment of 617€.
I count myself lucky to have stumbled on the PC on eBay that is pretty much perfectly aligned with what I wanted to do with it. This machine is probably overkill for its immediate intended use, but I’ll look into using it for other stuff eventually, but the upshot is that you can get a basic PC for less than you’d pay for a prepackaged NAS box and have considerably more power and flexibility using OpenSolaris. Granted it’s bigger and uglier, but it’s quiet and can be tucked into a corner.
I really wanted to use my old Shuttle, but for some reason it core dumps on OpenSolaris.
Why not wait for Snow Leopard?
Apple currently offers a read-only port of ZFS on Leopard, and a more complete port should be available in Snow Leopard, but the actual state of development shows that it will not contain all of the features that I’m looking for (notably the hybrid storage pools). The other issue is that in terms of form factor, I wanted something that I could continue to add disks into easily. My current approach of stackable external disks has been OK, but doesn’t scale up when you look under the desk and see how many power cables this requires.
Next up - the current environment and how it fits in…
* Except that the onboard Attansic network card is not yet natively supported in OpenSolaris. I did find source code for the drivers, but the basic install doesn’t include GCC, and since I have no network… you see where this is going. I finally did get drivers compiled in a virtual machine and then installed via USB key.