Continued from The conundrum of enterprise storage
There are a number of ways to ease the problems that have been created by the disparity of surface vs IO. One way is to transform your IO workload into something that is more sequential and less random. Currently, the best example that I’ve found of this solution are the products made by Datacore (SANMelody and SANSymphony). They interpose themselves between the storage and the servers and behave like native storage to the servers. But they add a very intelligent caching algorithm on top of a massive and (relatively) inexpensive cache by using regular RAM in the server which can go as high as 128Gb on current 32-bit architectures (maybe even higher, but then it’s no longer inexpensive).
All incoming write IO operations are held in the cache until they determine that there’s enough data for a given disk system to send a large write operation in the place of many smaller, discrete operations. The net result is that the random IO workload is transformed into something that’s a lot more sequential as far as the physical disk subsystems are concerned. The performance gains that I’ve observed have ranged as high as 3 to 1 under extreme stress comparing high-end fibre solutions against SAS backed local storage.
Additionally, your storage is assigned dynamically across pools of similar storage sources. So you create a number of 2Tb RAID sets and then they are concatenated into a pool of storage and the software distributes the allocation across all of the members of the pool. It’s not a RAID0, but in many ways it gives you many of the advantages since in a random IO workload, you’ll be making more physical spindles work at the same time.
The only downside of this approach is that you are still allocating storage in a rather gross manner and often high IO data is cohabiting with lower IO data because you don’t have the granularity of control within a given filesystem. In the virtualisation world, you may want to put everything associated with a given virtual machine on the same LUN in order to facilitate the use of backup tools like VMware Consolidated Backup which work better when all of the virtual disks cohabit on the same LUN. So you end up with situations where you’ve consolidated a system disk, database and logs on pools based on high performance disks since you want the application to perform well, but your level of granularity is non optimal since you’ve got your OS and application files in the same pool.
Tiered storage
There’s a lot of talk about tiered storage these days but the reality of the situation is that in most cases this simply means that you can mix disparate disk types in the same physical chassis. You still end up with the lack of granularity mentioned above. It’s useful from the standpoint that you buy one disk chassis and mix and match your SAS and SATA disks, but it’s far from being a panacea.
The only solution that I’ve seen to date that efficiently addresses these issues and is sufficiently open to accept new technologies is the Automated Tiered Storage offered by Compellent. In their solution, the abstraction of the storage goes even further than that of Datacore. Everything in the SAN is part of a large pool instead of multiple specific pools, but there is the additional notion of tiered performance by type of disk. From the servers’ standpoint, they see a LUN, while behind the scenes the actual disk blocks representing the data can be split across multiple types of storage and RAID types.
The intelligence comes from the tracking done by the SAN regarding the use of individual blocks of data. Blocks that are frequently solicited are positioned on higher performance disks, and those that are requested less often are migrated to higher capacity, lower cost disks. Since this is done at the block level and not the volume or filesystem level, it permits an exceedingly high granularity of control over which blocks need the higher IO disks and which don’t in a manner that is transparent to the servers requesting or writing the data. For example, the blocks representing the indices of a database will probably be positioned on the highest performing tier of disk, while the bulk of the actual data will be found on a lower tier. Since the movement between tiers is dynamic, your log files will be written out to high performance disk and then invisibly migrated to lower cost disks as their relevance diminishes, without changing anything about how you architect your applications.
This has the additional side effect of diminishing immensely the workload on the storage administration. You longer need to analyze in detail the performance profile of every single application in order to assign the most efficient RAID set or worry about how to integrate complex partitioning schemes with your backup systems.
New storage devices
This dynamic multi tier approach is also perfectly positioned to take advantage of the latest generation of high capacity SSHD products that are just starting to hit the market. By tracking the demands and moving frequently solicited blocks to SSHD, access will be practically instantaneous. But unlike most other solutions, you are not obligated to over purchase this type of storage because you are not mixing potentially stale data blocks on your most expensive storage. This is ideal for database logs, incoming email etc. which will be committed practically instantly but then moved down a tier, freeing up space on the SSHD tier for the next batch of incoming data. Every potential workload benefits from this approach. The blocks representing swap files with high IO volume will be able to respond at almost the same speed as RAM physically installed in the servers.
Managing Growth
I’ve observed that in most environments, the volume of truly active data is relatively stable, while there is an every growing volume of stale (but still important!) data. The dynamic tiered approach permits you to buy enough high performance storage to address the needs of your active data. You manage growth by only buying the most cost effective storage at the bottom tier. Obviously, if there is a massive change in the corporate use of data you may need to invest in more high performance storage, but the investment will be strictly limited as this additional storage will only be used by the blocks that absolutely require it.
On the purely pragmatic end of things, you can get some of the same basic benefits with the Datacore solution and a lot of low cost disks. By simply throwing enough spindles into the mix at the back end, you end run much of these issues by having IOps to spare. This approach does require massive disk density in your storage environment in order to be viable and not eat up all of your available rack space. Traditional front loaded disk arrays are horribly space-inefficient here so products like the NexSan Satabeast are ideal complements to this approach. The only downside is that you won’t truly be able to optimally take advantage of the upcoming move towards solid state storage the same way you could with some kind of automated tiered storage.