I’ve been reading a number of articles recently concerning LTFS as an opportunity for keeping tape alive as a new tier of data in big data environments. There are even some very interesting new appliances on the market that target this type of usage like Cache-a. As an archival tool, I find this a potentially useful approach. But as a near line cloud storage solution I have a few quibbles.
The more time I spend evaluating storage solutions, I keep coming back to a point that is currently only being addressed properly by ZFS and ReFS: guaranteed data integrity. In our current world of virtual-everything, you can pretty much be guaranteed that every single layer of the stack is lying in some way to the others and we have imperfect implementations and to top it off we have bit rot to deal with.
LTFS looks interesting as a method of maintaining non critical data available, albeit with high latency compared to a disk solution. But there is no accounting for end to end data integrity, nor any kind of internal redundancy other than that integrated into the tapes themselves. Doing any kind of RAID on tape volumes strikes me as an exceedingly futile gesture so redundancy is out as are any of the ReFS/ZFS auto-correction approaches that require parity data on a separate physical device.
So from this point of view, unless I have yet another copy of all of this data elsewhere, I’m presenting a risk of data loss or compromised data that I can’t identify. And even then, just how do I audit and verify the integrity of the different copies?
Moving data around
The other point being raised in favor of the use of tapes and LTFS is the possibility of being able to ship them to another site and make that data available relatively easily when WAN replication techniques are not viable due to the volume of data. If we start from the hypotheses of the traditional storage bay with traditional RAID controllers that require near-identical hardware at each end, obviously the tape solution is more interesting. But as we move towards commodity JBOD systems with on-disk structures like ZFS where, like LTFS, everything required to mount the filesystem is self-contained, shipping disks instead of tape is an entirely viable approach.
Then the question of data density comes into play. Currently LTO has a native capacity of 1.5Tb/tape. I’m going to leave compression out of consideration here since I can just as easily compress data to disk[1], and possibly go further with ZFS pool level deduplication.
Roughly, we have the following sizes and capacity measurements today in 2012:
MediaVolumeDataData DensityLTO-5248 cm31,5 Tb6.2 Gb/cm33.5” disk376 cm33 Tb8.2 Gb/cm32.5” disk105 cm31 Tb9.7 Gb/cm32.5” disk105 cm32 Tb19.5 Gb/cm3Update: Western Digital just announced a 2 Tb 2.5” drive.
Which means that under the current state of affairs, on a measurement of data by volume, using 2.5” disks is the most efficient means of transporting data with a 50% (Update: 300% with the new 2 Tb drives) advantage over LTO-5. Granted that considering data by weight the tapes will probably come out ahead if that’s the primary factor for shipping costs, although the new 2Tb 2.5 drives are probably in the same ballpark.
The secondary issue is obviously cost/Gb/cm3 where tape wins against the new 2Tb 2.5” drives, but if history has shows anything, it’s that the price for disk storage goes down very rapidly due to economies of scale and the overall size of the market.
Of course, LTO-6 will improve on this greatly, but will require equipment replacement, whereas higher density disk drives can plug into current equipment with a considerably longer lifecycle. Plus you have the advantage of being able to profit from incremental improvements in disk technologies with existing equipment.
Other use cases
I can think of a number of places where this would be exceedingly useful in some cloud environments. If your data is primarily self-correcting lossy data, and there’s more data than than is actively being solicited, this lends itself quite well to this kind of architecture. I’m thinking photo or video sharing sites where there are tons of truly cold data that has no SLA attached, and even if a bit flips here or there the data structures are sufficiently forgiving that it will probably pass unnoticed. Photos are especially well adapted since I can keep smaller versions on more expensive disk to present to users while loading up original hi-res versions in the background.
But there are unforgiving binary formats that will simply choke if presented with a data structure that does not conform to its expectations. A flipped bit results in completely useless data without manual review and correction, which may not even be possible.
Longevity
A side topic that often comes up is the much touted tape longevity which goes up to 30 years. Of course this is mostly a straw man argument since pretty much any data stored for 30 years will be completely unusable without the accompanying applications that generated the data. Text and a few very widely implemented binary formats will always be usable to a certain extent, but beyond that don’t bet on much. XML is not the answer. docx, xlsx and the family of the current generation formats will be useless if computing and the interpreters have changed significantly in the interim.
Recent history
I’ve just recently extracted a pile of data I had archived in 2000 on an AIT-1 tape. Absolutely no problems reading with the data stored on the tape once I had built a machine with my 2000 era OS and an old copy of the backup software (Retrospect). Kudos to Retrospect for a solid product that stands the test of time. The tape was perfectly reliable despite being 12 years old and surviving several international moves under less than ideal conditions but I did have to jump through a few hoops finding an AIT drive and getting the software to recatalog the contents. Purely from that standpoint, LTFS is a phenomenal step in the right direction.
But then came the step of dredging through the files. Fortunately this was an archive of a series of Mac systems where most of the application bundles are completely independent of the OS and I could open up many of the files and it would magically find the appropriate application that had been restored as well. But then there are a number of files that I don’t have the associated applications any more. Anyone seen a copy of Cricket Draw 1.5 or MacDraw or MacDraft from the mid-90s?
So whatever happens, the longevity of the physical media is largely irrelevant since any archival plan must also include review and conversion cycles to ensure the exploitability of the data. In this case, archives must integrate a continuous or cyclical migration process to ensure their viability, maintainability and accessibility.
While researching the issue, I’ve been revisiting the possibility of bringing tape back into my home backup solution, but at end of the day, it’s cost prohibitive for small to medium size environments. I can easily externalize naked drives with inexpensive eSATA or USB connections with a high confidence that in a disaster recovery situation, I can easily rebuild from these sources and that the necessary hardware is available and affordable. Asynchronous replication is even easier to deal with, but requires two well connected sites, which is currently outside of my budget.
The initial hardware tape investment is just too high to be practical and the hardware availability in a disaster recovery adds complexity and cost to the process. That said, my setup is only managing about 6 Tb in the home lab and production. The cost viability of moving to tape seems to be interesting once you’ve gone over 30-50 Tb of primary storage.
[1] zfs set compression=on pool/volume or more aggressively zpool set compression=gzip-9 pool/volume