LTFS is a standard, adopted by the LTO consortium, as a way to format tapes that makes them both standards-based and self-describing.
How is LTFS self-describing? It happened when LTO (Linear Tape Open) technology moved to a two-partition format. Up until that time, all data-tape had a single partition, and therefore any indexing of the tape either occurred on an external system, or was inserted using special index files that were themselves simply data on the tape (TAR files). As a storage medium, these tapes were considered much less brittle than non-LTFS tape in that there were external dependencies (e.g. back up software) that was required to maintain an index of the block-based data on the tape. If the dependency broke (if the database was lost or corrupted) the tape became virtually useless. Therefore, although tape was being used in various applications, the systems required a relatively high level of care and feeding, and were prone to problems without the proper management. Unfortunately, stories of “lost” data on tapes were all too common.
With the advent of LTO-5 and the second partition, a system was developed for using the second partition as a full index – functioning in much the same fashion as a file-index on a USB drive. LTFS as a format was quickly accepted by the LTO consortium (IBM, HP, Quantum) and was adopted as a standard in 2010.
The standard specified the exact format for an LTFS tape – using dual partitions and an XML index – that made the tape data-cartridges “self-describing” and able to operate independently of the system that wrote them.
This was groundbreaking in the tape data-cartridge world. LTFS allows each individual tape cartridge to exist on its own. It can be read independently by both tape robots and individual workstations or laptops (if equipped with the proper hardware.) And the system reading the data can run on Windows, Apple OSX, or Linux systems.
In other words, LTFS allows the tape data-cartridge medium to be as portable as USB or Firewire drives. Therefore tape data-cartridges can act independently, AND they can also be used in large automation robots, encrypted if security requires it, and they can function within the realm of any application, at any storage volume, where tape has traditionally been used.
In both long and short-term archive applications, the advantages of LTFS have become obvious:
Since each individual tape holds a complete index of its content, LTFS archives can be engineered with a “dual index.” One index can be maintained on a central system for ease of finding content across tapes, but if the central index is ever lost or corrupted, it can be rebuilt entirely from the data on the tapes. This feature makes LTFS archives secure and highly resilient.
LTFS tapes can exist in large (and small) automation robots, but each tape can also be removed from the system and its data accessed individually. Of course system administrators can limit this function, or require encryption on tape-sets. However, there are many applications where tape portability can become important. For example, the tapes from even a multi-petabyte archive can be removed and relocated if necessary. Additionally, single tapes can easily be removed (or copied and removed) from a system and presented to an analyst for further study. LTFS Portability also allows for the prospect of storing tape cartridges offline, since the straightforward file system of an LTFS tape allows any offline tapes to be easily reconsolidated into a larger system.
LTFS (LTO) tapes can last for decades, as has been noted. This, coupled with the ability of automated procedures for checking tape health, make LTFS a reliable technology for Long-Term archives. Additionally, if larger volume cartridges become available, LTFS will allow the consolidation of these files with an ease-of-use that far surpasses any other tape systems today. Additionally, Champions EchoLeaf LTFS line can create redundant copies of data either on separate tapes or at separate locations – transparently to the host applications. And these duplication strategies only really make sense with the low cost of LTO / LTFS data-cartridges.
LTFS offers a certain elegance and simplicity to data archiving. Both files and file structures are preserved with LTFS.