Linear Tape File System

From Wikipedia, the free encyclopedia

The Linear Tape File System (LTFS) is a file system that allows files stored on magnetic tape to be accessed in a similar fashion to those on disk or removable flash drives. It requires both a specific format of data on the tape media and software to provide a file system interface to the data.

The technology, based around a self-describing tape format developed by IBM, was adopted by the LTO Consortium in 2010.

Magnetic tape data storage has been used for over 70 years, with data being stored on tape using an extremely simple filesystem. A tape is presented to the user as a sequence of unnamed data "files", separated by "filemarks". A file to be read can be selected by its number in the sequence. A file to be written can be appended after the last filemark.

There are no requirements to how the data within a "file" is to be organized. As a result, tapes typically did not hold OS filesystem metadata, or if they did, it was not in a form easy to access or modify independent of the file content data. External databases were often used to index file metadata (file names, permissions, timestamps, directory hierarchy, etc.), but these external databases are commonly specific to backup or archiving tools and were generally not designed for interoperability.

In Unix-like systems, there is the tar (tape archive) interoperable standard which combines files and metadata into a format suitable for storing as a single file on tape. While effective for archival and interchange purposes, tar does not provide a normal filesystem interface or characteristics.

On Microsoft Windows systems, several tools, including Backup Exec and NTBackup, use the Microsoft Tape Format (mtf) for storing data on tape. While effective for archival and interchange purposes, mtf is not a normal filesystem nor does it provide such characteristics.

LTFS was born of the idea to have a standardized method for a tape to store data files, and an index with metadata, along with a standardized way to mount and use the tape as if it had a normal filesystem. By partitioning a tape into areas that can be written to independently, one partition can store the index, while the actual data can be written to the other.

LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at the NAB show in 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release.

The LTFS Format Specification and filesystem implementation were released on April 12, 2010 with the support of IBM, HP, Quantum, and the LTO Consortium.^[1]

LTFS v2.0.0 was released in March 2011, improving the text to clarify and remove ambiguity. It also added support for sparse files; persistent file identifiers; virtual extended attributes for filesystem metadata and control - and defined minimum and recommended blocksize values for LTFS volumes, for compatibility across various HBA hardware implementations.

LTFS v2.2.0 was released in December 2013. It was the first version to become an ISO standard (20919:2016).

LTFS v2.5.1 was released in May 2019. It became the second version of the ISO standard (20919:2021). Version 2.5 contained fairly major updates, as it was the first version to define Incremental (sparse) Indexes.

The ISO/IEC 20919:2021 standard defines the LTFS Format requirements for interchanged media that claims LTFS compliance. It defines the data format, independent of the physical storage media and the software commands format, to make data truly interchangeable. The ISO standard was prepared by SNIA. It is based on LTFS v2.5.1, and was adopted to ISO by a joint technical committee ISO/IEC JTC 1 Information Technology.

The SNIA workgroup continues to develop LTFS and release updates. Version 2.0.0 defines rules for how the version number may change in future, and how compatibility is maintained across varying implementations. All implementations must:

• correctly read media that was compliant with any prior version
• write media that is compliant with the version they claim compliance with

LTFS Format Specification History

Version	Published	ISO/IEC	Conforming Software
1.0[2]	April 2010	—	IBM Long Term File System (LTFS) v1.0.0, v1.0.1 HP Linear Tape File System (LTFS) v1.0.0, v1.1.0
2.0.0[3]	March 2011	—	IBM Linear Tape File System - Single Drive Edition (LTFS-SDE) v1.2.0 Oracle StorageTek Linear Tape File System, Open Edition v1.0.0
2.1.0[4]	October 2012	—	?
2.2.0[5]	July 2013	20919:2016	?
2.3.0[6]	March 2014	—	?
2.4.0[7]	November 2015	—	IBM Storage Archive - Single Drive Edition (LTFS-SDE) v2.4.7
2.5.0[8]	December 2018	20919:2021	?

In August 2012, SNIA announced^[9] that it was forming a TWG (Technical Work Group) to continue technical development of the specification. LTFS Format Specification v 2.1 is the baseline for the technical work and standards accreditation process; SNIA LTFS TWG members include HP, IBM, Oracle and Quantum.

While LTFS can make a tape appear to behave like a disk, it does not change the fundamentally sequential nature of tape. Files are always appended to the end of the tape. If a file is modified and overwritten or removed from the volume, the associated tape blocks used are not freed up, they are simply marked as unavailable and the used volume capacity is not recovered. Data is only deleted and capacity recovered if the whole tape is reformatted.^{[citation needed ]}

In spite of these disadvantages, there are several uses case where LTFS formatted tape is superior to spinning disk and other data storage technology. While LTO seek times can range from 10 to 100 seconds, the streaming data transfer rate can match or exceed spinning-disk data transfer rates. For example, LTO-9 is capable of a continuous transfer speed of 400 MB/s while spinning HDDs typically max out at 200–250 MB/s for the first 30% of their capacity.

Additionally, tape cartridges are easily transportable and recent tape formats hold far more data than any other removable data storage technology. The ability to copy a large file or a large selection of files (up to 30TB for LTO-10) to an LTFS formatted tape allows for easy exchange of data with a collaborator, or for saving of an archival copy.

Since LTFS is an open standard, LTFS formatted tapes are usable by a wide variety of computing systems and operating systems, avoiding the incompatibilities experienced with proprietary tape archive formats.^[10]

Tape drives manufacturers often offer their own versions of software conforming to the LTFS standard. There are also three different editions, one for Single Drives based on the LTFS Reference Implementation, one for Tape Libraries, and one for integration into a Enterprise storage platform.

The base form of LTFS allows tapes to be formatted as an LTFS volume, and for these volumes to be mounted - and users and applications access files and directories stored on the tape directly, including drag-and-drop of files.

• Current Reference implementation; Free/Open Source
• IBM Storage Archive - Single Drive Edition; initially released as "IBM Long Term File System"
• HPE StoreOpen Software LTFS; Free download, compatible with Windows, Mac, and Linux.
• Quantum Linear Tape File System; Linux and macOS support. Windows versions of Quantum LTFS utilities were discontinued July 2023; Quantum recommends Windows users to use TeraCopy instead.
• Oracle's StorageTek Linear Tape File System, Open Edition; Expanded support for LTFS to Oracle’s StorageTek T10000C and T10000D tape drives and Solaris.^[11]

An extension of the base form of LTFS, LTFS-LE presents a tape library full of LTFS tapes as online directories and mounts and unmounts tapes as needed. Each LTFS-formatted tape cartridge in the library appears as a separate folder under the filesystem mount point and the user or application can navigate into each of these folders to access the files stored on each tape. The LTFS-LE software automatically controls the tape library robotics to load and unload the necessary LTFS volumes.

• IBM Storage Archive - Library Edition
• Oracle's StorageTek Linear Tape File System, Library Edition; Supports the StorageTek SL8500 Modular Library System, the StorageTek SL3000 Modular Library System, and the StorageTek SL150 Modular Tape.^[12]

The Enterprise Edition integrates a large cluster filesystem, IBM's GPFS, with LTFS Library Edition, to expand a large online file system with an automatically managed tape tier. Files may be automatically migrated from disks to tapes when certain criteria are met, such as extended periods of inactivity, and recovered back to disk automatically when a user try to access them.

• IBM Storage Archive - Enterprise Edition

• HP DAT-160 and DAT-320

Support included in HP's 2017 release of their fork of the original LTFS codebase. ^[13] .

• IBM: TS1140, TS1150, TS1155, TS1160, TS1170^{[14] [15]}
• Oracle StorageTek: T10000C and T10000D ^[16]

• HP/HPE, IBM, Quantum, and Tandberg: LTO-5 to LTO-10

A full set of vendors are listed at LTO website.^[17]

• Thought Equity Motion ^[18] is executing a major film digitization and preservation project for the EYE Film Institute Netherlands. The project involves scanning more than 150 million discrete DPX files and storing them on LTO Gen5 using the LTFS format.^[19] More than 1 petabyte of film will be^{[needs update ]} scanned and archived over two years (2010–2012).

• IBM LTFS technology received a Pick Hit Award from Broadcast Engineering at NAB 2011.^[20]
• IBM and FOX Networks received an Engineering Emmy Award in 2011 for a project that uses LTFS to store, exchange, and archive video content.^[21]
• IBM received the 2011 Hollywood Post-Alliance (HPA) Engineering Excellence Award.^[22]

As of standard version 2.5.1, LTFS does not support hard links.^[8]

When files are deleted, they become invisible to the user. However, the space occupied by a file is not freed. Because of this, it is possible to "roll back" the tape to an earlier state, in order to recover erroneously deleted (or incorrectly updated) files. ^[8] To free up space a tape needs to be re-formatted.

• LTFS at LTO
• LTFS at SNIA
• LTFS for Dummies Book
• Implementations:
  • LinearTapeFileSystem on GitHub (LTFS Reference Implementation for stand alone tape drive)
  • IBM Spectrum Archive - IBM Spectrum Archive Single Drive Edition (SDE)
  • Oracle Tape Storage - Oracle Linear Tape File System, Open Edition
  • HP Linear Tape File System - HPE LTFS Software
  • Quantum Linear Tape File System - Quantum LTFS Source
• LTFS-Data-Management on GitHub

Retrieved from "https://en.wikipedia.org/w/index.php?title=Linear_Tape_File_System&oldid=1317497504"

Categories:

• Computer file systems
• Computer storage tape media
• IBM file systems

Hidden categories:

• Webarchive template archiveis links
• Articles with short description
• Short description is different from Wikidata
• Use dmy dates from September 2019
• All articles with unsourced statements
• Articles with unsourced statements from October 2015
• Wikipedia articles in need of updating from February 2021
• All Wikipedia articles in need of updating