An ever-increasing portfolio of storage media (NVDIMM, SSD, SMR, etc) has storage developers excited and storage users burdened. For storage developers, these storage media create exciting opportunities for us to build exotic tiered architectures and to brush off the old concepts we learned in school about prefetch, scheduling, victimization algorithms, starvation, etc. For storage users who'd prefer to focus on actual computational science, these storage tiers are pure frustration. Lustre users can either ignore these emerging tiered storage systems at their own performance peril, or they can attempt to learn how to use them. Unfortunately for users, these storage tiers are entering a period of rapid flux in which the multiple vendors appear nowhere near close to converging on a standard.
In this talk, we will present an overview of the various vendor offerings for tiered storage systems (colloquially called burst buffers). These include DDN's IME, Cray's DataWarp, IBM's eponymously named Burst Buffers, Seagate's NXD, and the EMC research prototype 2Tiers. The talk will provide analysis of areas in which the various systems have general commonality with only slight terminology differences such as staging mechanisms which are offered under many different names such as promote, persist, transfer, stage, and migrate. It will also discuss several interesting features unique to only a single system such as attaching key-values to a staging operation as well as querying paths to provide unique access to metadata services.
At the end of the presentation, the audience will be familiar with the general similarities and differences in the various tiered storage offerings. They will understand their functionality and usability and terminology. Although the presentation will absolutely not be an evaluation of the respective value of the various offerings, the audience will have the information needed for them to begin their own evaluation of which functions and features are most appropriate for their Lustre systems and workloads. This information will also be useful to the Lustre community as these concepts can be considered in the Luster user requirement gathering exercise.
