Higher storage capacity and throughput needs on High Performance Computing demands utilization of Lustre file system not only as primary (scratch) storage, but also as long-term and permanent shared filesystem and archiving system, requiring features and capabilities beyond the traditional HPC storage.
The industry’s needs require compute, network and storage resources in a converged datacenter with multiple virtualization and security techniques that simplify the overall architecture yet reducing the operational costs. Many organizations have started build these private clouds on HPC systems leveraging Openstack, and providing a fusion between HPC and Cloud environment.
OpenStack is an open source software stack used to build and manage virtualization of Cloud, HPC and BigData, private and public, infrastructure. Openstack software consists of many components that manages compute, network and storage resources via integrated set of APIs and tools such GUI dashboards, etc. On the storage layer Openstack supports two types of implementations: SWIFT, utilized for object storage and Cinder for block storage access.
Cinder drivers varies according to the storage layer to be used and there are multiple implementations available that relies on the storage hardware, local filesystem and network filesystems to be used. However, there’s no Cinder driver for Lustre yet.
DDN has ported the OpenStack Cinder driver for Lustre and submitted all the patches to the upstream Openstack’s branch, contributing to both Lustre and Openstack community. Lustre Cinder driver provides a block device storage service for OpenStack. OpenStack’s administrators are now capable to create block devices for virtual machines with Openstack using the Lustre Cinder driver, thus storing VMs images on a parallel, high performance, file system.
In this talk we will present some use cases of Openstack with Lustre Cinder driver and demonstrate how it works and what’s the current functionalities. The presentation will also cover the design of Lustre cinder driver and some preliminary benchmark results.
