I/O intensive applications are a significant bottleneck for current HPC systems and they can have noticeable impact on the performance of an entire system’s workload. On general purposes HPC systems, the I/O nodes and the network that provisions them are resources that are shared by all jobs on the system. Applications thus have to compete for resources. The overall objective of the Next Generation I/O Project (NEXTGenIO) is to design and prototype a new, scalable, high-performance, energy efficient computing platform, designed to address the challenge of delivering the necessary scalable I/O performance to applications at the Exascale. I/O-wise, the architecture covers a novel multi-tier storage hierarchy ranging from a speed-focused, limited capacity NVRAM tier over I/O fabrics to a high-capacity storage tier built by a hierarchy of one or more solid state drive tiers and conventional storage, like hard disk drives and tape.
This presentation is about performance analysis techniques that will enable applications to take full advantage of the new memory and I/O models afforded by the new non-volatile memory technology that we expect to be seen soon in many systems. We will discuss the performance analysis opportunities and necessities for NVRAM-based I/O in combination with parallel filesystem I/O provided by a Lustre file system, and high level I/O libraries like NetCDF or HDF5. The overall goal of this presentation is to show the comprehensive performance analysis of a complex I/O stack using the NEXTGenIO system as an example.
