After a Lustre file system is put in production, and the usage model fluctuates with the variability of applications and users creating and deleting files overtime, available storage capacity tends to decrease while file system fragmentation increases. This affects the overall performance throughput, bandwidth, compared to a pristine file system, or a file system which is just fully populated. Seagate will discuss a study around both capacity and fragmentation, including the methodologies, nomenclature, and tools used to study both capacity and the introduction of fragmentation at different capacity points to analyze the overall throughput performance impact. The talk will illustrate how, at various percentages of capacity, the largest impact is the amount of fragmentation that exists, and not just the utilized capacity of the file system.
The overall determination is that despite some popular belief, performance impacts can occur at very low percentage of utilization depending on patterns of utilization and fragmentation that may be generated by a file system's overall operational production utilization. As such modeling behaviors, RFI’s and RFP’s should be modified to consider factors outside a pristine file system to more properly reflect real world operational requirements for customers. Customers can thus consider the inclusion of such factors in their solicitation for file system proposals. Vendors in turn, will be encouraged to perform product benchmarks that are more realistic to both fragmentation and capacity as major constraints in Lustre file system utilization for responses to said requirements, irrespective of whether those requirements are aligned to a more traditional Lustre on disk filesystem, LDISKFS, or for alternate technologies such as zfs.
