Many large-scale production parallel programs often run for a very long time and require data checkpoint periodically to save
the state of the computation for program restart and/or tracing the progress. Such a write-only pattern has become a dominant
part of an application’s I/O workload and implies the importance of its optimization. Existing approaches for write-behind
data buffering at both file system and MPI I/O levels have been proposed, but challenges still exist for efficient design
to maintain data consistency among distributed buffers. To address this problem, we propose a buffering scheme that coordinates
the compute processes to achieve the consistency control. Different from other earlier work, our design can be applied to
files opened in read-write mode and handle the patterns with mixed MPI collective and independent I/O calls. Performance evaluation
using BTIO and FLASH IO benchmarks is presented, which shows a significant improvement over the method without buffering.
Keywords Write behind - MPI I/O - file consistency - data buffering - I/O thread