A parallel system relies on both process scheduling and I/O scheduling for
efficient use of resources, and a program's performance hinges on the resource
on which it is bottlenecked. Existing process schedulers and I/O schedulers
are independent. However, when the bottleneck is I/O, there is an opportunity
to alleviate it via cooperation between the I/O and process schedulers: the
service efficiency of I/O requests can be highly dependent on their issuance
order, which in turn is heavily influenced by process scheduling. We propose a
data-driven program execution mode in which process scheduling and request
issuance are coordinated to facilitate effective I/O scheduling for high disk
efficiency. Our implementation, Dual Par, uses process suspension and
resumption, as well as pre-execution and prefetching techniques, to provide a
pool of pre-sorted requests to the I/O scheduler. This data-driven execution
mode is enabled when I/O is detected to be the bottleneck, otherwise the
program runs in the normal computation-driven mode. Dual Par is implemented in
the MPICH2 MPI-IO library for MPI programs to coordinate I/O service and
process execution. Our experiments on a 120-node cluster using the PVFS2 file
system show that Dual Par can increase system I/O throughput by 31% on
average, compared to existing MPI-IO with or without using collective I/O.