Limits of Work-Stealing Scheduling
Željko Vrba18, 19, Håvard Espeland18, 19, Pål Halvorsen18, 19 and Carsten Griwodz18, 19
| (18) |
Simula Research Laboratory, Oslo, |
| (19) |
Department of Informatics, University of Oslo, |
Abstract
The number of applications with many parallel cooperating processes is steadily increasing, and developing efficient runtimes
for their execution is an important task. Several frameworks have been developed, such as MapReduce and Dryad, but developing
scheduling mechanisms that take into account processing and communication requirements is hard. In this paper, we explore the limits of work stealing scheduler, which has empirically
been shown to perform well, and evaluate load-balancing based on graph partitioning as an orthogonal approach. All the algorithms
are implemented in our Nornir runtime system, and our experiments on a multi-core workstation machine show that the main cause
of performance degradation of work stealing is when very little processing time, which we quantify exactly, is performed per
message. This is the type of workload in which graph partitioning has the potential to achieve better performance than work-stealing.
References secured to subscribers.