We consider embedded Runge-Kutta (RK) methods for the solution of ordinary differential equations (ODEs) arising from space discretizations of partial differential equations and study their efficient implementation on modern microprocessors with memory hierarchies. For those systems of ODEs, we present a block oriented pipelining approach with diagonal sweeps over the stage and approximation vector computations of RK methods. Comparisons with other efficient implementations show that this pipelining technique improves the locality behavior considerably. Runtime experiments are performed with the DOPRI5 method.