We employ a data-sparse, recursive matrix representation, so-called -matrices, for the efficient treatment of discretized integral operators. We obtain this format using local tensor product interpolants of the kernel function and replacing high-order approximations with piecewise lower-order ones. The scheme has optimal, i.e., linear, complexity in the memory requirement and time for the matrix-vector multiplication. We present an error analysis for integral operators of order zero. In particular, we show that the optimal convergence (h) is retained for the classical double layer potential discretized with piecewise constant functions.