Both Linear Discriminant Analysis and Support Vector Machines compute hyperplanes that are optimal with respect to their individual objectives. However, there can be vast differences in performance between the two techniques depending on the extent to which their respective assumptions agree with problems at hand. In this paper we compare the two techniques analytically and experimentally using a number of data sets. For analytical comparison purposes, a unified representation is developed and a metric of optimality is proposed.