Heuristic methods for the rejection of noisy training examples in the support vector machine (SVM) are introduced. Rejection of training errors, either offline or online, rsults in a sparser model that is less affected by noisy data. A simple offline heuristic provides sparser models with similar generalization performance to the standard SVM, at the expense of longer training times. An online approximation of this heuristic reduces training time and provides a sparser model than the SVM with a slight decrease in generalization performance.