We consider two algorithms and seek to verify the claim that one has superior performance with respect to some property. We start with finding a measure that represents the ‘goodness’ of the property of interest, follow with determining a suitable hypothesis test and the number of images and, finally, with interpreting the results of the experiment. We show how the use of a statistical framework can allow us not only to draw conclusions based on a sound methodology but also to provide a confidence level for these conclusions. We give a concrete application of the approach to the problem of comparing two spread spectrum watermarking algorithms with respect to their robustness to JPEG2000 compression. We discuss the intricacies of the choices at various stages of this approach, and their effects on the results.