Image matching is an important problem in image processing and arises in such diverse fields as video compression, optical character recognition, medical imaging, watermarking etc. Given two images, image matching determines a transformation that changes the first image such that it most closely resembles the second. Common approaches require either exponential time, or find only an approximate solution, even when only rotations and scalings are allowed. This paper provides the first general polynomial time algorithm to find the exact solution to the image matching problem under projective, affine or linear transformations. Subsequently, nontrivial lower bounds on the number of different transformed images are given which roughly induce the complexity of image matching under the three classes of transformations.