Recently many researchers are interested in objects of interest in an image, which are useful for efficient image matching based on them and bridging the semantic gap between higher concept of users and low-level image features. In this paper, we introduce a computational approach that classifies an object of interest into a natural or a man-made class, which can be of great interest for semantic indexing applications processing very large image databases. We first show that Gabor energy maps for man-made objects tend to have dominant orientation features through analysis of Gabor filtering results for many object images. Then a sum of Gabor orientation energy differences is proposed as a classification measure, which shows a classification accuracy of 82.9% in a test with 2,600 object images.