This paper presents an attempt at ‘non-verbal mapping’ between music and images. We use physical parameters of key, height and timbre as sound, and hue, brightness and chroma as color, to clarify their direct correspondence. First we derive a mapping rule between sound and color from those with such special abilities as ‘colored hearing’. Next we apply the mapping to everyday people using a paired comparison test and key identification training, and we find similar phenomena to colored hearing among everyday people. The experimental result shows a possibility that they also have potential of ability of sound and color mapping.