This study developed two testing devices, namely the animation-based test (ABT) and the graphic-based test (GBT) in the area of earth sciences covering four domains that ranged from astronomy, meteorology, oceanography to geology. Both the students’ achievements of and their attitudes toward ABT compared to GBT were investigated. The purposes of this study were fourfold as follows: (1) to examine the validity and the reliability of ABT, (2) to compare the difference of ABT and GBT in student achievements, (3) to investigate the impact of ABT versus GBT on student achievements with different levels of prior knowledge and (4) to explore the ABT participants’ attitudes toward ABT in comparison with GBT. A total of 314 students, divided into two groups, participated in the study. Upon completion of the test, the students who took the ABT were given the survey, Attitude toward Animated Assessment Scale (AAAS). The results of the study indicated that ABT was a valid and reliable way of testing. While no significant difference was found between the test formats in student achievements in general, practical significance existed when the study further compared the impact of ABT versus GBT in student achievements with various levels of prior knowledge. It was found that low prior knowledge students performed better in ABT while high prior knowledge students performed better in GBT. Finally, more than 60% of the participants who took ABT were satisfied and held positive attitudes toward ABT.