We present findings from a balanced repeated-measurement evaluation where 16 subjects read texts of similar length and difficulty using the traditional Page and the dynamic Rapid Serial Visual Presentation (RSVP) format on a mobile device. Apart from monitoring reading speed, comprehension, and NASA-TLX task load, we also devised a system that enabled us to keep track of subjects eye movements. The results indicate no significant differences in reading speed or comprehension, but for task load, RSVP increased the Temporal task load factor. However, the most striking differences were found in the eye movement recordings. RSVP was found to decrease the overall number of eye movements significantly. But, RSVP was also found to significantly increase the number of regressions, although it decreased the number of saccades. These findings contradict common claims and their implications for the improvement of readability on mobile devices are discussed.