With the rapid development of wireless networks and consumer electronics, various mobile applications have emerged. However, due to some constraints such as weak computational power, limited memory and small display screen, traditional video coding applications can not work well on mobile devices. In this paper, we proposed a software-based video codec framework and its implementation which is suitable for real-time video coding applications on mobile devices. Some key optimizing techniques, such as fast predictive motion estimation (ME), zero-coefficients prejudgment and multiplierless integer discrete cosine transform (DCT), are used in our codec. Experimental results demonstrate the flexibility of our framework and the good speedup we achieved while video quality degradation is negligible. The codec is suitable for scenarios where low-complexity computing is required.