We describe a very low bit rate video coding framework in which motion correlation between successive video frames is exploited in the multiwavelet transform domain. Some complicated techniques, such as spatial prediction in intra coding, adaptive block size motion estimation, more than one previous frames for prediction in inter frames, and content adaptive binary arithmetic coding (CABAC) are used in H.26L standard. The testing results show that H.26L can greatly outperform MPEG-4 ASP in both PSNR and visual quality. However, the encoding of H.26L costs too much time for it is complex to use fast motion search in adaptive block size motion estimation, and CABAC needs much time to generate the code list for entropy coding. Whereas, only four types of symbol are generated after zero tree wavelet coding so that the entropy coding can cost less time than CABAC. Moreover, if we select 8× 8 sized block as a basic mode, which can be united into the large size mode if neighbored 8× 8 sized blocks have same reference frame and motion vector, then the fast motion estimation can be feasible. Accordingly, a fast motion search algorithm, multiwavelet transform, and a novel adaptive quantization schemer are applied to the proposed coding frame. Experimental results reveal 0.2–0.5 dB increase in coded PSNR at low bit rates over the state-of-the-art H.26L recommendation, and similar improvements over MPEG-4 at high bit rates, with a considerable improvement in subjective reconstruction quality, while simultaneously supporting a scalable representation.