The explosive growth of video data demands the video presentation technique which supports fast browsing of video content. In this paper, we present an automatic procedure for constructing a compact synthesized collage from a video sequence. The synthesized image, called “Video Collage”, is a kind of static video summary—to select the most representative images from video, to extract salient regions of interest (ROIs) from these images, and to seamlessly arrange ROIs on a given canvas with the temporal structure of video content preserved. We formulate the generation of Video Collage as a unified energy minimization problem in which each of above desirability is represented by an energy term. We start from the basic setting of Video Collage in which both the shape of ROIs and collage are fixed as rectangular, and then show how it can support arbitrary shapes of ROIs, as well as a variety of collage templates and region of interest (ROI) arrangement layouts (i.e., book, diagonal, and spiral). The experiments show its effectiveness to present a video in a very compact and visually appealing form while preserving the necessary information to understand the video.