Organizing video sequences from shot level to scene level is a challenging task, however, without the scene information, it will be very hard to extract the content of the video. In this paper, a visual and rule information integrated strategy is proposed to detect the scene information of the News programs. First, we extract out the video caption and anchor person (AP) shots in the video. Second, the rules among the scene, video caption, AP shots will be used to detect the scenes. Third, the visual features will also be used to analysis the scene information in the region which can not be covered by the rules. And experimental results based on this strategy are provided and analyzed on broadcast News videos.