The Video Event Awareness Workbench (VEAW) analyzes surveillance video from thousands of video cameras and automatically detects complex events in near-real-time-at pace with their input video streams. For events of interest to security personnel, VEAW generates and routes alerts and related video evidence to subscribing security personnel. Complex event processing in VEAW is driven by user-authored awareness specifications comprised of inter-connected spatio-temporal stream and statistical operators that consume and produce events described in VEAW’s surveillance ontology. In this paper we introduce VEAW’s event driven architecture and describe its solutions for automating video surveillance, including the orchestration of continuous and tasked video analysis algorithms (e.g., for entity tracking and identification), fusion of events from multiple sources in an installation-specific “world” model, and proactive information gathering to deal with missing or incomplete information (this is done by tasking video analysis algorithms and security personnel to provide it). We also discuss how VEAW deals with late arriving information (due to out-of-band video analysis tasks and overhead), as well as a related resource optimization aimed at minimizing computation costs. We illustrate the benefits of VEAW by illustrating its application on the automation of real-world security policies.