Intelligent pipeline inspection gauges (PIGs) are inspection vehicles that move along within a gas (or oil) pipeline and acquire
signals from their surrounding rings of sensors. By analyzing the signals captured by intelligent PIGs, we can detect pipeline
defects, such as holes, curvatures and other potential causes of gas explosions. We notice that the size of collected data
using a PIG is usually in GB range. Thus, analyzer software must handle such scalable data and provide various kinds of visualization
tools so that analysts can easily detect any defects in the pipeline. In this paper, we propose a scalable pipeline data processing
framework using database and visualization techniques. Specifically, we analyze requirements for our system, giving its overall
architecture of our system. Second, we describe several important subsystems in our system: such as a scalable pipeline data
store, integrated multiple visualization, and automatic summary report generator. Third, by performing experiments with GB-range
real data, we show that our system is scalable to handle such large pipeline data. Experimental results show that our system
outperforms a relational database management system (RDBMS) based repository by up to 31.9 times.
Keywords Scalable processing - Time series data - Intelligent PIGs