My philosophy has always been that where possible ALL spatial data should be managed in a spatial database management system. Where I used to work this was ESRI's ArcSDE on top of Oracle running on a UNIX or LINUX server.Our approach to pipeline data was no different. We used an APDM inspired data model and also hooked into the pipeline engineering group's inspection and maintenance database - so we could also integrate intelligent pigging data. Our periodic ROV (Remotely Operated Vehicle - see above picture) surveys would churn out hundreds of kilomteres (= hours = terrabytes) of video data. Specialist (proprietary) software solutions were required to review the data. Commonly used systems are: VisualReview (from VisualSoft) and Starfix.DVS (from Fugro). Each of these systems stores and manages data differently. None of them are GIS based (or were at the time). So you can see that a certain point it is possible for a company to be running multiple systems for essentially the same type of data.
We solved this by deciding that the data should be delivered by the ROV survey contractor in a format that we could easily suck into the spatial database and then integrate with video. We started by allowing the spatial / time / KP tagged "event" data to be delivered in a simple ASCII CSV or UKOOA P5/94 form. The video data files were named in such a way that they indicated start of acquisition time for the video fragment. Cross profiles and current density profiles were similarly named and delivered in ASCII form.
We built a routine that loaded the video data onto NAS data storage and indexed the video files against KP and time in the database - meaning that for any location / time or KP we could find the appropriate video and go straight to the point within the video that corresponded with the location on the pipeline. Essentially we geo-coded the videos. There are now standards developed by the Sony Corporation that include geo-coded tags in video files H.264 (AVC HD / MPEG-4) and I may do something with these standards in the future (stay tuned).The video index was extremely useful because it also meant that we could automatically run a query that would identify videos in which no major pipeline inspection events were observed and archive them off-line (hence saving storage space). Our full online store was up to 6Tb!
Now that the data was safely stored in our spatial database - it was important for us to provide a tool to our users that would allow them to review the ROV survey data using a GUI metaphor that they were familiar with. We chose the emulate the design of VisualReview / Starfix.DVS and built an ArcObjects application that sat on top of the data in ArcSDE / Video store. The resultant GUI can be seen in the screen capture below. It includes synchronised map display (ArcMap Component), video cameras, current density profile, vertical profile, observed events and cross profiles.
