Real-time video scene analysis with heterogeneous processors

Blair, Calum Grahame (2014) Real-time video scene analysis with heterogeneous processors. EngD thesis, University of Glasgow.

Abstract

Field-Programmable Gate Arrays (FPGAs) and General Purpose Graphics Processing Units (GPUs) allow acceleration and real-time processing of computationally intensive computer vision algorithms. The decision to use either architecture in any application is determined by task-specific priorities such as processing latency, power consumption and algorithm accuracy. This choice is normally made at design time on a heuristic or fixed algorithmic basis; here we propose an alternative method for automatic runtime selection.

In this thesis, we describe our PC-based system architecture containing both platforms; this provides greater flexibility and allows dynamic selection of processing platforms to suit changing scene priorities. Using the Histograms of Oriented Gradients (HOG) algorithm for pedestrian detection, we comprehensively explore algorithm implementation on FPGA, GPU and a combination of both, and show that the effect of data transfer time on overall processing performance is significant. We also characterise performance of each implementation and quantify tradeoffs between power, time and accuracy when moving processing between architectures, then specify the optimal architecture to use when prioritising each of these.

We apply this new knowledge to a real-time surveillance application representative of anomaly detection problems: detecting parked vehicles in videos. Using motion detection and car and pedestrian HOG detectors implemented across multiple architectures to generate detections, we use trajectory clustering and a Bayesian contextual motion algorithm to generate an overall scene anomaly level. This is in turn used to select the architectures to run the compute-intensive detectors for the next frame on, with higher anomalies selecting faster, higher-power implementations. Comparing dynamic context-driven prioritisation of system performance against a fixed mapping of algorithms to architectures shows that our dynamic mapping method is 10% more accurate at detecting events than the power-optimised version, at the cost of 12W higher power consumption.

• Item Type: Thesis (EngD)
• Qualification Level: Doctoral
• Keywords: fpga, gpu, heterogeneous architecture, object detection, computer vision
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science; Q Science > QA Mathematics > QA76 Computer software; T Technology > TK Electrical engineering. Electronics Nuclear engineering
• Colleges/Schools: College of Science and Engineering; College of Science and Engineering > School of Engineering; Precurrent Departments
• Supervisor's Name: Robertson, Dr Neil M
• Date of Award: 2014
• Depositing User: Dr Calum G Blair
• Unique ID: glathesis:2014-5061
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 08 Jul 2014 15:03
• Last Modified: 08 Jul 2014 15:05
• URI: https://theses.gla.ac.uk/id/eprint/5061