Design and Implementation of Embedded Video Surveillance Hardware

Research output: ThesisDoctoral Thesis (compilation)

Abstract

The demands on video surveillance systems are rapidly increasing regarding parameters such as frame rate and resolution. Furthermore, with an ever increasing number of video streams, an automated process for extracting relevant information is required. Due to the large amount of input data and the computational complexity of the algorithms, software implementations are not sufficient to sustain real-time performance for reasonable resolutions. In this thesis an automated digital surveillance system running on an embedded platform in real-time is presented. Algorithms that are well suited for hardware implementation with streamlined dataflow are chosen and dedicated hardware accelerators have been developed. The presented hardware platform has been developed with the goal of presenting a proof of concept for the surveillance system and to identify computational and memory bottlenecks. Furthermore, when proposing modifications to the original algorithms extensive simulations are needed, especially if long-term effects in the video sequences can be envisioned. Utilizing a reconfigurable platform based on a field programmable gate array (FPGA) reduces the simulation and development time considerably.

The main bottleneck of the proposed system, as well as for most image processing algorithms, is the high memory requirements. Therefore, simplifications have been proposed both on the algorithmic and the implementation levels. Furthermore, elevating the point of view from block to system level made it possible to perform cross boundary optimization and to explore other system architectures. On a block level, a low complexity morphological datapath unit, a contour tracing label unit that utilize Green's formula to extract object features, and color feature extraction on streaming data are presented. From a system level perspective the following were identified: unique labels are not required; changing color space from RGB to YCbCr both improved the segmentation result and allowed a simple face detector; a stall-free morphological unit will allow operations directly on the input data without buffering. It is also shown how retiming of the system can lead to substantial reduction of memory requirements. On an architectural level, replacing the morphology and labeling block with an image projection unit was investigated. To accommodate more advanced functionality such as face recognition and object classification advanced features have to be used. In this thesis one such feature detector, MSER, has been implemented.

Details

Authors
  • Fredrik Kristensen
Organisations
Research areas and keywords

Subject classification (UKÄ) – MANDATORY

  • Electrical Engineering, Electronic Engineering, Information Engineering

Keywords

  • Elektronik och elektroteknik, Electronics and Electrical technology, image features, tracking, labeling, morphology, real-time operations, Hardware design, Image processing, Imaging, image processing, Bildbehandling, Signal processing, Signalbehandling
Original languageEnglish
QualificationDoctor
Awarding Institution
Supervisors/Assistant supervisor
  • Peter Nilsson, Supervisor
Award date2007 Sep 28
Publisher
  • Electro and information technology
Publication statusPublished - 2007
Publication categoryResearch

Bibliographic note

Defence details Date: 2007-09-28 Time: 10:15 Place: Room E:1406, E-building, Ole Römers Väg 3, Lund University Faculty of Engineering External reviewer(s) Name: Turbell, Henrik Title: Ph.D. Affiliation: Senior Specialist R&D, at SICK IVP AB, Linköping ---