Background segmentation beyond RGB

To efficiently classify and track video objects in a surveillance application, it is essential to reduce the amount of streaming data. One solution is to segment the video into background, i.e. stationary objects, and foreground, i.e. moving objects, and then discard the background. One such motion segmentation algorithm that has proven reliable is the Stauffer and Crimson algorithm. This paper investigates how different color spaces affect the segmentation result in terms of noise and shadow sensitivity. Shadows are especially problematic since they not only distort shape but can also result in falsely connected objects that will complicate tracking and classification. Therefore, a new decision kernel for the segmentation algorithm is presented. This kernel alters the probability of foreground detection to reduce shadows and to increase the chance of correct segmentation for objects with a skin tone color, e.g. faces.