Computer Vision Based Analysis of Animal Behavior

The behavior of animals is commonly studied in medicine and biology. There is a large variation in what animals are studied, in experimental paradigms and purpose.
However, many studies on animal behavior have at least one thing in common - it typically involves measuring or studying the kinematics of the animal. To allow for verifiable and quantitative behavioral analysis, the experiments are recorded and kinematic data is extracted from the videos using computer vision methods. This thesis deals with the development of methods that takes recorded videos as input and provides behavioral data as output. The system of methods can be split into three parts - tracking of animal pose, extraction of kinematic features and analysis of the features. This thesis focus mainly on the two first parts. However, an important aspect in the design of the system is that all parts should be compatible. Therefore, all method development has been conducted in collaboration with medical/biological scientists.

This thesis contains computer vision methods for tracking rats, marmosets, zebrafish, jellyfish and zooplankton.
Most of the projects are represented by a scientific paper that outlines the computer vision methods, and a paper that focus on the medical/biological application of the computer vision based system.
One of the methods is applied to study the correlation between fine-kinematic behavior and neuronal activity in rats. Another method is used to characterize the long term effects of the marmoset model of Parkinson's disease. Thirdly, a high-throughput system is developed to quantify drug-induced changes in zebrafish larvae behavior. Fourthly, steps are taken towards a system that allows for studying the correlation between visual stimuli and movement output in the box jellyfish. Lastly, nonlinear positioning methods are proposed for the purpose of studying e.g. multiple threat response in zooplankton inside an aquarium.

Additionally, and seemingly an outlier, this thesis features a novel method for estimating relative camera motion in an underwater setting. The method is not applied for analyzing animal behavior, but it is related to the geometrical problems of refraction encountered while positioning the zooplankton.
In this project, we leverage the pseudo-depth information that is contained in underwater images to design a three point relative pose algorithm.