Understanding sensor information is a fundamental problem in computer science. Scientific challenges cover the entire pipeline from single-sensor processing, over spatial and temporal fusion of multiple and divergent sensor modalities to the complete description of large-scale multimodal sensor streams. At the same time we observe a tremendous increase in both the quantity as well as the diversity of sensor information due to the increasing number of sensors (such as cameras, GPS, or inertial sensors) embedded in a wide variety of digital devices and environments as well as due to the increasing storage of multimodal sensor data (such as surveillance data, personal storage of digital information, multimedia databases, or simply the Internet). While storing and indexing large amounts of sensor data has made tremendous progress, understanding of this multimodal sensor data still lacks far behind. Therefore the long-term goal of our research is to make progress on how to process, structure, access and truly understand multi-sensory data both for online use as well as for large-scale databases.

The group currently focuses on two main areas of the broader field, namely computer vision and multimodal sensor processing. In the area of computer vision we address some of the most basic functionalities of image and video understanding such as 3D object class recognition or 3D people detection and tracking. We also look at the problem of 3D scene understanding of traffic scenes as a case study for complete scene understanding. In the area of multimodal computing we currently focus on the problem of human activity recognition as a means to study how ubiquitous or wearable computing may benefit from better sensor understanding. As a final cross-cutting theme for both areas we also work in the area of machine learning. It is clear that only advanced machine learning techniques will enable to infer higher-level information from noisy sensor data and enable to deal with the large-scale nature of current and future multimodal databases and sensor-streams.