Sparse Spatial Shading in Augmented Reality

In this work, we present a method for acquiring, storing, and using scene data to enable realistic shading of virtual objects in an augmented reality application. Our method allows for sparse sampling of the environment’s lighting condition while still delivering a convincing shading to the rendered objects. We use common camera parameters, provided by a head-mounted camera, to get lighting information from the scene and store them in a tree structure, saving both locality and directionality of the data. This makes our approach suitable for implementation in augmented reality applications where the sparse and unpredictable nature of the data samples captured from a head-mounted device can be problematic. The construction of the data structure and the shading of virtual objects happen in real time, and without requiring high-performance hardware. Our model is designed for augmented reality devices with optical see-through displays, and in this work we used Microsoft’s HoloLens 2.