Fine-Grained Image Classification of Groceries for Assisting Visually Impaired People

Description

n recent years, computer vision-based assistive technologies have enabled visually impaired people to use automatic image classification on their mobile phones. However, the in-built image classifiers often lack the ability to recognise fine-grained object information that could be important for the user. A particular application of such situations is classifying groceries, where the challenge is to classify visually similar items in high-variation environments. In this seminar, I will present a dataset with mobile phone images of groceries that simulates the scenario of using an image classifier to identify food items in grocery stores. Furthermore, I will illustrate how the mobile phone images can be combined with web-scraped images and text descriptions using a variational autoencoder to train more accurate classifiers, compared to training with mobile phone images only. Finally, I will introduce how to continually update the image classifier with new items without retraining from scratch, and discuss some future directions for the development of computer vision-based assistive mobile apps.

Period	2022 Oct 12
Event type	Seminar
Location	Lund, SwedenShow on map