Continuous Optimization for Multimodal Data Fusion in Vector Search

The project aims to develop and evaluate methodologies for continuous optimization of vector search with dynamically collected multimodal data, with a primary application in e-commerce. This includes behavioural signals, product attributes, images, and any other data complementary to textual queries that may help to capture the context of a search. The first part of the project focuses on learning an efficient multimodal vector representation for both products and queries. The second part of the project will employ continuous optimization mechanisms to find the optimal set of modality weights that would maximize the conversion rate or some other metric of interest. The project outcomes are expected to not only benefit the primary application domain but also extend to other industries that can leverage a generic vector search with arbitrary modalities optimized for specific ranking metrics. This is a joint project between Lund University and Theca Systems AB.