"Multi-Modal Joint Embedding Space Learning for Cross-Modalit Retrieval"
Speaker: Matthias Dorfer (JKU, Department of Computational Perception)
Cross-modality retrieval encompasses retrieval tasks where the fetched items are of a different type than the search query, e.g., retrieving pictures relevant to a given test query. The state-of-the-art approach to cross-modalties, where items from either modality are retrieved using nearest-neighbour search. In the talk Matthias Dorfer weill review two different learning paradigms - Deep Canonical Correlation Analysis and Pairwise Ranking Losses - which both yield embedding spaces exhibiting properties beneficial for retrieval. Dorfer will present potential application scenarios as well as experimental retrieval results on two different modality pairs, namely test and images as well as audio and sheet-music.
Matthias Dorfer finished his Master studies of Medical Informatics at the Vienna University of Technology in March 2013. He worked in a research project for the early diagnosis of the Alzheimer's Disease using EEG signal analysis at the Austrian Institute of Technology from 2011 to 2012. He was member of the CIR group from 2012 to 2014, and is now Researcher at the Department of Computational Perception of the Johannes Kepler University Linz, Austria.