MeMoT is a proof-of-concept prototype that uses social semantic knowledge, as a basis to build algorithms for supporting the archival and the retrieval of images from personal photo collections. The MeMoT retrieval power stems from the multidimensional nature of the context-space, as it enables the combinational power of the information present in the dimensions, covering the “who”, “what”, “when” and “where” (4Ws).

It is specially tailored to deal with unpredictable queries, with requirements of different grains. To deal with the unpredictable size of the results set, photos are summarised in such a way they maintain the most important cues to the context comprehension. They are aggregated by context similarity, where each group has a proper short description, and there is a global selected detail for each dimension. Based on the summary, a user can judge the adequacy of the result without traversing it, as he has the notion of the underlying context of the set. The overview is the aggregated data with a proper description for each group. The selected detail can be used to filter the groups for each available dimension, and is the starting point to drill up and down the context’s details.

The goal of this work is to integrate the MSS summarisation algorithm (described here) into the MeMoT prototype. Such integration needs to address scale problems, the selection of the proper image to represent each cluster and the proper interactive mechanics to support a contextual browsing.

Requirements: A student should consider applying to this M.Sc. project if he/she has a strong background on web developing, both client and server sides.

Supervision: The work will be supervised by Prof. Nuno Datia, from IPL/ISEL and co-supervised at 50% by Prof. João Moura Pires, from FCT/UNL.

Hosting Institution: IPL/ISEL

Additional information: This thesis is available for 2015/16. All the development will be based on the ideas and algorithms described in the paper by N. Datia, J. Moura Pires and N. Correia, 2014, entitled “Summarised Presentation of Personal Photo Sets”. There is a current implementation in python that can be used as a baseline evaluation. Further details can be found on the thesis “Using social semantic knowledge to improve annotations in personal photo collections” (ask for a copy if you need). The re-implementation of the algorithm allows the MeMot prototype to have a better interface to display the retrieved photos. To see the prototype in action, please watch this video.