[MSc-OpenSubject] – Segmentation of personal memories in a multiple publisher scenario

Taking a photo is now a common activity driven by technological evolutions in digital devices. Besides, the evolutions in storage devices make available to ordinary people bigger storage devices for less money. These factors contribute to the steady increasing of digital photos for personal and social consumption. Managing those collections is hard and users benefit from an automatic organisation into groups of photos that share a common context – the events. A common scenario in this domain is the dissemination, among friends and family, of a set of photos taken at the same spatio-temporal location, using different cameras operated by different photographers. However, the moment of archival of each set is different. They differ in size and depicted items, the temporal information are misaligned between cameras and the spatial data can be sparse, even when users use smart-phones to take the photos. There is a current implementation of a segmentation algorithm that uses spatio-temporal information to carry a one time segmentation. Besides, each segment may be annotated by a user.

This work addresses a challenging problem of combining, iteratively, ordered sets of photos, refining a segmentation that unveils their structure. The key challenge are the maintained of the coherence between the depicted items and the spatio-temporal information of each segment, knowing the underlying information can have error or can be partially missing. Such coherence extends to the visual narrative inside each segment, to the (eventual) annotations that already exist and to the textual description that a user may have entered during the archival.

Requirements: A student should consider applying to this M.Sc. project if he/she has a strong background on programming algorithms and their analysis.

Supervision: The work will be supervised by Prof. Nuno Datia, from IPL/ISEL and co-supervised at 50% by Prof. João Moura Pires, from FCT/UNL.

Hosting Institution: IPL/ISEL

Additional information: This thesis is available for 2015/16. All the algorithms and interfaces developed in the thesis will be assessed using user tests. Further details can be found on the thesis “Using social semantic knowledge to improve annotations in personal photo collections” (ask for a copy if you need). To see the current segmentation algorithm in action, please watch this video.