011

Emotion felt and depicted: Consequences for multimedia retrieval.

The great increase of multimedia documents in collaborative services
like YouTube, Flickr and Last.fm brings with it the challenge of indexing and
retrieving these documents. One aspect which has come into the focus of information science research is capturing emotions in videos, pictures and music. The goal is to identify emotions and enable the retrieval of documents based on these emotions. Research in this area is, however, still in the early stages and an application of the concept in the World Wide Web has yet to be implemented. Other fields, like psychology, film and musicology, on the other hand, pay great attention to the emotional description of films, images and music. In order to attain satisfying results regarding the capture of the emotional impact of multimedia documents and make these captured emotions retrievable, an interdisciplinary approach is necessary. The following chapter provides a look at the fundamentals of emotion research and applies them to the identification of emotions in multimedia documents. In the process, it becomes clear that it is necessary to distinguish between emotions depicted in the medium and emotions felt by the person viewing the medium. This has lasting consequences for indexing emotions and the retrieval of emotional content in multimedia documents, since it requires us to differentiate between these two types of emotions at both stages. We are using the search engine MEMOSE to exemplify how such a differentiation between depicted and felt emotions can be implemented.

PDF