Event-based Retrieval of Networked Media

GLOCAL DIT-PRJ-10-014

Homepage http://www.glocal-project.eu
Status NOT active project
DISI role Coordinator
Project type Research Project
Dimension International
Acquisition date 2009-12-01
Start date 2009-12-01
End date 2012-11-30

Project details

Project astract We describe the world by using words. Yet, words usually bring to mind different mental views of the world in different individuals, because of their personal experience and context. This is the reason why the gap between our conceptualizations of the world, expressed using language, and our experience of the world, whose most direct representations are media, is far beyond the reach of current systems; and it is also why, so far, a universal solution of the problem of contextualizing search, navigation, and media management in general to the user needs and the operating environment has not been found.<br/><br/>The key idea underlying GLOCAL is to use events as the primary means to organize and index media, e.g., photos, videos, journal articles. Instead of starting from media and seeing, a posteriori, how we can meaningfully understand their contents (e.g., by tagging them), we organize a priori our data and knowledge in terms of events and use media to populate them, thus providing their experiential dimension. Events provide the common framework inside which the local experience-driven contextual information can be not only coded, but also shared and reduced to a common denominator.<br/><br/>Events have both a local and a global dimension. The local dimension enables the mapping of tags (conceptualizations) to media (personal experiences), while the global dimension enables the sharing of event descriptions (thus enabling social sharing and networking of events, media, and tags) but also of event structures across similar events, thus providing a common way to index media (social sharing and networking of event structures). In turn, the networking of events and event structures enables the creation of networked communities inside which common (global) descriptions of the world can be built and continuously enriched by the continuous flow of individual (local) descriptions.<br/>
Keywords conceptualizations, networking of events, social sharing, 3D Internet, Networked Media, Networked search, retrieval
Fundings 5114030 €
Partners
  • DIT - UniTN
  • Intelligent Software Components S.A.
  • Alinari 24ore SpA
  • Centre for Research and Technology Hellas
  • Yahoo! Iberia
  • Agence France-Presse
  • Deutsche Forschungszentrum für Künstliche Intelligenz
  • Exalead
  • Gottfried Wilhelm Leibniz Universitat Hannover (L3S Research Centre)
  • Institut National de Recherche en Informatique et en Automatique

DISI Sub-project details

Project astract Overall objective of this sub-project part is to enable a rich multimedia experience in search by developing techniques for event-based multimedia content indexing. Enabling the indexing of each content item not only by means of low-level features or concepts, as in current state-of-the-art approaches, but also by means of real-life events will significantly boost the efficiency of multimedia content retrieval and management and will allow the integration of advanced personalized multimedia retrieval functionalities in the search schemes investigated in WP4. Classical feature-driven content-based multimedia indexing approaches will be considered in a new perspective to define innovative approaches to assist in bridging the gap between the quantitative data and user perceptions. Exploitation of the context of multimedia data and in particular the interaction with the user will help to bridge the gap between the capabilities of current multimedia understanding algorithms and the richness and subjectivity of semantics in human interpretations of multimedia data. The extraction of concepts and facts from multimedia data and of their relations with the context will be supported, thereby progressively increasing the semantics of the dataset. This will be made possible through an integrated approach jointly using both advanced features extracted from the data, meta-representations available (partial annotation, meta-data connected to location or time) and advanced multidimensional media analysis for concept/object extraction (shape, places, actions). The synergy among low-level and high-level media representations will allow the definition of concept networks eventually supporting event-based content indexing.<br/><br/>Then we will investigate the local dynamics of the personal/subjective knowledge and will model the evolution of this knowledge based on the interaction with the user and with her local context. The main idea is to define suitable models to describe the semantics at the local level so that each user can benefit from the universal knowledge (modelled in WP1), can make the knowledge at the local level evolve in time and can share the knowledge with others in a social networking community (WP3). We will model (1) the personal experience in which the contents take a private meaning according to the user's events and semantics, (2) the possibility of adapting and personalizing the knowledge according to subjective semantics and personal interpretation of the world based on the interaction with the specific user and (3) the knowledge evolution by progressive augmentation of the local knowledge by adding the user's personal semantics to the data through different implicit and explicit means.<br/><br/>Global Dynamics will target the following objectives: <br/>Develop techniques to exploit the diverse knowledge coming from different sources, for providing event and annotations recommendations<br/>Develop techniques for collaborative data organization and ontology management<br/>Develop methods for detecting event-based user communities<br/>Develop privacy preserving data summarizations and models, as well as user friendly access management tools for data sharing<br/><br/>Provide solutions for multimedia search in an event-enriched environment, on an Internet scale. Events can be classified in multiple ways (e.g. financial events vs. political events, personal events vs. public events). We are also interested in organizing detected events by different categories. In order to do that, the essential elements of an event, such as the related persons, time, and location, should be extracted from multimedia objects correctly. Effective information extraction techniques should be employed and developed in this regard. The tasks in this work package center on temporal and spatial evolution of event mining and search. There are more than 50 million geo-tagged images from Flickr, more than 100 million additional images from Flickr, and large amounts of tagged data from other news and Internet sources available for the project.<br/><br/>
Keywords Networked search, Networked retrieval
Fundings 853350 €
Manager Francesco De Natale