The MediaMill TRECVID 2007 Semantic Video Search Engine

Snoek, C.G.M.; Everts, I.; Gemert, J.C. van; Geusebroek, J-M; Huurnink, B.; Koelma, D.C.; Liempt, M. van; Rooij, O. de; Sande, K.E.A. van de; Smeulders, A.W.M.; Uijlings, J.R. R.; Worring, M.

by C.G.M. Snoek, I. Everts, J.C. van Gemert, J-M Geusebroek, B. Huurnink, D.C. Koelma, M. van Liempt, O. de Rooij, K.E.A. van de Sande, A.W.M. Smeulders, J.R. R. Uijlings, M. Worring

Abstract:

In this paper we describe our TRECVID 2007 experiments. The MediaMill team participated in two tasks: concept detection and search. For concept detection we extract region-based image features, on grid, keypoint, and segmentation level, which we combine with various supervised learners. In addition, we explore the utility of temporal image features. A late fusion approach of all region-based analysis methods using geometric mean was our most successful run. What is more, using MediaMill Challenge and LSCOM annotations, our visual-only approach generalizes to a set of 572 concept detectors. To handle such a large thesaurus in retrieval, an engine is developed which automatically selects a set of relevant concept detectors based on text matching, ontology querying, and visual concept likelihood. The suggestion engine is evaluated as part of the automatic search task and forms the entry point for our interactive search experiments. For this task we experiment with two browsers for interactive exploration: the well-known CrossBrowser and the novel ForkBrowser. It was found that, while retrieval performance varies substantially per topic, the ForkBrowser is able to produce the same overall results as the CrossBrowser. However, the ForkBrowser obtains top-performance for most topics with less user interaction. Indicating the potential of this browser for interactive search. Similar to previous years our best interactive search runs yield high overall performance, ranking 3rd and 4th.

View PDF

Reference:

C.G.M. Snoek, I. Everts, J.C. van Gemert, J-M Geusebroek, B. Huurnink, D.C. Koelma, M. van Liempt, O. de Rooij, K.E.A. van de Sande, A.W.M. Smeulders, J.R. R. Uijlings, M. Worring, "The MediaMill TRECVID 2007 Semantic Video Search Engine", In Proceedings of the 5th TRECVID Workshop, Gaithersburg, USA, 2007.

Bibtex Entry:

@INPROCEEDINGS{SnoekTRECVID07,
  author = {C.G.M. Snoek and I. Everts and J.C. van Gemert and J-M Geusebroek
	and B. Huurnink and D.C. Koelma and M. van Liempt and O. de Rooij
	and K.E.A. van de Sande and A.W.M. Smeulders and J.R. R. Uijlings
	and M. Worring},
  title = {The {MediaMill} {TRECVID} 2007 Semantic Video Search Engine},
  booktitle = {Proceedings of the 5th TRECVID Workshop},
  year = {2007},
  address = {Gaithersburg, USA},
  month = {November},
  abstract = { In this paper we describe our TRECVID 2007 experiments. The MediaMill
	team participated in two tasks: concept detection and search. For
	concept detection we extract region-based image features, on grid,
	keypoint, and segmentation level, which we combine with various supervised
	learners. In addition, we explore the utility of temporal image features.
	A late fusion approach of all region-based analysis methods using
	geometric mean was our most successful run. What is more, using MediaMill
	Challenge and LSCOM annotations, our visual-only approach generalizes
	to a set of 572 concept detectors. To handle such a large thesaurus
	in retrieval, an engine is developed which automatically selects
	a set of relevant concept detectors based on text matching, ontology
	querying, and visual concept likelihood. The suggestion engine is
	evaluated as part of the automatic search task and forms the entry
	point for our interactive search experiments. For this task we experiment
	with two browsers for interactive exploration: the well-known CrossBrowser
	and the novel ForkBrowser. It was found that, while retrieval performance
	varies substantially per topic, the ForkBrowser is able to produce
	the same overall results as the CrossBrowser. However, the ForkBrowser
	obtains top-performance for most topics with less user interaction.
	Indicating the potential of this browser for interactive search.
	Similar to previous years our best interactive search runs yield
	high overall performance, ranking 3rd and 4th. },
  file = {mediamill-TRECVID2007-final.pdf:http\://staff.science.uva.nl/~cgmsnoek/pub/mediamill-TRECVID2007-final.pdf:PDF},
  url = {http://staff.science.uva.nl/~cgmsnoek/pub/mediamill-TRECVID2007-final.pdf}
}