FACOLTA’ DI INGEGNERIA

Dipartimento di Ingegneria Elettrica ed Elettronica
Scuola di Dottorato in Ingegneria dell’Informazione

Seminario

Il Dr. Apostol (Paul) Natsev, responsabile del settore "Multimedia" presso il centro di ricerca IBM "IBM Watson Research Center", terrà un seminario di 8 ore sul tema Visual Content-Based and Semantic Concept-Based Multimedia Indexing and Search

Il seminario si terrà in aula I (Padiglione centrale Facoltà di Ingegneria) nei giorni

Lunedi’ 20 Settembre dalle 9 alle 13
Mercoledi’ 22 Settembre dalle 9 alle 13

Tutti gli interessati sono invitati a partecipare.

Di seguito si riporta un sommario degli argomenti che verranno trattai nel seminario
===============
Social media creation and use has skyrocketed in recent years and has become an indelible part of our lives -- from the way we entertain and inform ourselves to the way we communicate, socialize, or learn.   More than 24 hours of new video content are uploaded on YouTube each minute, and over 100M new photos are uploaded on Facebook every day.  With the tremendous growth of online multimedia content come great opportunities but also great expectations and challenges.  Users expect images and video to be searchable as easily as text but technology has unfortunately not kept pace.

In this short course, I will present the current state of art in multimedia search, and will review the current approaches for visual content-based as well as semantic indexing and search. Emphasis will be given on a new promising direction of research, semantic concept-based retrieval, which aims to boost both the effectiveness and usability of multimedia search.  I will describe techniques that
leverage the computer’s ability to effectively analyze visual features of images and video, and apply statistical machine learning techniques to classify and label visual scenes, objects, people, and activities, automatically.  I will also describe methods that leverage such automatically generated labels to improve the quality of multimedia indexing and search as well as to enable new applications and content monetization models.  The above approaches will be presented and demonstrated in the context of a state-of-art multimedia analysis and retrieval system developed at IBM Research (http://www.alphaworks.ibm.com/tech/imars).

Here is a preliminary synopsis and list of topics to be covered (still subject to change but not by much):
1. Introduction and motivation
1.1 Opportunities of multimedia analysis and retrieval
1.2 Challenges and basic problems
1.3 Example applications and demos

2. Content-based retrieval
2.1 Global Visual Features
       - Color spaces and color features
        - Color feature representations (color histograms, correlograms, moments)
        - Texture features (structural, statistical, spectral)
        - Edge and shape features
2.2 Local visual features
        - Interest point detection
        - Local descriptor representation
        - Local point matching and spatial registration
2.2 Similarity measures and evaluation metrics
2.3 Video segmentation and matching
2.4 Advanced techniques of content-based retrieval
2.4 Video fingerprinting and near-duplicate detection

3. Semantic concept-based retrieval
3.1 Definitions and motivation
3.2 Semantic concept vocabulary design
3.3 Semantic concept modeling and extraction
3.4 Multi-modal fusion and semantic context exploitation
3.5 Retrieval by semantic concepts
3.6 Concept-based query expansion

4. Multi-modal video retrieval -- a case study
4.1 Speech-based retrieval
4.2 Visual content-based retrieval
4.3 Semantic concept-based retrieval
4.4 Query-dependent multi-modal fusion
4.5 Performance evaluation (TRECVID)
4.6 Demos and other applications