跨媒體分析的理論和方法

盧漢清 劉靜 黃萱菁

摘 要：該年度的研究主要圍繞多媒體對象的多粒度語義分析與關(guān)聯(lián)挖掘等方面展開，考察媒體對象與語義標(biāo)簽關(guān)聯(lián)矩陣的縱橫不同視角，充分發(fā)掘底層特征之間的關(guān)聯(lián)性，語義特征之間的關(guān)聯(lián)性，以及媒體對象在底層特征空間相似度和語義標(biāo)簽空間相似度的一致性，同時(shí)注重與應(yīng)用背景的緊密結(jié)合，力爭將研究成果做實(shí)做細(xì)。依照“課題計(jì)劃任務(wù)書及其后3年調(diào)整方案”要求，該課題在多媒體異構(gòu)特征拓?fù)浣Y(jié)構(gòu)分析、媒體對象的多粒度語義解析等方面取得了突破性進(jìn)展，課題整體進(jìn)展順利，已完成本年度計(jì)劃的各項(xiàng)預(yù)期目標(biāo)。在媒體數(shù)據(jù)的層次化語義分析方面，重點(diǎn)關(guān)注社會(huì)標(biāo)簽在媒體信息理解任務(wù)中的重要作用，引入標(biāo)簽行為的參與者（即用戶）以及地理位置等多屬性信息，以提高社會(huì)媒體網(wǎng)站中多媒體對象的語義理解性能。同時(shí)，我們還在多媒體內(nèi)容的細(xì)粒度語義解析方面展開研究工作。在基于語義的媒體內(nèi)容檢索與應(yīng)用方面，重點(diǎn)考慮媒體數(shù)據(jù)的多模態(tài)與多關(guān)聯(lián)特性，在已取得層次化語義分析成果的基礎(chǔ)上，進(jìn)一步關(guān)注用戶對媒體檢索的高、精、準(zhǔn)的實(shí)際需求，力圖實(shí)現(xiàn)網(wǎng)絡(luò)媒體數(shù)據(jù)檢索的快速性與準(zhǔn)確性，并結(jié)合實(shí)際應(yīng)用開發(fā)了相關(guān)的檢索服務(wù)原型系統(tǒng)。

關(guān)鍵詞：跨媒體 多粒度語義分析 關(guān)聯(lián)挖掘

Abstract：Our work in this year focuses on the multi-granularity semantic analysis and correlation mining for the multimedia information. We attempt to utilize correlations within low-level and high-level features， and their similarity consistence to better understand multimedia objects. Our project goes well， and has reached the goals of this year. There are totally 33 publications in this year， in which 18 papers are published on international journals or transactions （SCI indexed）， and 15 papers are published on international conferences （e.g.， ACM Multimedia， ICCV， CIKM， and CVPR， EI indexed ）. Besides， we have one authorized patent and two pending patents. In the following， we will introduce our finished work in this year in details. （1）Multimedia feature representation and correlation construction：We have proposed a set of effective methods to solve the problem of the multi-modal feature fusion and selection when given a large-scale， noisy， and high dimensional multimedia dataset. One is the multi-view learning approach considering the consistency and complementarity of different features， one is the sub-space learning based robust feature selection， and the other is the topological feature structure analysis. The related works have published on important journals of TNNLS and CVIU， and top conferences of ACM MM and WWW， etc. （2）Hierarchical semantic analysis of multimedia data：We attempt to semantically understand multimedia data （video and image） from different semantic levels including low-level visual appearance， object part， object， and scene. To this goal， we utilized the important role of social tags to enhance the performance of multimedia semantic understanding. Other relevant attributes to social tags， i.e.， tagging users and geographic positions， are also considered for the task. The related works have published on important journals of TMM， Pattern Recognition， and TALSP， and top conferences of CVPR， ICCV， and ICME， etc. （3）Semantic retrieval and other applications：To integrate and verify our proposed approaches in the project， we attempt to develop and design some prototype systems for multimedia retrieval. The systems can meet user real requirements in retrieval process. The related works have published on important journals of TKDE， TOMCCAP， and TMM， and top conferences of ACM MM， WWW， and ICIP， etc.

Key Words：Cross-Media；Multi-granularity Semantic Analysis；Correlation Mining

閱讀全文鏈接（需實(shí)名注冊）：http：//www.nstrs.cn/xiangxiBG.aspx？id=51008&flag=1