Research Frontiers

Abstract

In this book, we investigate some application-motivated problems, namely the research problems of micro-video understanding. To solve these problems, we design some general principles, methodologies, and optimizations by jointly learning from multiple correlated modalities of the given micro-videos, including the textual, visual, acoustic, and social ones. They are empirically validated on multiple real-world datasets. In particular, we first introduce the proliferation of micro-video services and identify three practical tasks of micro-video understanding: popularity prediction, venue category estimation, and micro-video routing. Based upon these tasks, we analyze the unique research challenges of micro-videos that are distinct from traditional long videos, such as information sparseness, hierarchical structure, low-quality, multimodal sequential data, as well as lack of benchmark datasets. To address these problems, we present a series of multimodal learning methods, consisting of multimodal transductive learning, multimodal cooperative learning, multimodal transductive learning and multimodal sequential learning. These theoretical methods are verified over three datasets we constructed. To facilitate other researchers, we have released the codes, parameter settings, as well as the three datasets. We have to emphasize that learning from multiple modalities of the given micro-videos is still a young and highly promising research field. There are many unexplored but fruitful future directions and challenging research issues. We illustrate a few of them here.