NICA课题组1篇论文被国际顶级会议——第26届国际多媒体学术会议(ACM International Conference on Multimedia, 简称ACM MM 2018)录用。国际多媒体学术会议(ACM MM)是计算机学科公认的多媒体领域和计算机视觉领域的顶级国际会议,被中国计算机学会(CCF)列为A类会议。 本届会议,在全球757个投稿中,共有144篇被接受为poster,接受率约27%。
此次被接受论文的第一作者为杜长德同学, 论文标题为《Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data》。论文针对情感识别中的多模态,标签不完整,模态缺失等情况, 作者提出了统一的多视图深度生成式模型,为不完整多模学习提供了有效的解决方案。以下为论文摘要:
There are threefold challenges in emotion recognition. First, it is difficult to recognize human's emotional states only considering a single modality. Second, it is expensive to manually annotate the emotional data. Third, emotional data often suffers from missing modalities due to unforeseeable sensor malfunction or configuration issues. In this paper, we address all these problems under a novel multi-view deep generative framework. Specifically, we propose to model the statistical relationships of multi-modality emotional data using multiple modality-specific generative networks with a shared latent space. By imposing a Gaussian mixture assumption on the posterior approximation of the shared latent variables, our framework can learn the joint deep representation from multiple modalities and evaluate the importance of each modality simultaneously. To solve the labeled-data-scarcity problem, we extend our multi-view model to semi-supervised learning scenario by casting the semi-supervised classification problem as a specialized missing data imputation task. To address the missing-modality problem, we further extend our semi-supervised multi-view model to deal with incomplete data, where a missing view is treated as a latent variable and integrated out during inference. This way, the proposed overall framework can utilize all available (both labeled and unlabeled, as well as both complete and incomplete) data to improve its generalization ability. The experiments conducted on two real multi-modal emotion datasets demonstrated the superiority of our framework.