基于多属性学习的航拍视频理解

Understanding UAV Aerial Videos from a Multi-attribute Perspective

近年来,无人机迅猛地发展并普及所产生的大量视频,为航拍视频理解带来了前所未有的机遇和挑战。由于现有的绝大多数航拍视频数据集都是单标签标注的,不能细致地描述复杂的无人机视频场景,多属性学习的任务也无法加以实现,例如多标签视频分类。因此,在本文中,我们设计、创建、展示和验证了第一个多属性无人机视频数据库,即"Multiattribute ERA"数据库。我们从地面目标、环境特征和人类情感三个方面入手,找到28个具有代表性的视频属性,并对来自ERA数据库的2864个时长5秒、分属25个事件类的视频进行多属性标注。在此基础上,我们设计并实现了基于多属性学习的多标签视频分类任务,即在该数据库上评估了4种当前处在计算机视觉领域内前沿、基于深度卷积神经网络的视频分类方法。实验结果表明,我们提出的"Multi-attribute ERA"数据库,是可以补充当下各单标签无人机视频数据集局限性的领域内重要基准,无论是在遥感科学领域、还是计算机科学范畴内,都可以作为开发多属性视频学习模型的新挑战,促进视频理解的研究。

As the popularity of unmanned aerial vehicles(UAVs) has rapidly increased in recent years, the large volume of UAV videos gives rise to new opportunities and challenges for video understanding with an aerial view. Since most existing aerial video datasets are single-label annotated, they cannot describe complex UAV video scenes with sufficient details. As a result, multi-attribute learning tasks, such as multi-label video classi?cation, cannot be performed.In this paper, we design, create, present and validate the first UAV video attribute database, the "Multi-attribute ERA" database. A multi-attribute learning task is designed and executed as multi-label video classification for database verification, in which 4 state-of-the-art video classification convolutional neural networks(CNNs) methods are evaluated on the proposed database. The experimental results demonstrate that the multi-attribute UAV video database, Multi-attribute ERA database, is a significant benchmark that complements the current single-label UAV video datasets and can serve as a new challenge to develop multi-attribute video learning models for aerial video understanding researches.