We propose a novel unsupervised image captioning method.Image captioning involves two fields of deep learning,natural language processing and computer vision.The excessive pursuit ofmodel evaluation results makes the ...We propose a novel unsupervised image captioning method.Image captioning involves two fields of deep learning,natural language processing and computer vision.The excessive pursuit ofmodel evaluation results makes the caption style generated by the model too monotonous,which is difficult to meet people’s demands for vivid and stylized image captions.Therefore,we propose an image captioning model that combines text style transfer and image emotion recognition methods,with which the model can better understand images and generate controllable stylized captions.The proposed method can automatically judge the emotion contained in the image through the image emotion recognition module,better understand the image content,and control the description through the text style transfermethod,thereby generating captions thatmeet people’s expectations.To our knowledge,this is the first work to use both image emotion recognition and text style control.展开更多
基金supported by the National Key Research&Development Program (Grant No.2018YFC0831700)National Natural Science Foundation of China (Grant No.61671064,No.61732005).
文摘We propose a novel unsupervised image captioning method.Image captioning involves two fields of deep learning,natural language processing and computer vision.The excessive pursuit ofmodel evaluation results makes the caption style generated by the model too monotonous,which is difficult to meet people’s demands for vivid and stylized image captions.Therefore,we propose an image captioning model that combines text style transfer and image emotion recognition methods,with which the model can better understand images and generate controllable stylized captions.The proposed method can automatically judge the emotion contained in the image through the image emotion recognition module,better understand the image content,and control the description through the text style transfermethod,thereby generating captions thatmeet people’s expectations.To our knowledge,this is the first work to use both image emotion recognition and text style control.
