ChatGPT可否充当情感专家?——调查其在情感与隐喻分析的潜力

Can ChatGPT Be Served as the Sentiment Expert?An Evaluation of ChatGPT on Sentiment and Metaphor Analysis

为了探索ChatGPT情感分析能力以及对主观性和隐喻性理解的潜力,将ChatGPT在5个情感、幽默与隐喻基准数据集上展开评估,通过与领域内最前沿的模型对比,讨论其在不同任务上的优势与局限。此外,还通过对比ChatGPT与人类在情感分析中的性能差别,发现ChatGPT在情感、幽默与隐喻任务上与人类结果分别相差9.52%,16.64%和6.69%。实验结果表明,尽管ChatGPT在对话生成方面获得最佳表现,但是其在情感理解方面仍具有改进的潜力。最后,通过改善提示模板,调查ChatGPT在情感理解场景下对提示模板的敏感性。

To explore the potential for subjective understanding,the subjectivity and metaphorical nature of ChatGPT,this paper evaluates ChatGPT on five sentiment,humor,and metaphor benchmark datasets and discusses its strengths and limitations on different tasks by comparing it with the most cutting-edge models in the field.In addition,this paper also compares the performance of ChatGPT and humans in sentiment analysis,with gaps of 9.52%,16.64%and 6.69%in human results on sentiment,humor and metaphor tasks.The results suggest that although ChatGPT achieves the best performance in dialogue generation,it still has potential for improvement in sentiment understanding.Finally,this paper investigates ChatGPT’s sensitivity to cueing templates in an emotion understanding scenario by improving the cueing templates.