ChatGPT changes the way of knowledge production and information space structure of human society.In the healthcare industry,ChatGPT's powerful question-and-answer capability will drive its application in automated...ChatGPT changes the way of knowledge production and information space structure of human society.In the healthcare industry,ChatGPT's powerful question-and-answer capability will drive its application in automated question answering in online healthcare communities.However,because ChatGPT answers are limited by factors such as the quality of data sets,their authority and accuracy cannot be guaranteed,and they are prone to misdiagnosis and damage to life and health.Therefore,the identification of ChatGPT answers in online medical communities with physician answers is crucial.In this paper,we collected medical question-answering data generated by the Haodafu platform and ChatGPT,respectively,constructed feature vectors from semantic features,syntactic features,and the fusion of both,and combined different feature vectors with XGBoost models to construct BERT-XGBoost,POS-XGBoost and Merge-XGBoost models for identifying ChatGPT answers and physician answers in online medical communities.The three models achieved accuracy rates of 0.960,0.968,and 0.986,respectively.The difference in performance between the three models reflects the degrees of variation in different features of ChatGPT answers versus physician answers.The results indicate that the differences between ChatGPT and physicians in syntactic features,i.e.,linguistic expression habits,are greater than their differences in semantic features,i.e.,specific content suggestions.展开更多
基金supported in part by National Natural Science Foundation,PR China(Grant No.72374158)。
文摘ChatGPT changes the way of knowledge production and information space structure of human society.In the healthcare industry,ChatGPT's powerful question-and-answer capability will drive its application in automated question answering in online healthcare communities.However,because ChatGPT answers are limited by factors such as the quality of data sets,their authority and accuracy cannot be guaranteed,and they are prone to misdiagnosis and damage to life and health.Therefore,the identification of ChatGPT answers in online medical communities with physician answers is crucial.In this paper,we collected medical question-answering data generated by the Haodafu platform and ChatGPT,respectively,constructed feature vectors from semantic features,syntactic features,and the fusion of both,and combined different feature vectors with XGBoost models to construct BERT-XGBoost,POS-XGBoost and Merge-XGBoost models for identifying ChatGPT answers and physician answers in online medical communities.The three models achieved accuracy rates of 0.960,0.968,and 0.986,respectively.The difference in performance between the three models reflects the degrees of variation in different features of ChatGPT answers versus physician answers.The results indicate that the differences between ChatGPT and physicians in syntactic features,i.e.,linguistic expression habits,are greater than their differences in semantic features,i.e.,specific content suggestions.
