Large Language Models(LLMs)are increasingly demonstrating their ability to understand natural language and solve complex tasks,especially through text generation.One of the relevant capabilities is contextual learning...Large Language Models(LLMs)are increasingly demonstrating their ability to understand natural language and solve complex tasks,especially through text generation.One of the relevant capabilities is contextual learning,which involves the ability to receive instructions in natural language or task demonstrations to generate expected outputs for test instances without the need for additional training or gradient updates.In recent years,the popularity of social networking has provided a medium through which some users can engage in offensive and harmful online behavior.In this study,we investigate the ability of different LLMs,ranging from zero-shot and few-shot learning to fine-tuning.Our experiments show that LLMs can identify sexist and hateful online texts using zero-shot and few-shot approaches through information retrieval.Furthermore,it is found that the encoder-decoder model called Zephyr achieves the best results with the fine-tuning approach,scoring 86.811%on the Explainable Detection of Online Sexism(EDOS)test-set and 57.453%on the Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter(HatEval)test-set.Finally,it is confirmed that the evaluated models perform well in hate text detection,as they beat the best result in the HatEval task leaderboard.The error analysis shows that contextual learning had difficulty distinguishing between types of hate speech and figurative language.However,the fine-tuned approach tends to produce many false positives.展开更多
Mo_(2)C is an excellent electrocatalyst for hydrogen evolution reaction(HER).However,Mo_(2)C is a poor electrocatalyst for oxygen evolution reaction(OER).Herein,two different elements,namely Co and Fe,are incorporated...Mo_(2)C is an excellent electrocatalyst for hydrogen evolution reaction(HER).However,Mo_(2)C is a poor electrocatalyst for oxygen evolution reaction(OER).Herein,two different elements,namely Co and Fe,are incorporated in Mo_(2)C that,therefore,has a finely tuned electronic structure,which is not achievable by incorporation of any one of the metals.Consequently,the resulting electrocatalyst Co_(0.8)Fe_(0.2)-Mo_(2)C-80 displayed excellent OER catalytic performance,which is evidenced by a low overpotential of 214.0(and 246.5)mV to attain a current density of 10(and 50)mA cm^(-2),an ultralow Tafel slope of 38.4 mV dec^(-1),and longterm stability in alkaline medium.Theoretical data demonstrates that Co_(0.8)Fe_(0.2)-Mo_(2)C-80 requires the lowest overpotential(1.00 V)for OER and Co centers to be the active sites.The ultrahigh catalytic performance of the electrocatalyst is attributed to the excellent intrinsic catalytic activity due to high Brunauer-Emmett-Teller specific surface area,large electrochemically active surface area,small Tafel slope,and low chargetransfer resistance.展开更多
As the realm of enterprise-level conversational AI continues to evolve, it becomes evident that while generalized Large Language Models (LLMs) like GPT-3.5 bring remarkable capabilities, they also bring forth formidab...As the realm of enterprise-level conversational AI continues to evolve, it becomes evident that while generalized Large Language Models (LLMs) like GPT-3.5 bring remarkable capabilities, they also bring forth formidable challenges. These models, honed on vast and diverse datasets, have undoubtedly pushed the boundaries of natural language understanding and generation. However, they often stumble when faced with the intricate demands of nuanced enterprise applications. This research advocates for a strategic paradigm shift, urging enterprises to embrace a fine-tuning approach as a means to optimize conversational AI. While generalized LLMs are linguistic marvels, their inability to cater to the specific needs of businesses across various industries poses a critical challenge. This strategic shift involves empowering enterprises to seamlessly integrate their own datasets into LLMs, a process that extends beyond linguistic enhancement. The core concept of this approach centers on customization, enabling businesses to fine-tune the AI’s functionality to fit precisely within their unique business landscapes. By immersing the LLM in industry-specific documents, customer interaction records, internal reports, and regulatory guidelines, the AI transcends its generic capabilities to become a sophisticated conversational partner aligned with the intricacies of the enterprise’s domain. The transformative potential of this fine-tuning approach cannot be overstated. It enables a transition from a universal AI solution to a highly customizable tool. The AI evolves from being a linguistic powerhouse to a contextually aware, industry-savvy assistant. As a result, it not only responds with linguistic accuracy but also with depth, relevance, and resonance, significantly elevating user experiences and operational efficiency. In the subsequent sections, this paper delves into the intricacies of fine-tuning, exploring the multifaceted challenges and abundant opportunities it presents. It addresses the technical intricacies of data integration, ethical considerations surrounding data usage, and the broader implications for the future of enterprise AI. The journey embarked upon in this research holds the potential to redefine the role of conversational AI in enterprises, ushering in an era where AI becomes a dynamic, deeply relevant, and highly effective tool, empowering businesses to excel in an ever-evolving digital landscape.展开更多
古汉语文本承载着丰富的历史和文化信息,对这类文本进行实体关系抽取研究并构建相关知识图谱对于文化传承具有重要作用.针对古汉语文本中存在大量生僻汉字、语义模糊和复义等问题,提出了一种基于BERT古文预训练模型的实体关系联合抽取模...古汉语文本承载着丰富的历史和文化信息,对这类文本进行实体关系抽取研究并构建相关知识图谱对于文化传承具有重要作用.针对古汉语文本中存在大量生僻汉字、语义模糊和复义等问题,提出了一种基于BERT古文预训练模型的实体关系联合抽取模型(entity relation joint extraction model based on BERT-ancient-Chinese pretrained model,JEBAC).首先,通过融合BiLSTM神经网络和注意力机制的BERT古文预训练模型(BERT-ancientChinese pre-trained model integrated BiLSTM neural network and attention mechanism,BACBA),识别出句中所有的subject实体和object实体,为关系和object实体联合抽取提供依据.接下来,将subject实体的归一化编码向量与整个句子的嵌入向量相加,以更好地理解句中subject实体的语义特征;最后,结合带有subject实体特征的句子向量和object实体的提示信息,通过BACBA实现句中关系和object实体的联合抽取,从而得到句中所有的三元组信息(subject实体,关系,object实体).在中文实体关系抽取DuIE2.0数据集和CCKS 2021的文言文实体关系抽取CCLUE小样本数据集上,与现有的方法进行了性能比较.实验结果表明,该方法在抽取性能上更加有效,F1值分别可达79.2%和55.5%.展开更多
在互联网时代,越来越多的财务公司选择在财经新闻平台上发表自己的见解,这些评论文本作为舆情的载体,可以充分反映财务公司的情绪,影响公众的投资决策和市场走势.情感分析为分析海量的经济类文本情感类型提供了有效的研究手段.但是,由...在互联网时代,越来越多的财务公司选择在财经新闻平台上发表自己的见解,这些评论文本作为舆情的载体,可以充分反映财务公司的情绪,影响公众的投资决策和市场走势.情感分析为分析海量的经济类文本情感类型提供了有效的研究手段.但是,由于特定领域文本的专业性和大标签数据集的不适用性,经济类文本情感分析给传统的情感分析模型带来了巨大的挑战.当将一般情感分析模型应用于经济等特定领域时,模型在准确率与召回率上表现较差.为了克服这些挑战,文章针对财经新闻平台上的经济类文本的情感分析任务,从词表示模型出发,提出了基于知识蒸馏方法的双路BERT(Two-way BERT based on knowledge distillation method)情感分析模型,与文本卷积神经网络(Text-CNN)、卷积递归神经网络(CRNN)、双向长时和短时记忆网络(Bi-LSTM)等算法进行对比实验,结果得出该改进方法相较于其他算法在准确率、召回率和F1值均提升了1%~3%,具有较好的泛化性能.展开更多
针对民航陆空通话领域语料难以获取、实体分布不均,以及意图信息提取中实体规范不足且准确率有待提升等问题,为了更好地提取陆空通话意图信息,提出一种融合本体的基于双向转换编码器(bidirectional encoder representations from transf...针对民航陆空通话领域语料难以获取、实体分布不均,以及意图信息提取中实体规范不足且准确率有待提升等问题,为了更好地提取陆空通话意图信息,提出一种融合本体的基于双向转换编码器(bidirectional encoder representations from transformers,BERT)与生成对抗网络(generative adversarial network,GAN)的陆空通话意图信息挖掘方法,并引入航班池信息对提取的部分信息进行校验修正,形成空中交通管制(air traffic control,ATC)系统可理解的结构化信息。首先,使用改进的GAN模型进行陆空通话智能文本生成,可有效进行数据增强,平衡各类实体信息分布并扩充数据集;然后,根据欧洲单一天空空中交通管理项目定义的本体规则进行意图的分类与标注;之后,通过BERT预训练模型生成字向量并解决一词多义问题,利用双向长短时记忆(bidirectional long short-term memory,BiLSTM)网络双向编码提取上下句语义特征,同时将该语义特征送入条件随机场(conditional random field,CRF)模型进行推理预测,学习标签的依赖关系并加以约束,以获取全局最优结果;最后,根据编辑距离(edit distance,ED)算法进行意图信息合理性校验与修正。对比实验结果表明,所提方法的宏平均F_(1)值达到了98.75%,在民航陆空通话数据集上的意图挖掘性能优于其他主流模型,为其加入数字化进程奠定了基础。展开更多
基金This work is part of the research projects LaTe4PoliticES(PID2022-138099OBI00)funded by MICIU/AEI/10.13039/501100011033the European Regional Development Fund(ERDF)-A Way of Making Europe and LT-SWM(TED2021-131167B-I00)funded by MICIU/AEI/10.13039/501100011033the European Union NextGenerationEU/PRTR.Mr.Ronghao Pan is supported by the Programa Investigo grant,funded by the Region of Murcia,the Spanish Ministry of Labour and Social Economy and the European Union-NextGenerationEU under the“Plan de Recuperación,Transformación y Resiliencia(PRTR).”。
文摘Large Language Models(LLMs)are increasingly demonstrating their ability to understand natural language and solve complex tasks,especially through text generation.One of the relevant capabilities is contextual learning,which involves the ability to receive instructions in natural language or task demonstrations to generate expected outputs for test instances without the need for additional training or gradient updates.In recent years,the popularity of social networking has provided a medium through which some users can engage in offensive and harmful online behavior.In this study,we investigate the ability of different LLMs,ranging from zero-shot and few-shot learning to fine-tuning.Our experiments show that LLMs can identify sexist and hateful online texts using zero-shot and few-shot approaches through information retrieval.Furthermore,it is found that the encoder-decoder model called Zephyr achieves the best results with the fine-tuning approach,scoring 86.811%on the Explainable Detection of Online Sexism(EDOS)test-set and 57.453%on the Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter(HatEval)test-set.Finally,it is confirmed that the evaluated models perform well in hate text detection,as they beat the best result in the HatEval task leaderboard.The error analysis shows that contextual learning had difficulty distinguishing between types of hate speech and figurative language.However,the fine-tuned approach tends to produce many false positives.
基金financial support from the SERB-SURE under file number of SUR/2022/003129Jong Hyeok Park acknowledges the support of the National Research Foundation of Korea (NRF)funded by the Ministry of Science and ICT (RS-2023-00302697,RS-2023-00268523).
文摘Mo_(2)C is an excellent electrocatalyst for hydrogen evolution reaction(HER).However,Mo_(2)C is a poor electrocatalyst for oxygen evolution reaction(OER).Herein,two different elements,namely Co and Fe,are incorporated in Mo_(2)C that,therefore,has a finely tuned electronic structure,which is not achievable by incorporation of any one of the metals.Consequently,the resulting electrocatalyst Co_(0.8)Fe_(0.2)-Mo_(2)C-80 displayed excellent OER catalytic performance,which is evidenced by a low overpotential of 214.0(and 246.5)mV to attain a current density of 10(and 50)mA cm^(-2),an ultralow Tafel slope of 38.4 mV dec^(-1),and longterm stability in alkaline medium.Theoretical data demonstrates that Co_(0.8)Fe_(0.2)-Mo_(2)C-80 requires the lowest overpotential(1.00 V)for OER and Co centers to be the active sites.The ultrahigh catalytic performance of the electrocatalyst is attributed to the excellent intrinsic catalytic activity due to high Brunauer-Emmett-Teller specific surface area,large electrochemically active surface area,small Tafel slope,and low chargetransfer resistance.
文摘As the realm of enterprise-level conversational AI continues to evolve, it becomes evident that while generalized Large Language Models (LLMs) like GPT-3.5 bring remarkable capabilities, they also bring forth formidable challenges. These models, honed on vast and diverse datasets, have undoubtedly pushed the boundaries of natural language understanding and generation. However, they often stumble when faced with the intricate demands of nuanced enterprise applications. This research advocates for a strategic paradigm shift, urging enterprises to embrace a fine-tuning approach as a means to optimize conversational AI. While generalized LLMs are linguistic marvels, their inability to cater to the specific needs of businesses across various industries poses a critical challenge. This strategic shift involves empowering enterprises to seamlessly integrate their own datasets into LLMs, a process that extends beyond linguistic enhancement. The core concept of this approach centers on customization, enabling businesses to fine-tune the AI’s functionality to fit precisely within their unique business landscapes. By immersing the LLM in industry-specific documents, customer interaction records, internal reports, and regulatory guidelines, the AI transcends its generic capabilities to become a sophisticated conversational partner aligned with the intricacies of the enterprise’s domain. The transformative potential of this fine-tuning approach cannot be overstated. It enables a transition from a universal AI solution to a highly customizable tool. The AI evolves from being a linguistic powerhouse to a contextually aware, industry-savvy assistant. As a result, it not only responds with linguistic accuracy but also with depth, relevance, and resonance, significantly elevating user experiences and operational efficiency. In the subsequent sections, this paper delves into the intricacies of fine-tuning, exploring the multifaceted challenges and abundant opportunities it presents. It addresses the technical intricacies of data integration, ethical considerations surrounding data usage, and the broader implications for the future of enterprise AI. The journey embarked upon in this research holds the potential to redefine the role of conversational AI in enterprises, ushering in an era where AI becomes a dynamic, deeply relevant, and highly effective tool, empowering businesses to excel in an ever-evolving digital landscape.
文摘古汉语文本承载着丰富的历史和文化信息,对这类文本进行实体关系抽取研究并构建相关知识图谱对于文化传承具有重要作用.针对古汉语文本中存在大量生僻汉字、语义模糊和复义等问题,提出了一种基于BERT古文预训练模型的实体关系联合抽取模型(entity relation joint extraction model based on BERT-ancient-Chinese pretrained model,JEBAC).首先,通过融合BiLSTM神经网络和注意力机制的BERT古文预训练模型(BERT-ancientChinese pre-trained model integrated BiLSTM neural network and attention mechanism,BACBA),识别出句中所有的subject实体和object实体,为关系和object实体联合抽取提供依据.接下来,将subject实体的归一化编码向量与整个句子的嵌入向量相加,以更好地理解句中subject实体的语义特征;最后,结合带有subject实体特征的句子向量和object实体的提示信息,通过BACBA实现句中关系和object实体的联合抽取,从而得到句中所有的三元组信息(subject实体,关系,object实体).在中文实体关系抽取DuIE2.0数据集和CCKS 2021的文言文实体关系抽取CCLUE小样本数据集上,与现有的方法进行了性能比较.实验结果表明,该方法在抽取性能上更加有效,F1值分别可达79.2%和55.5%.
文摘在互联网时代,越来越多的财务公司选择在财经新闻平台上发表自己的见解,这些评论文本作为舆情的载体,可以充分反映财务公司的情绪,影响公众的投资决策和市场走势.情感分析为分析海量的经济类文本情感类型提供了有效的研究手段.但是,由于特定领域文本的专业性和大标签数据集的不适用性,经济类文本情感分析给传统的情感分析模型带来了巨大的挑战.当将一般情感分析模型应用于经济等特定领域时,模型在准确率与召回率上表现较差.为了克服这些挑战,文章针对财经新闻平台上的经济类文本的情感分析任务,从词表示模型出发,提出了基于知识蒸馏方法的双路BERT(Two-way BERT based on knowledge distillation method)情感分析模型,与文本卷积神经网络(Text-CNN)、卷积递归神经网络(CRNN)、双向长时和短时记忆网络(Bi-LSTM)等算法进行对比实验,结果得出该改进方法相较于其他算法在准确率、召回率和F1值均提升了1%~3%,具有较好的泛化性能.
文摘针对民航陆空通话领域语料难以获取、实体分布不均,以及意图信息提取中实体规范不足且准确率有待提升等问题,为了更好地提取陆空通话意图信息,提出一种融合本体的基于双向转换编码器(bidirectional encoder representations from transformers,BERT)与生成对抗网络(generative adversarial network,GAN)的陆空通话意图信息挖掘方法,并引入航班池信息对提取的部分信息进行校验修正,形成空中交通管制(air traffic control,ATC)系统可理解的结构化信息。首先,使用改进的GAN模型进行陆空通话智能文本生成,可有效进行数据增强,平衡各类实体信息分布并扩充数据集;然后,根据欧洲单一天空空中交通管理项目定义的本体规则进行意图的分类与标注;之后,通过BERT预训练模型生成字向量并解决一词多义问题,利用双向长短时记忆(bidirectional long short-term memory,BiLSTM)网络双向编码提取上下句语义特征,同时将该语义特征送入条件随机场(conditional random field,CRF)模型进行推理预测,学习标签的依赖关系并加以约束,以获取全局最优结果;最后,根据编辑距离(edit distance,ED)算法进行意图信息合理性校验与修正。对比实验结果表明,所提方法的宏平均F_(1)值达到了98.75%,在民航陆空通话数据集上的意图挖掘性能优于其他主流模型,为其加入数字化进程奠定了基础。
文摘源代码漏洞检测常使用代码指标、机器学习和深度学习等技术.但是这些技术存在无法保留源代码中的句法和语义信息、需要大量专家知识对漏洞特征进行定义等问题.为应对现有技术存在的问题,提出基于BERT(bidirectional encoder representations from transformers)模型的源代码漏洞检测模型.该模型将需要检测的源代码分割为多个小样本,将每个小样本转换成近似自然语言的形式,通过BERT模型实现源代码中漏洞特征的自动提取,然后训练具有良好性能的漏洞分类器,实现Python语言多种类型漏洞的检测.该模型在不同类型的漏洞中实现了平均99.2%的准确率、97.2%的精确率、96.2%的召回率和96.7%的F1分数的检测水平,对比现有的漏洞检测方法有2%~14%的性能提升.实验结果表明,该模型是一种通用的、轻量级的、可扩展的漏洞检测方法.
