Leveraging Vision-Language Pre-Trained Model and Contrastive Learning for Enhanced Multimodal Sentiment Analysis

Multimodal sentiment analysis is an essential area of research in artificial intelligence that combines multiple modes,such as text and image,to accurately assess sentiment.However,conventional approaches that rely on unimodal pre-trained models for feature extraction from each modality often overlook the intrinsic connections of semantic information between modalities.This limitation is attributed to their training on unimodal data,and necessitates the use of complex fusion mechanisms for sentiment analysis.In this study,we present a novel approach that combines a vision-language pre-trained model with a proposed multimodal contrastive learning method.Our approach harnesses the power of transfer learning by utilizing a vision-language pre-trained model to extract both visual and textual representations in a unified framework.We employ a Transformer architecture to integrate these representations,thereby enabling the capture of rich semantic infor-mation in image-text pairs.To further enhance the representation learning of these pairs,we introduce our proposed multimodal contrastive learning method,which leads to improved performance in sentiment analysis tasks.Our approach is evaluated through extensive experiments on two publicly accessible datasets,where we demonstrate its effectiveness.We achieve a significant improvement in sentiment analysis accuracy,indicating the supe-riority of our approach over existing techniques.These results highlight the potential of multimodal sentiment analysis and underscore the importance of considering the intrinsic semantic connections between modalities for accurate sentiment assessment.

Semantic Document Layout Analysis of Handwritten Manuscripts

A document layout can be more informative than merely a document’s visual and structural appearance.Thus,document layout analysis(DLA)is considered a necessary prerequisite for advanced processing and detailed document image analysis to be further used in several applications and different objectives.This research extends the traditional approaches of DLA and introduces the concept of semantic document layout analysis(SDLA)by proposing a novel framework for semantic layout analysis and characterization of handwritten manuscripts.The proposed SDLA approach enables the derivation of implicit information and semantic characteristics,which can be effectively utilized in dozens of practical applications for various purposes,in a way bridging the semantic gap and providingmore understandable high-level document image analysis and more invariant characterization via absolute and relative labeling.This approach is validated and evaluated on a large dataset ofArabic handwrittenmanuscripts comprising complex layouts.The experimental work shows promising results in terms of accurate and effective semantic characteristic-based clustering and retrieval of handwritten manuscripts.It also indicates the expected efficacy of using the capabilities of the proposed approach in automating and facilitating many functional,reallife tasks such as effort estimation and pricing of transcription or typing of such complex manuscripts.

An Analysis Model of Sports Human Body Based on Computer Vision Tracking Technology

This paper proposes the analysis model of sports human body based on computer vision tracking technology. Visual target tracking is an important research field of the computer vision, motion trajectory and it can provide not only the goal, to provide the initial data movement analysis, scene understanding, behavior or the event detection in intelligent surveillance, human-computer interaction, robot visual navigation and motion recognition based on field has a broad application prospect. For this reason, it is possible to consider the use of a large number of unlabeled samples to assist the training classifier to improve its performance. This type of machine learning method using both labeled and that unlabeled samples is called the semi-supervised learning. This paper proposes the novel idea of the related research topics to propose the new perspective of the model that will be later give us the novel idea of making it efficient for further development of sport science.

Posture Detection of Heart Disease Using Multi-Head Attention Vision Hybrid(MHAVH)Model

Cardiovascular disease is the leading cause of death globally.This disease causes loss of heart muscles and is also responsible for the death of heart cells,sometimes damaging their functionality.A person’s life may depend on receiving timely assistance as soon as possible.Thus,minimizing the death ratio can be achieved by early detection of heart attack(HA)symptoms.In the United States alone,an estimated 610,000 people die fromheart attacks each year,accounting for one in every four fatalities.However,by identifying and reporting heart attack symptoms early on,it is possible to reduce damage and save many lives significantly.Our objective is to devise an algorithm aimed at helping individuals,particularly elderly individuals living independently,to safeguard their lives.To address these challenges,we employ deep learning techniques.We have utilized a vision transformer(ViT)to address this problem.However,it has a significant overhead cost due to its memory consumption and computational complexity because of scaling dot-product attention.Also,since transformer performance typically relies on large-scale or adequate data,adapting ViT for smaller datasets is more challenging.In response,we propose a three-in-one steam model,theMulti-Head Attention Vision Hybrid(MHAVH).Thismodel integrates a real-time posture recognition framework to identify chest pain postures indicative of heart attacks using transfer learning techniques,such as ResNet-50 and VGG-16,renowned for their robust feature extraction capabilities.By incorporatingmultiple heads into the vision transformer to generate additional metrics and enhance heart-detection capabilities,we leverage a 2019 posture-based dataset comprising RGB images,a novel creation by the author that marks the first dataset tailored for posture-based heart attack detection.Given the limited online data availability,we segmented this dataset into gender categories(male and female)and conducted testing on both segmented and original datasets.The training accuracy of our model reached an impressive 99.77%.Upon testing,the accuracy for male and female datasets was recorded at 92.87%and 75.47%,respectively.The combined dataset accuracy is 93.96%,showcasing a commendable performance overall.Our proposed approach demonstrates versatility in accommodating small and large datasets,offering promising prospects for real-world applications.

Deforming analysis of sheet metal based on stereo vision and coordinate grid

A new approach based on stereo vision technology is introduced to analyzesheet metal deformation. By measuring the deformed circle grids that are printed on the sheetsurface before forming, the strain distribution of the workpiece is obtained. The measurement andanalysis results can be used to verify numerical simulation results and guide production. To getgood accuracy, some new techniques are employed: camera calibration based on genetic algorithm,feature abstraction based on self-adaptive technology, image matching based on structure feature andcamera modeling pre-constrains, and parameter calculation based on curve and surface optimization.The experimental values show that the approach proposed is rational and practical, which can providebetter measurement accuracy with less time than the conventional method.

Studies on Cold Workability Limits of Brass Using Machine Vision System and its Finite Element Analysis

Cold Workability limits of Brass were studied as a function of friction, aspect ratio and specimen geometry. Five standard shapes of the axis symmetric specimens of cylindrical with aspect ratios 1.0 and 1.5, ring, tapered and flanged were selected for the present investigation. Specimens were deformed in compression between two flat platens to predict the metal flow at room temperature. The longitudinal and oblique cracks were obtained as the two major modes of surface fractures. Cylindrical and ring specimen shows the oblique surface crack while the tapered and flanged shows the longitudinal crack. Machine Vision system using PC based video recording with a CCD camera was used to analyze the deformation of 4 X 4 mm square grid marked at mid plane of the specimen. The strain paths obtained from different specimens exhibited nonlinearity from the beginning to the end of the strain path. The circumferential stress component Os increasingly becomes tensile with continued deformation. On the other hand the axial stress Oz , increased in the very initial stages of deformation but started becoming less compressive immediately as barreling develops. The nature of hydrostatic stress on the rim of the flanged specimen was found to be tensile. Finite element software ANSYS has been applied for the analysis of the upset forming process. When the stress values obtained from finite element analysis were compared to the measurements of grids using Machine Vision system it was found that they were in close proximity.

Vector analysis of Contoura Vision for the correction of myopia and myopic astigmatism

AIM: To evaluate the visual outcomes of Contoura Vision(CV) with automatic eye tracking system in eyes with myopia and myopic astigmatism.METHODS: This prospective study included 160 eyes(80 patients) with moderate myopia and irregular astigmatism between January and August 2018. Subjects were randomly divided into CV group(80 eyes) that under went CV femtosecond laser-assisted in situ keratomileusis(FS-LASIK) and a control group(80 eyes) that underwent wavefrontoptimized FS-LASIK. Visual outcomes and astigmatic vector analysis were evaluated and compared between preoperatively and 3 mo postoperatively. RESULTS: Basic details were similar in both groups(P>0.05). At 3 mo postoperatively, uncorrected distance visual acuity was 20/16, 20/20, and 20/25 in 24, 76, and 80 eyes of patients in CV group, respectively. The CV group was better in predictability of astigmatism correction at 3 mo postoperatively. In CV group, 64 eyes had deviation of astigmatic axis within 15° and 28 eyes had deviation of astigmatic axis within 5°, both were better than those in the control group. The number of eyes with residual astigmatism within 0.5 D were less in CV group(48 eyes) than the control group(40 eyes). Compared with the preoperative, C7 significantly reduced to 0.056±0.030 in CV group at 3 mo after the procedure(P<0.05), and were significantly lower than those in the control group(P<0.05).CONCLUSION: CV with automatic eye tracking system is safe and effective for the correction of myopia and myopic astigmatism.

Review of Computer Vision Applications in Fabric Recognition and Color Analysis

Detecting various parameters of woven fabrics is one of the important methods to evaluate the quality of fabrics.In the early stage of industrial development,fabrics were mainly relied on manual to determine the quality,which was inefficient and unstable,so intelligent inspection is a popular development trend today.In recent years,computer vision technology has been widely used in the fields of fabric density measurement,color analysis,and weave pattern recognition.Based on the above three aspects,the advanced research progress of global researchers is reviewed in this paper and the shortcomings of current research and possible research directions in the future are analyzed.Computer vision technology is not only objective evaluation,but also has the advantages of accuracy and efficiency,and has a good development prospect in the field of textiles.

Vision-based Dynamics Monitoring(VDM)for Diagnosing the Variations of Wind Turbine Tower Foundation Conditions

A slight uneven settlement of the foundation may cause the wind turbine to shake,tilt,or even collapse,so it is increasingly necessary to realize remote condition monitoring of the foundations.At present,the wind turbine foundation monitoring system is incomplete.The current monitoring research of the tower foundation is mainly of contact measurements,using acceleration sensors and static-level sensors for monitoring multiple reference points.Such monitoring methods will face some disadvantages,such as the complexity of monitoring deployment,the cost of manpower,and the load effect on the tower structure.To solve above issues,this paper aims to investigate wind turbine tower foundation variation dynamic monitoring based on machine vision.Machine vision monitoring is a kind of noncontact measurement,which helps to realize comprehensive diagnosis of early foundation uneven settlement and loose faults.The FEA model is firstly investigated as the theoretical foundation to investigate the dynamics of the tower foundation.Second,the Gaussian-based vibration detection is adopted by tracking the tower edge points.Finally,a tower structure with distributed foundation support is tested.The modal parameters obtained from the visual measurement are compared with those from the accelerometer,proving the vision method can effectively monitor the issues with tower foundation changes.

Fish Behavior Analysis Based on Computer Vision: A Survey

Fish behavior refers to various movements of fish. Fish behavior is closely related to the ecology of fish, physiological changes of fish, aquaculture and so on. Related applications will be expanded if fish behavior is analyzed properly. Traditional analysis of fish behavior mainly relies on the observation of human eyes. With the deepening and extension of application and the rapid development of computer technology, computer vision technology is increasingly used to analyze fish behaviors. This paper summarized the research status, research progress and main problems of fish behavior analysis by using computer vision and made forecast about future research.

Correlation analysis-based image segmentation approach for automatic agriculture vehicle

It is important to segment image correctly to extract guidance information for automatic agriculture vehicle. If we can make the computer know where the crops are, we can extract the guidance line easily. Images were divided into some rec-tangle small windows, then a pair of 1-D arrays was constructed in each small windows. The correlation coefficients of every small window constructed the features to segment images. The results showed that correlation analysis is a potential approach for processing complex farmland for guidance system, and more correlation analysis methods must be researched.

Prevalence of color vision deficiency among Chinese college students and their quality of life

AIM:To investigate the prevalence of color vision deficiency(CVD)among college students and their quality of life(QoL)in a Chinese college.METHODS:This cross-sectional study was performed in Sichuan University in Chengdu,China.The questionnaire containing participants’demographic data,as well as CVD related QoL was distributed to students who were screened as CVD[by Color Vision Examination Plates(Version 6)]in 2022 freshman entrance examination.Color blindness QoL(CBQoL)and utility analysis were used to evaluate the QoL of CVD students.RESULTS:There were 381 of 17303(2.20%)students screened as CVD,including 368(4.11%)males and 13(0.16%)females.A total of 317 students completed the questionnaire,the response rate was 83.20%.Only 166 participants(52.3%)knew they have CVD before the physical examination for college entrance examination,while a total of 145 participants(45.74%)hoped to be diagnosed earlier.The medians of CBQoL score and utility were 5.85(range 2.2-6)and 1(range 0-1),respectively.The proportions of students whose QoL is affected by CVD were 67.63%(211/312)and 42.27%(134/317)measured by CBQoL and utility analysis respectively.CONCLUSION:The prevalence of CVD in males is much higher than that in females.The time when CVD students first became aware of their CVD is relatively late.The QoL of the study population is quite high,while a large proportion of students’QoL are affected by CVD.It is suggested that as a congenital defect,CVD screening in China should be earlier,and appropriate guidance and support are needed for CVD patients to help them better adapt to life,study,and work.

Sparse Crowd Flow Analysis of Tawaaf of Kaaba During the COVID-19 Pandemic

The advent of the COVID-19 pandemic has adversely affected the entire world and has put forth high demand for techniques that remotely manage crowd-related tasks.Video surveillance and crowd management using video analysis techniques have significantly impacted today’s research,and numerous applications have been developed in this domain.This research proposed an anomaly detection technique applied to Umrah videos in Kaaba during the COVID-19 pandemic through sparse crowd analysis.Managing theKaaba rituals is crucial since the crowd gathers from around the world and requires proper analysis during these days of the pandemic.The Umrah videos are analyzed,and a system is devised that can track and monitor the crowd flow in Kaaba.The crowd in these videos is sparse due to the pandemic,and we have developed a technique to track the maximum crowd flow and detect any object(person)moving in the direction unlikely of the major flow.We have detected abnormal movement by creating the histograms for the vertical and horizontal flows and applying thresholds to identify the non-majority flow.Our algorithm aims to analyze the crowd through video surveillance and timely detect any abnormal activity tomaintain a smooth crowd flowinKaaba during the pandemic.

Estimation of Low Nutrients in Tomato Crops Through the Analysis of Leaf Images Using Machine Learning

Tomato crops are considered the most important agricultural products worldwide.However,the quality of tomatoes depends mainly on the nutrient levels.Visual inspection is made by farmers to anticipate the nutrient deficiency of the plants.Recently,precision agriculture has explored opportunities to automate nutrient level monitoring.Previous work has demonstrated that a convolutional neural network is able to estimate low nutrients in tomato plants using images of their leaves.However,the performance of the convolutional neural network was not adequate.Thus,this work proposes a novel convolutional neural network-based classifier,namely,CNN+AHN,for estimating low nutrients in tomato crops using an image of the tomato leaves.The CNN+AHN incorporates a set of convolutional layers as the feature extraction part,and a supervised learning method called artificial hydrocarbon network as the dense layer.Different combinations of the architecture of CNN+AHN were examined.Experimental results showed that our best CNN+AHN classifier is able to estimate low nutrients in tomato plants with an accuracy of 95.57%and F1-score of 95.75%,outperforming the literature.

Human and Machine Vision Based Indian Race Classification Using Modified-Convolutional Neural Network

The inter-class face classification problem is more reasonable than the intra-class classification problem.To address this issue,we have carried out empirical research on classifying Indian people to their geographical regions.This work aimed to construct a computational classification model for classifying Indian regional face images acquired from south and east regions of India,referring to human vision.We have created an Automated Human Intelligence System(AHIS)to evaluate human visual capabilities.Analysis of AHIS response showed that face shape is a discriminative feature among the other facial features.We have developed a modified convolutional neural network to characterize the human vision response to improve face classification accuracy.The proposed model achieved mean F1 and Matthew Correlation Coefficient(MCC)of 0.92 and 0.84,respectively,on the validation set,outperforming the traditional Convolutional Neural Network(CNN).The CNN-Contoured Face(CNN-FC)model is developed to train contoured face images to investigate the influence of face shape.Finally,to cross-validate the accuracy of these models,the traditional CNN model is trained on the same dataset.With an accuracy of 92.98%,the Modified-CNN(M-CNN)model has demonstrated that the proposed method could facilitate the tangible impact in intra-classification problems.A novel Indian regional face dataset is created for supporting this supervised classification work,and it will be available to the research community.

Night Vision Object Tracking System Using Correlation Aware LSTM-Based Modified Yolo Algorithm

Improved picture quality is critical to the effectiveness of object recog-nition and tracking.The consistency of those photos is impacted by night-video systems because the contrast between high-profile items and different atmospheric conditions,such as mist,fog,dust etc.The pictures then shift in intensity,colour,polarity and consistency.A general challenge for computer vision analyses lies in the horrid appearance of night images in arbitrary illumination and ambient envir-onments.In recent years,target recognition techniques focused on deep learning and machine learning have become standard algorithms for object detection with the exponential growth of computer performance capabilities.However,the iden-tification of objects in the night world also poses further problems because of the distorted backdrop and dim light.The Correlation aware LSTM based YOLO(You Look Only Once)classifier method for exact object recognition and deter-mining its properties under night vision was a major inspiration for this work.In order to create virtual target sets similar to daily environments,we employ night images as inputs;and to obtain high enhanced image using histogram based enhancement and iterative wienerfilter for removing the noise in the image.The process of the feature extraction and feature selection was done for electing the potential features using the Adaptive internal linear embedding(AILE)and uplift linear discriminant analysis(ULDA).The region of interest mask can be segmen-ted using the Recurrent-Phase Level set Segmentation.Finally,we use deep con-volution feature fusion and region of interest pooling to integrate the presently extremely sophisticated quicker Long short term memory based(LSTM)with YOLO method for object tracking system.A range of experimentalfindings demonstrate that our technique achieves high average accuracy with a precision of 99.7%for object detection of SSAN datasets that is considerably more than that of the other standard object detection mechanism.Our approach may therefore satisfy the true demands of night scene target detection applications.We very much believe that our method will help future research.

面向交叉创新人才培养的“人工智能+仪器分析”课程教学探索

人工智能与仪器分析技术的交叉创新与应用,对培养创新型人才提出了新的挑战。通过分析“人工智能+仪器分析”现有教学方式的不足,探索针对人工智能与仪器分析交叉领域的课程教学的解决方案,提高仪器分析专业学生掌握人工智能技术交叉创新的能力,为培养应用型和研究型创新人才提供参考。

低视力患者一般自我效能感LPA及其与生存质量的关系研究

目的探究低视力患者一般自我效能感潜在类别,探讨不同潜在类别与生存质量的关系及生存质量的影响因素。方法采用横断面临床研究设计,纳入2021年5月—2022年4月在首都医科大学附属北京同仁医院就诊的低视力患者234例(289只眼),采用一般自我效能感量表(GSES)和中文版低视力生存质量量表(CLVQOL)进行调查,使用Mplus8.0和SPSS 26.0统计学软件分析一般自我效能感潜在类别及与生存质量的关系,采用多重线性回归法分析生存质量的影响因素。结果(1)剖面分析结果:当一般自我效能感潜在剖面数为3时,艾凯克信息准则(AIC)、贝叶斯信息准则(BIC)、调整贝叶斯信息准则(aBIC)值最大,entropy更接近1,剖面3为最佳模型。(2)类别特点及命名:本研究中的低视力患者分为3个潜在类别,类别1(23.9%)一般自我效能感水平最高,命名为高效能感型;类别2(54.7%)位于整体得分的中间水平,命名为中效能感型;类别3(21.4%)一般自我效能感得分最低,命名为低效能感型。(3)类别的基本特征:一般自我效能感不同潜在类别的低视力患者基本特征比较显示,年龄(χ^(2)=16.727,P=0.033)、文化程度(χ^(2)=11.389,P=0.023)、医疗费用来源(χ^(2)=27.914,P=0.000)、户籍类型(χ^(2)=16.503,P=0.002)、视力损伤时长(χ^(2)=36.607,P=0.000)、职业类型(χ^(2)=11.207,P=0.004)、视力(χ^(2)=19.433,P=0.001)和既往眼部手术史(χ^(2)=8.293,P=0.016)在3个不同潜在类别间的差异均有统计学意义,其他社会人口学特征差异均无统计学意义(P>0.05)。(4)类别与生存质量的关系:不同自我效能感潜在类别的低视力患者生存质量总分比较,差异有统计学意义(F=8.187,P=0.000),生存质量各维度比较显示,远视力、移动、光感(F=6.057,P=0.003)、阅读和精细工作(F=9.648,P=0.000)、调节能力(F=5.214,P=0.007)、日常生活能力(F=8.164,P=0.000),差异均有统计学意义。(5)生存质量影响因素的多因素分析:视力(t=-6.582,P=0.000)、一般自我效能感不同潜在类别(t=-2.092,P=0.039)为生存质量的独立影响因素,差异均有统计学意义。结论低视力患者的一般自我效能感存在异质性,应重点关注低效能感型和视力受损严重的低视力患者,探索一般自我效能感干预策略,提高其生存质量水平。

机器视觉技术在大坝表面变形监测中的适用性研究

针对大坝表面变形自动化监测中GNSS系统垂直位移监测精度低、费用昂贵和静力水准仪布设难度大、维修成本高等问题,利用机器视觉技术对大坝表面变形进行监测。室内试验结果表明,相机与标靶距离在100 m范围内,机器视觉的测量精度可达到亚毫米级,且水平和垂直变形位移精度相近。由误差分析可知,标靶与相机的距离与测量精度有明显的相关性,而环境因素是影响远距离监测精度的重要因素,并利用大坝视准线原理优化了机器视觉监测系统。最后以浙江省大力塘水库为试点,对大坝表面变形进行实时监测,结果表明,机器视觉技术完全可满足实际工程需求,且优化后的机器视觉测量技术可明显提高监测数据的精度和稳定性。

基于计算机视觉的工业人员行为分析实验平台

该文面向本科生与研究生实践教学,利用计算机视觉技术开发了工业人员行为分析实验平台。依托工程实训中心,通过多通道数据采集装置收集人员工作环境、作业行为以及劳保用品等数据。基于PyQt5构建工业人员行为分析实验平台,该平台将计算机视觉技术与工业安全管控标准深度融合,以实现工业人员流程类与非流程类作业的行为分析。为验证实验平台的有效性,使用工业人员行为测试数据对所研发的实验平台进行了实验验证。实验结果表明,该实验平台能对工人多种作业进行有效评判。该实验平台有助于提升本科生和研究生在智能图像处理领域的实践能力。