Smaller & Smarter: Score-Driven Network Chaining of Smaller Language Models

With the continuous evolution and expanding applications of Large Language Models (LLMs), there has been a noticeable surge in the size of the emerging models. It is not solely the growth in model size, primarily measured by the number of parameters, but also the subsequent escalation in computational demands, hardware and software prerequisites for training, all culminating in a substantial financial investment as well. In this paper, we present novel techniques like supervision, parallelization, and scoring functions to get better results out of chains of smaller language models, rather than relying solely on scaling up model size. Firstly, we propose an approach to quantify the performance of a Smaller Language Models (SLM) by introducing a corresponding supervisor model that incrementally corrects the encountered errors. Secondly, we propose an approach to utilize two smaller language models (in a network) performing the same task and retrieving the best relevant output from the two, ensuring peak performance for a specific task. Experimental evaluations establish the quantitative accuracy improvements on financial reasoning and arithmetic calculation tasks from utilizing techniques like supervisor models (in a network of model scenario), threshold scoring and parallel processing over a baseline study.

Research on the key technology of mobile ERP

A new business management mode-agile & convenient management mode (ACM) was proposed because of large scale mobile computing and mobile business. As an efficient approach to ACM, Mobile ERP (MERP), including its concept and four-tier architecture was introduced firstly. Then, the two key technologies of implementing MERP about message service and wireless WAP service were developed. Meanwhile, an official MERP system was also implemented based on our CERP software system by aglets. And it was applied by Henan Tongli Cement Group in China. In comparison with the traditional ERP system, the new system could improve the efficiency and level of management, and reduce the management cost of enterprise.

Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

GUARDIAN: A Multi-Tiered Defense Architecture for Thwarting Prompt Injection Attacks on LLMs

This paper introduces a novel multi-tiered defense architecture to protect language models from adversarial prompt attacks. We construct adversarial prompts using strategies like role emulation and manipulative assistance to simulate real threats. We introduce a comprehensive, multi-tiered defense framework named GUARDIAN (Guardrails for Upholding Ethics in Language Models) comprising a system prompt filter, pre-processing filter leveraging a toxic classifier and ethical prompt generator, and pre-display filter using the model itself for output screening. Extensive testing on Meta’s Llama-2 model demonstrates the capability to block 100% of attack prompts. The approach also auto-suggests safer prompt alternatives, thereby bolstering language model security. Quantitatively evaluated defense layers and an ethical substitution mechanism represent key innovations to counter sophisticated attacks. The integrated methodology not only fortifies smaller LLMs against emerging cyber threats but also guides the broader application of LLMs in a secure and ethical manner.

Whispered Tuning: Data Privacy Preservation in Fine-Tuning LLMs through Differential Privacy

The proliferation of Large Language Models (LLMs) across various sectors underscored the urgency of addressing potential privacy breaches. Vulnerabilities, such as prompt injection attacks and other adversarial tactics, could make these models inadvertently disclose their training data. Such disclosures could compromise personal identifiable information, posing significant privacy risks. In this paper, we proposed a novel multi-faceted approach called Whispered Tuning to address privacy leaks in large language models (LLMs). We integrated a PII redaction model, differential privacy techniques, and an output filter into the LLM fine-tuning process to enhance confidentiality. Additionally, we introduced novel ideas like the Epsilon Dial for adjustable privacy budgeting for differentiated Training Phases per data handler role. Through empirical validation, including attacks on non-private models, we demonstrated the robustness of our proposed solution SecureNLP in safeguarding privacy without compromising utility. This pioneering methodology significantly fortified LLMs against privacy infringements, enabling responsible adoption across sectors.

Geographic,Geometrical and Semantic Reconstruction of Urban Scene from High Resolution Oblique Aerial Images

An effective approach is proposed for 3D urban scene reconstruction in the form of point cloud with semantic labeling. Starting from high resolution oblique aerial images,our approach proceeds through three main stages: geographic reconstruction, geometrical reconstruction and semantic reconstruction. The absolute position and orientation of all the cameras relative to the real world are recovered in the geographic reconstruction stage. Then, in the geometrical reconstruction stage,an improved multi-view stereo matching method is employed to produce 3D dense points with color and normal information by taking into account the prior knowledge of aerial imagery.Finally the point cloud is classified into three classes(building,vegetation, and ground) by a rule-based hierarchical approach in the semantic reconstruction step. Experiments on complex urban scene show that our proposed 3-stage approach could generate reasonable reconstruction result robustly and efficiently.By comparing our final semantic reconstruction result with the manually labeled ground truth, classification accuracies from86.75% to 93.02% are obtained.

A hybrid neural network system for prediction and recognition of promoter regions in human genome

This paper proposes a high specificity and sensitivity algorithm called PromPredictor for recognizing promoter regions in the human genome. PromPredictor extracts compositional features and CpG islands information from genomic sequence,feeding these features as input for a hybrid neural network system (HNN) and then applies the HNN for prediction. It combines a novel promoter recognition model, coding theory, feature selection and dimensionality reduction with machine learning algorithm.Evaluation on Human chromosome 22 was ～66% in sensitivity and ～48% in specificity. Comparison with two other systems revealed that our method had superior sensitivity and specificity in predicting promoter regions. PromPredictor is written in MATLAB and requires Matlab to run. PromPredictor is freely available at http://www.whtelecom.com/Prompredictor.htm.

Random Walk Based Method for Automatic Segmentation of Intravascular Ultrasound Images

Intravascular ultrasound( IVUS) is an important imaging technique that is used to study vascular wall architecture for diagnosis and assessment of the vascular diseases. Segmentation of lumen and media-adventitia boundaries from IVUS images is a basic and necessary step for quantitative assessment of the vascular walls.Due to ultrasound speckles, artifacts and individual differences,automated segmentation of IVUS images represents a challenging task. In this paper,a random walk based method is proposed for fully automated segmentation of IVUS images. Robust and accurate determination of the seed points for different regions is the key to successful use of the random walk algorithm in segmentation of IVUS images and is the focus of the present work. Performance of the proposed algorithm was evaluated over an image database with 900 IVUS image frames of nine patient cases. The preliminary experimental results show the potential of the proposed IVUS image segmentation approach.

Design and Implementation of Memory Access Fast Switching Structure in Cluster-Based Reconfigurable Array Processor

Memory access fast switching structures in cluster are studied,and three kinds of fast switching structures（ FS,LR2 SS,and LAPS） are proposed. A mixed simulation test bench is constructed and used for statistic of data access delay among these three structures in various cases. Finally these structures are realized on Xilinx FPGA development board and DCT,FFT,SAD,IME,FME,and de-blocking filtering algorithms are mapped onto the structures. Compared with available architectures,our proposed structures have lower data access delay and lower area.

Outdoor guide system based on the mobile augmented reality technology

This paper proposes an outdoor guide system using vision-based augmented reality(AR) on mobile devices.Augmented reality provides a virtual-real fusion display interface for outdoor guide.Vision-based methods are more accurate than GPS or other hardware-based methods.However,vision-based methods require more resources and relatively strong computing power of mobile devices.A C/S framework for vision based augmented reality system is introduced in this paper.In a server,a vocabulary tree is used for location recognition.In a mobile device,BRISK feature is combined with optical flow methods to track the offline keyframe.The system is tested on UKbench datasets and in real environment.Experimental results show that the proposed vision-based augmented reality system works well and yields relatively high recognition rate and that the mobile device achieves realtime recognition performance.

The Effect of Online-to-Mobile Trust Transfer on the Foundation of Mobile Banking Trust

The paper mainly studies the influences of trust transfer on the establishment of consumers’ initial trust. Based on the theory of signal transmission and self-efficiency, the study builds a trust transfer model aiming at the same subject between different environments. The results shows that when the consumers’ perceived change of environment is little, prior successful experiences will improve the consumers’ perceptions of self-ability, which probably lowers the effect of bank’s role on the establishment of initial trust. Therefore, banks should cultivate consumers’ perceptions of their relative advantages in the original environment and thus improve the consumers’ dependency in the new environment to avoid the loss of consumers and build a long-term relationship.

Analysis of Multi-Scale Fractal Dimension for Image Interpolation

This article presents a novel image interpolation based on rational fractal fimction. The rational function has a simple and explicit expression. At the same time, the fi＇actal interpolation surface can be defined by proper parameters. In this paper, we used the method of ＇covering blanket＇ combined with multi-scale analysis; the threshold is selected based on the multi-scale analysis. Selecting different parameters in the rational function model, the texture regions and smooth regions are interpolated by rational fractal interpolation and rational interpolation respectively. Experimental results on benchmark test images demonstrate that the proposed method achieves very competitive performance compared with the state-of-the-art interpolation algorithms, especially in image details and texture features.

基于数据驱动EKF算法的电动汽车及插电式混合动力汽车动力电池SOC估计（英文）

电池荷电状态(SOC)估计一直是动力电池和新能源汽车(含电动汽车(EV)、插电式电动汽车(PDEV)等)领域的研究热点。本文针对传统EKF(扩展卡尔曼滤波)算法对于动力电池模型精确性要求与全生命周期内的动态性要求之间的矛盾问题,提出了一种基于数据驱动EKF算法的SOC估计方法。该方法的创新之处在于:数据驱动的EKF动力电池SOC估计可以在动力电池内部参数未知的情况下,利用可观测的端电压和电流数据,实现动力电池SOC的在线准确估计。通过将基于数据的SOC估算方法与基于模型的SOC估计算法有效结合,有效避免了传统基于数据的SOC估计算法的累积误差问题以及基于模型的SOC估计算法对于电池模型精度的依赖性,使其更加适用于环境复杂多变(例如EV和PHEV)的动力电池全生命周期SOC估算。实验结果表明,本文所提出的基于数据驱动的EKF动力电池SOC估计算法具有更好的鲁棒性和实用性。

一种新的融合LCGMM与改进LFDA的多模态过程监测方法（英文）

Complex processes often work with multiple operation regions, it is critical to develop effective monitoring approaches to ensure the safety of chemical processes. In this work, a discriminant local consistency Gaussian mixture model(DLCGMM) for multimode process monitoring is proposed for multimode process monitoring by integrating LCGMM with modified local Fisher discriminant analysis(MLFDA). Different from Fisher discriminant analysis(FDA) that aims to discover the global optimal discriminant directions, MLFDA is capable of uncovering multimodality and local structure of the data by exploiting the posterior probabilities of observations within clusters calculated from the results of LCGMM. This may enable MLFDA to capture more meaningful discriminant information hidden in the high-dimensional multimode observations comparing to FDA. Contrary to most existing multimode process monitoring approaches, DLCGMM performs LCGMM and MFLDA iteratively, and the optimal subspaces with multi-Gaussianity and the optimal discriminant projection vectors are simultaneously achieved in the framework of supervised and unsupervised learning. Furthermore, monitoring statistics are established on each cluster that represents a specific operation condition and two global Bayesian inference-based fault monitoring indexes are established by combining with all the monitoring results of all clusters. The efficiency and effectiveness of the proposed method are evaluated through UCI datasets, a simulated multimode model and the Tennessee Eastman benchmark process.

Privacy-Preserving Relative Location Based Services for Mobile Users

Location-aware applications have been used widely with the assistance of the latest positioning features in Smart Phone such as GPS,AGPS,etc.However,all the existing applications gather users' geographical data and transfer them into the pertinent information to give meaning and value.For this kind of solutions,the user's privacy and security issues might be raised because the geographical location has to be exposed to the service provider.A novel and practical solution is proposed in this article to provide the relative location of two mobile users based on their WiFi scanned results without any additional sensors.There is no privacy concern in this solution because end users will not collect and send any sensitive information to the server.This solution adopts a Client/Server(C/S) architecture,where the mobile user as a client reports the ambient WiFi APs and the server calculates the distances based on the WiFi AP's topological relationships.A series of technologies are explored to improve the accuracy of the estimated distance and the corresponding algorithms are proposed.We also prove the feasibility with the prototype of "Circle Your Friends" System(CYFS) on Android phone which lets the mobile user know the distance between him and his social network friends.

An application-layer based centralized information access control for VPN

With the rapid development of Virtual Private Network (VPN), many companies and organizations use VPN to implement their private communication. Traditionally, VPN uses security protocols to protect the confidentiality of data, the message integrity and the endpoint authentication. One core technique of VPN is tunneling, by which clients can access the in- ternal servers traversing VPN. However, the tunneling technique also introduces a concealed security hole. It is possible that if one vicious user can establish tunneling by the VPN server, he can compromise the internal servers behind the VPN server. So this paper presents a novel Application-layer based Centralized Information Access Control (ACIAC) for VPN to solve this problem. To implement an efficient, flexible and multi-decision access control model, we present two key techniques to ACIAC—the centralized management mechanism and the stream-based access control. Firstly, we implement the information center and the constraints/events center for ACIAC. By the two centers, we can provide an abstract access control mechanism, and the material access control can be decided dynamically by the ACIAC’s constraint/event mechanism. Then we logically classify the VPN communication traffic into the access stream and the data stream so that we can tightly couple the features of VPN communication with the access control model. We also provide the design of our ACIAC prototype in this paper.

Plant Disease Detection with Deep Learning and Feature Extraction Using Plant Village

Nowadays, crop diseases are a crucial problem to the world’s food supplies, in a world where the population count is around 7 billion people, with more than 90% not getting access to the use of tools or features that would identify and solve the problem. At present, we live in a world dominated by technology on a significant scale, major network coverage, high-end smart-phones, as long as there are great discoveries and improvements in AI. The combination of high-end smart-phones and computer vision via Deep Learning has made possible what can be defined as “smartphone-assisted disease diagnosis”. In the area of Deep Learning, multiple architecture models have been trained, some achieving performance reaching more than 99.53% [1]. In this study, we evaluate CNN’s architectures applying transfer learning and deep feature extraction. All the features obtained will also be classified by SVM and KNN. Our work is feasible by the use of the open source Plant Village Dataset. The result obtained shows that SVM is the best classifier for leaf’s diseases detection.

A new model for verification

Formal verification is playing a significant role in IC design.However,the common models for verification either have their complexity problems or have applicable limitations.In order to overcome the deficiencies,a novel model-WGL(Weighted Generalized List)is proposed,which is based on the general-list decomposition of polynomials,with three different weights and manipulation rules introduced to effect node sharing and the canonicity.Timing parameters and operations on them are also considered.Examples show the word-level WGL is the only model to linearly represent the common word-level functions and the bit-level WGL is especially suitable for arithmetic intensive circuits.The model is proved to be a uniform and efficient model for both bit-level and word-level functions.Then based on the WGL model,a backward-construction verification approach is proposed,which reduces time and space complexity for multipliers to polynomial complexity(time complexity is less than O(n3.6)and space complexity is less than O(n1.5))without hierarchical partitioning.Both the model and the verification method show their theoretical and applicable significance in IC design.

Policy driven and multi-agent based fault tolerance for Web services

This paper proposes a policy driven and multi-agent based model to enhance the fault tolerance and recovery capabilities of Web services in distributed environment. The evaluation function of fault specifications and the corresponding handling mechanisms of the services are both defined in policies, which are expressed in XML. During the implementation of the services,the occurrences of faults are monitored by the service monitor agent through the local knowledge on the faults. Such local knowledge is dynamically generated by the service policy agent through querying and parsing the service policies from the service policies repository. When the fault occurs, the service process agent will focus on the process of fault handling and service recovery, which will be directed with the actions defined in the policies upon the specific conditions. Such a policy driven and multi-agent based fault handling approach can address the issues of flexibility, automation and availability.

New multi-layer data correlation algorithm for multi-passive-sensor location system

Under the scenario of dense targets in clutter, a multi-layer optimal data correlation algorithm is proposed. This algorithm eliminates a large number of false location points from the assignment process by rough correlations before we calculate the correlation cost, so it avoids the operations for the target state estimate and the calculation of the correlation cost for the false correlation sets. In the meantime, with the elimination of these points in the rough correlation, the disturbance from the false correlations in the assignment process is decreased, so the data correlation accuracy is improved correspondingly. Complexity analyses of the new multi-layer optimal algorithm and the traditional optimal assignment algorithm are given. Simulation results show that the new algorithm is feasible and effective.