Evaluating Privacy Leakage and Memorization Attacks on Large Language Models (LLMs) in Generative AI Applications

The recent interest in the deployment of Generative AI applications that use large language models (LLMs) has brought to the forefront significant privacy concerns, notably the leakage of Personally Identifiable Information (PII) and other confidential or protected information that may have been memorized during training, specifically during a fine-tuning or customization process. We describe different black-box attacks from potential adversaries and study their impact on the amount and type of information that may be recovered from commonly used and deployed LLMs. Our research investigates the relationship between PII leakage, memorization, and factors such as model size, architecture, and the nature of attacks employed. The study utilizes two broad categories of attacks: PII leakage-focused attacks (auto-completion and extraction attacks) and memorization-focused attacks (various membership inference attacks). The findings from these investigations are quantified using an array of evaluative metrics, providing a detailed understanding of LLM vulnerabilities and the effectiveness of different attacks.

Parallel Inference for Real-Time Machine Learning Applications

Hyperparameter tuning is a key step in developing high-performing machine learning models, but searching large hyperparameter spaces requires extensive computation using standard sequential methods. This work analyzes the performance gains from parallel versus sequential hyperparameter optimization. Using scikit-learn’s Randomized SearchCV, this project tuned a Random Forest classifier for fake news detection via randomized grid search. Setting n_jobs to -1 enabled full parallelization across CPU cores. Results show the parallel implementation achieved over 5× faster CPU times and 3× faster total run times compared to sequential tuning. However, test accuracy slightly dropped from 99.26% sequentially to 99.15% with parallelism, indicating a trade-off between evaluation efficiency and model performance. Still, the significant computational gains allow more extensive hyperparameter exploration within reasonable timeframes, outweighing the small accuracy decrease. Further analysis could better quantify this trade-off across different models, tuning techniques, tasks, and hardware.

基于改进Kinky Inference的输出调节自适应无拖曳控制

在空间引力波探测任务中,航天器内部检验质量因存在载荷硬件噪声、环境噪声及微推力器耦合噪声等复杂干扰,影响其无拖曳控制精度,难以实现超净、超稳控制需求。提出一种基于惰性适配Lipschitz常数Kinky Inference (LACKI)的航天器自适应无拖曳控制方法,运用监督学习规则实现先验知识不足、样本数据存在损坏时外界干扰的逼近和抑制,及基于输出调节的模型参考自适应控制(MRAC)方法实现检验质量精确的无拖曳控制。数值仿真验证了无拖曳控制中敏感轴平动和转动自由度的状态响应性能及LACKI规则针对外界干扰的估计效果,通过与常规线性控制方法的对比,验证了所提方法对于提高无拖曳控制精度的有效性。

APPLICATION OF FUZZY INFERENCE IN IDENTIFICATION OF HELICOPTER MODEL

Helicopter mathematical model mainly depends on design helicopter control system, flight simulator, and real time control simulation system. But it is difficult to establish a helicopter flight dynamics mathematical model that has features such as rapidness, reliability and precision, because there is no unique and precise expression to some sophisticated phenomenon of helicopter. In this paper a fuzzy helicopter flight model is constructed based on the flight experimental data. The fuzzy model, which is identified by fuzzy inference, has characteristics of computed rapidness and high precision. In order to guarantee the precision of the identified fuzzy model, a new method is adopted to handle the conflict fuzzy rules. Additionally, using fuzzy clustering technology can effectively reduce the number of rules of fuzzy model, namely, the order of the fuzzy model. The simulation results indicate that the method of this paper is effective and feasible.

A Slice Analysis-Based Bayesian Inference Dynamic Power Model for CMOS Combinational Circuits

To improve the accuracy and speed in cycle-accurate power estimation, this paper uses multiple dimensional coefficients to build a Bayesian inference dynamic power model. By analyzing the power distribution and internal node state, we find the deficiency of only using port information. Then, we define the gate level number computing method and the concept of slice, and propose using slice analysis to distill switching density as coefficients in a special circuit stage and participate in Bayesian inference with port information. Experiments show that this method can reduce the power-per-cycle estimation error by 21.9% and the root mean square error by 25.0% compared with the original model, and maintain a 700 ＋ speedup compared with the existing gate-level power analysis technique.

Ensemble Bayesian method for parameter distribution inference:application to reactor physics

The estimation of model parameters is an important subject in engineering.In this area of work,the prevailing approach is to estimate or calculate these as deterministic parameters.In this study,we consider the model parameters from the perspective of random variables and describe the general form of the parameter distribution inference problem.Under this framework,we propose an ensemble Bayesian method by introducing Bayesian inference and the Markov chain Monte Carlo(MCMC)method.Experiments on a finite cylindrical reactor and a 2D IAEA benchmark problem show that the proposed method converges quickly and can estimate parameters effectively,even for several correlated parameters simultaneously.Our experiments include cases of engineering software calls,demonstrating that the method can be applied to engineering,such as nuclear reactor engineering.

An Initial Residual Stress Inference Method by Incorporating Monitoring Data and Mechanism Model

Initial residual stress is the main reason causing machining deformation of the workpiece,which has been deemed as one of the most important aspects of machining quality issues.The inference of the distribution of initial residual stress inside the blank has significant meaning for machining deformation control.Due to the principle error of existing residual stress detection methods,there are still challenges in practical applications.Aiming at the detection problem of the initial residual stress field,an initial residual stress inference method by incorporating monitoring data and mechanism model is proposed in this paper.Monitoring data during machining process is used to represent the macroscopic characterization of the unbalanced residual stress,and the finite element numerical model is used as the mechanism model so as to solve the problem that the analytic mechanism model is difficult to establish;the policy gradient approach is introduced to solve the gradient descent problem of the combination of learning model and mechanism model.Finally,the initial residual stress field is obtained through iterative calculation based on the fusing method of monitoring data and mechanism model.Verification results show that the proposed inference method of initial residual stress field can accurately and effectively reflect the machining deformation in the actual machining process.

The Jena-Based Ontology Model Inference andRetrieval Application

Ontology as an important representation model of semantic web has valuable application. A new ontology model on the basis of Computer Graphics (CG) knowledge is proposed, called CG ontology model. The protégé is used to build this ontology model conveniently. The Jena API is applied to store CG owl documents in MySQL, set inference rule and achieve search queries on the ontology database. Finally, the Jena-based ontology model retrieval system is developed.

Empirical Likelihood Inference for Generalized Partially Linear Models with Longitudinal Data

In this article, we propose a generalized empirical likelihood inference for the parametric component in semiparametric generalized partially linear models with longitudinal data. Based on the extended score vector, a generalized empirical likelihood ratios function is defined, which integrates the within-cluster?correlation meanwhile avoids direct estimating the nuisance parameters in the correlation matrix. We show that the proposed statistics are asymptotically?Chi-squared under some suitable conditions, and hence it can be used to construct the confidence region of parameters. In addition, the maximum empirical likelihood estimates of parameters and the corresponding asymptotic normality are obtained. Simulation studies demonstrate the performance of the proposed method.

Causal inference with marginal structural modeling for longitudinal data in laparoscopic surgery: A technical note

Causal inference prevails in the field of laparoscopic surgery.Once the causality between an intervention and outcome is established,the intervention can be applied to a target population to improve clinical outcomes.In many clinical scenarios,interventions are applied longitudinally in response to patients’conditions.Such longitudinal data comprise static variables,such as age,gender,and comorbidities;and dynamic variables,such as the treatment regime,laboratory variables,and vital signs.Some dynamic variables can act as both the confounder and mediator for the effect of an intervention on the outcome;in such cases,simple adjustment with a conventional regression model will bias the effect sizes.To address this,numerous statistical methods are being developed for causal inference;these include,but are not limited to,the structural marginal Cox regression model,dynamic treatment regime,and Cox regression model with time-varying covariates.This technical note provides a gentle introduction to such models and illustrates their use with an example in the field of laparoscopic surgery.

Explore of English reading strategies based on mental model pragmatic inference

With the continuous advance education reform and improvement of the quality of English teaching reform, it has become an important content to meet the needs of social development, and promote the comprehensive development of students＇ comprehensive quality and ability. In English teaching, learning English requires a lot of reading, and English reading effects and improve students＇ English scores are closely linked the relationship between the development of future good student. In this paper, the mental model of pragmatic reasoning and English reading give an overview of the psychological impact of the model, they were analyzed using the language to English reading brings reasoning, mental models proposed Pragmatic Inference in English reading, and it can effectively improve English reading efficiency, and promote English Reading level rise.

Causal inference using regression-based statistical control: Confusion in Econometrics

Regression is a widely used econometric tool in research. In observational studies, based on a number of assumptions, regression-based statistical control methods attempt to analyze the causation between treatment and outcome by adding control variables. However, this approach may not produce reliable estimates of causal effects. In addition to the shortcomings of the method, this lack of confidence is mainly related to ambiguous formulations in econometrics, such as the definition of selection bias, selection of core control variables, and method of testing for robustness. Within the framework of the causal models, we clarify the assumption of causal inference using regression-based statistical controls, as described in econometrics, and discuss how to select core control variables to satisfy this assumption and conduct robustness tests for regression estimates.

面向边缘智能的神经网络模型生成与部署研究

随着移动计算、第五代移动通信技术(5G)以及物联网(IoT)技术的不断演进,各类终端设备的数量呈现指数级增长。这种激增的终端设备连接到网络产生了巨大的数据流,对于需要实时处理和快速响应用户任务的需求提出了新的挑战。尤其是在这些海量数据中,半结构化和非结构化数据所占比例较大,这使得神经网络因其独特的优势而得到了广泛应用。为了提高数据处理能力和推理精度,神经网络模型会被设计得非常复杂,其存储和运行均需要消耗大量的计算资源。然而,边缘设备通常只配置有限的计算资源,无法满足存储和运行复杂神经网络模型的需求,需要借助云计算中心来完成这些任务。这种云协同会引发响应延时和增加网络带宽消耗,并带来用户隐私数据泄露等潜在风险。为了解决这些问题,提出一种面向边缘智能的神经网络模型快速生成与自动化部署(NGD)方法,根据边缘设备的硬件配置和承载的具体计算任务需求,生成与其匹配的神经网络模型,并将其快速部署在目标设备上,实现设备本地推理。在3种典型的硬件平台上的神经网络模型生成与部署实验结果表明,NGD方法能够高效地为资源受限的边缘设备生成匹配的神经网络模型,并快速地将其部署在设备上进行推理任务。

基于条件变分推断与内省对抗学习的多样化图像描述生成

现有多样化图像描述生成方法受到隐空间表示能力和评价指标制约,很难同时兼顾描述生成的多样性和准确性.为此,本文提出了一种新的多样化图像描述生成模型,该模型由一个条件变分推断编码器和一个生成器组成.编码器利用全局注意力学习每个单词的隐向量空间,以提升模型对描述多样化的建模能力.生成器根据给定图像和序列隐向量生成多样化的描述语句.同时,引入内省对抗学习的思想,条件变分推断编码器同时作为鉴别器来区分真实描述和生成的描述,赋予模型自我评价生成的描述语句的能力,克服预定义评价指标的局限性.在MSCOCO数据集上的实验表明,与传统方法相比,在随机生成100个描述语句时,多样性指标mBLEU(mutual overlap-BiLingual Evaluation Understudy)提升了1.9%,同时准确性指标CIDEr(Consensus-based Image Description Evaluation)显著提升了7.5%.与典型多模态大模型相比,所提出方法在较小参数量的条件下更适用于生成多样化的陈述性描述语句.

基于银行间同业拆放利率的长记忆随机利率模型研究

研究表明利率序列具有长记忆性,故本文使用分数布朗运动代替经典CIR模型中的几何布朗运动,构建分数CIR模型,并通过欧拉离散对分数CIR过程进行路径模拟。由于分数布朗运动的非马尔可夫性和增量不独立,无法使用极大似然估计和马尔可夫链蒙特卡洛方法对分数CIR模型进行参数估计,故本文引入间接推断估计法,并通过蒙特卡洛模拟证明该方法的可行性。本文使用间接推断估计法对我国银行间同业拆放利率数据进行实证分析及样本外预测,将经典CIR模型、分数O-U过程、分数CIR模型的拟合轨道与真实轨道进行分析对比,得出分数CIR模型更适用于描述具有长记忆性的利率序列。本文重点研究一种用于分数CIR模型的参数估计方法,未来将继续探究其他参数估计方法并对比这些方法的有效性和稳健性。

基于特征蒸馏的变分编码器交通流预测模型

针对交通流数据高维非线性和时空依赖性复杂,本文构建了基于特征蒸馏的变分贝叶斯编码器交通流预测模型.对每段时间序列对应的时间窗口特征,构建了基于多模态时间槽和空间槽的交通流特征提取模型.以时空槽特征提取模型作为特征知识蒸馏架构的输入.通过知识蒸馏结构提取的时空特征结晶体,利用教师模型指导学生模型的学习过程,从而提高学生模型的泛化能力.变分贝叶斯编码器对交通流时空特征结晶编码获取交通流数据的隐变量,根据隐变量的生成采样,利用解码器将其解码重构成新的预测值.实验结果表明,本文提出的模型预测性能显著提升,且中长期预测中鲁棒性更优.

基于局部变分贝叶斯推断的分布式交互式多模型估计

针对目前部分多模型算法预先设定运动模型转移概率矩阵对状态估计精度的不利影响,本文提出了一种基于局部变分贝叶斯推断的分布式交互式多模型估计算法.不同于传统交互式多模型估计中运动模型转移概率矩阵为先验已知的假设条件,在分布融合估计框架下,首先基于最小化Kullback-Leibler散度准则的递归优化策略实现对运动模型转移概率矩阵的预测与更新;在此基础上,结合变分贝叶斯推断实现对当前时刻目标状态与模型概率的联合估计;最后依据协方差交叉融合策略完成对局部状态估计融合.仿真结果表明:新算法通过对运动模型转移概率矩阵以及模型概率自适应在线估计,有效提升了机动目标的状态估计精度.

信息计量学的创新之道:近年优秀国际期刊论文的启示

[目的/意义]在智联网时代,信息计量学的研究环境、研究对象、研究内容和研究方法都发生了深刻变化。揭示近年信息计量学研究的创新之道,有助于了解信息计量学领域的前沿知识,为信息计量学的高质量发展提供参考。[方法/过程]文章系统梳理了2019—2022年的国际信息计量学研究,归纳出三类创新之道。[结果/结论]一是通过引入计算机科学、数学、经济学和社会学等领域的先进知识,实现交叉融合创新;二是辩证思考经典的计量方法、计量指标和计量模型,实现分析方法创新;三是顺应智能预测和因果推断的时代潮流,实现研究范式创新。结论表明,为适应新时代的发展需要,信息计量学研究者必须积极汲取多学科知识,拓展分析方法的适用范围,不断突破面向国家需求的战略问题。

基于力学机理的贝叶斯动态线性岩质边坡变形预测模型研究

考虑岩质边坡变形的力学机理,将边坡变形分解为弹塑性损伤变形、粘弹性变形、周期波动及观测误差等分量,以Bayesian动态线性模型理论为基础,建立了基于力学机理的边坡变形动态演化预测模型。模型以边坡岩体力学参数为系统控制变量,应用Bayesian推断,通过观测数据及相关信息,估计其动态演化规律,并以此预测边坡变形。通过对某水电站边坡位移监测数据分析表明:监测点处边坡变形主要为弹塑性损伤蠕变,监测初期,岩体迅速损伤,变形15天后进入加速蠕变阶段,与现场巡查结果一致。

GNNSched:面向GPU的图神经网络推理任务调度框架

由于频繁的显存访问,图神经网络GNN在GPU上运行时往往资源利用率较低。现有的推理框架由于没有考虑GNN输入的不规则性,直接适用到GNN进行推理任务共置时可能会超出显存容量导致任务失败。对于GNN推理任务,需要根据其输入特点预先分析并发任务的显存占用情况,以确保并发任务在GPU上的成功共置。此外,多租户场景提交的推理任务亟需灵活的调度策略,以满足并发推理任务的服务质量要求。为了解决上述问题,提出了GNNSched,其在GPU上高效管理GNN推理任务的共置运行。具体来说,GNNSched将并发推理任务组织为队列,并在算子粒度上根据成本函数估算每个任务的显存占用情况。GNNSched实现了多种调度策略来生成任务组,这些任务组被迭代地提交到GPU并发执行。实验结果表明,GNNSched能够满足并发GNN推理任务的服务质量并降低推理任务的响应时延。