Design of Polynomial Fuzzy Neural Network Classifiers Based on Density Fuzzy C-Means and L2-Norm Regularization

In this paper, polynomial fuzzy neural network classifiers (PFNNCs) is proposed by means of density fuzzy c-means and L2-norm regularization. The overall design of PFNNCs was realized by means of fuzzy rules that come in form of three parts, namely premise part, consequence part and aggregation part. The premise part was developed by density fuzzy c-means that helps determine the apex parameters of membership functions, while the consequence part was realized by means of two types of polynomials including linear and quadratic. L2-norm regularization that can alleviate the overfitting problem was exploited to estimate the parameters of polynomials, which constructed the aggregation part. Experimental results of several data sets demonstrate that the proposed classifiers show higher classification accuracy in comparison with some other classifiers reported in the literature.

L1/2 Regularization Based on Bayesian Empirical Likelihood

Bayesian empirical likelihood is a semiparametric method that combines parametric priors and nonparametric likelihoods, that is, replacing the parametric likelihood function in Bayes theorem with a nonparametric empirical likelihood function, which can be used without assuming the distribution of the data. It can effectively avoid the problems caused by the wrong setting of the model. In the variable selection based on Bayesian empirical likelihood, the penalty term is introduced into the model in the form of parameter prior. In this paper, we propose a novel variable selection method, L1/2 regularization based on Bayesian empirical likelihood. The L1/2 penalty is introduced into the model through a scale mixture of uniform representation of generalized Gaussian prior, and the posterior distribution is then sampled using MCMC method. Simulations demonstrate that the proposed method can have better predictive ability when the error violates the zero-mean normality assumption of the standard parameter model, and can perform variable selection.

基于L1/2正则化理论的地震稀疏反褶积

地震反褶积是一种重要的压缩地震子波、提高薄层纵向分辨率的地震数据处理方法。在层状地层的假设下,反射系数可视作稀疏的脉冲序列,所以地震反褶积可以描述为一个稀疏求解问题,L 1正则化被广泛用于解决稀疏问题,但近年来一些文献证明L 1正则化的稀疏表达能力不是最优的。针对这一问题,基于快速发展的L 1/2正则化理论,提出将L 1/2正则化作为反射系数的稀疏约束进行地震反褶积处理,并使用其特定的阈值迭代算法进行求解,对单道模型的测试证实了该方法对正则化参数和噪声有较好的适应能力。简单二维模型和Marmousi2模型数据的测试结果表明,基于该方法的反演结果能较好地拟合反射系数振幅,并且对噪声干扰的鲁棒性更强,能够更好地保护弱反射系数。实际数据应用结果表明,该方法能有效消除子波影响,较好地分辨出薄层结构和透镜体结构,为地震数据高分辨处理提供了有力工具。

包含式完全正则（T31/2^＊）是相对可乘的

本文讨论包含式完全正则空间（T31/2^＊）关于相对积拓扑的可乘性问题．

Robust Latent Factor Analysis for Precise Representation of High-Dimensional and Sparse Data

High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurately represent them is of great significance.A latent factor(LF)model is one of the most popular and successful ways to address this issue.Current LF models mostly adopt L2-norm-oriented Loss to represent an HiDS matrix,i.e.,they sum the errors between observed data and predicted ones with L2-norm.Yet L2-norm is sensitive to outlier data.Unfortunately,outlier data usually exist in such matrices.For example,an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users.To address this issue,this work proposes a smooth L1-norm-oriented latent factor(SL-LF)model.Its main idea is to adopt smooth L1-norm rather than L2-norm to form its Loss,making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix.Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.

Single color image super-resolution using sparse representation and color constraint

Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent and are easy to cause the problem of blurring details and color artifacts in color reconstructed images.This paper presents a color super-resolution reconstruction method combining the L2/3 sparse regularization model with color channel constraints.The method converts the low-resolution color image from RGB to YCbCr.The L2/3 sparse regularization model is designed to reconstruct the brightness channel of the input low-resolution color image.Then the color channel-constraint method is adopted to remove artifacts of the reconstructed highresolution image.The method not only ensures the reconstruction quality of the color image details,but also improves the removal ability of color artifacts.The experimental results on natural images validate that our method has improved both subjective and objective evaluation.

An Optimized Deep Residual Network with a Depth Concatenated Block for Handwritten Characters Classification

Even though much advancements have been achieved with regards to the recognition of handwritten characters,researchers still face difficulties with the handwritten character recognition problem,especially with the advent of new datasets like the Extended Modified National Institute of Standards and Technology dataset(EMNIST).The EMNIST dataset represents a challenge for both machine-learning and deep-learning techniques due to inter-class similarity and intra-class variability.Inter-class similarity exists because of the similarity between the shapes of certain characters in the dataset.The presence of intra-class variability is mainly due to different shapes written by different writers for the same character.In this research,we have optimized a deep residual network to achieve higher accuracy vs.the published state-of-the-art results.This approach is mainly based on the prebuilt deep residual network model ResNet18,whose architecture has been enhanced by using the optimal number of residual blocks and the optimal size of the receptive field of the first convolutional filter,the replacement of the first max-pooling filter by an average pooling filter,and the addition of a drop-out layer before the fully connected layer.A distinctive modification has been introduced by replacing the final addition layer with a depth concatenation layer,which resulted in a novel deep architecture having higher accuracy vs.the pure residual architecture.Moreover,the dataset images’sizes have been adjusted to optimize their visibility in the network.Finally,by tuning the training hyperparameters and using rotation and shear augmentations,the proposed model outperformed the state-of-the-art models by achieving average accuracies of 95.91%and 90.90%for the Letters and Balanced dataset sections,respectively.Furthermore,the average accuracies were improved to 95.9%and 91.06%for the Letters and Balanced sections,respectively,by using a group of 5 instances of the trained models and averaging the output class probabilities.

Generating Cartoon Images from Face Photos with Cycle-Consistent Adversarial Networks

The generative adversarial network(GAN)is first proposed in 2014,and this kind of network model is machine learning systems that can learn to measure a given distribution of data,one of the most important applications is style transfer.Style transfer is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image.CYCLE-GAN is a classic GAN model,which has a wide range of scenarios in style transfer.Considering its unsupervised learning characteristics,the mapping is easy to be learned between an input image and an output image.However,it is difficult for CYCLE-GAN to converge and generate high-quality images.In order to solve this problem,spectral normalization is introduced into each convolutional kernel of the discriminator.Every convolutional kernel reaches Lipschitz stability constraint with adding spectral normalization and the value of the convolutional kernel is limited to[0,1],which promotes the training process of the proposed model.Besides,we use pretrained model(VGG16)to control the loss of image content in the position of l1 regularization.To avoid overfitting,l1 regularization term and l2 regularization term are both used in the object loss function.In terms of Frechet Inception Distance(FID)score evaluation,our proposed model achieves outstanding performance and preserves more discriminative features.Experimental results show that the proposed model converges faster and achieves better FID scores than the state of the art.

I(L)型诱导空间的性质

本文讨论了Fuzzy拓扑空间的I（L）型诱导空间的闭包和内部运算,并讨论了它的可分性、C_Ⅰ、C_Ⅱ和分离性。

A Sharp Nonasymptotic Bound and Phase Diagram of L1/2 Regularization

We derive a sharp nonasymptotic bound of parameter estimation of the L1/2 regularization. The bound shows that the solutions of the L1/2 regularization can achieve a loss within logarithmic factor of an ideal mean squared error and therefore underlies the feasibility and effectiveness of the L1/2 regularization. Interestingly, when applied to compressive sensing, the L1/2 regularization scheme has exhibited a very promising capability of completed recovery from a much less sampling information. As compared with the Lp （0 〈 p 〈 1） penalty, it is appeared that the L1/2 penalty can always yield the most sparse solution among all the Lv penalty when 1/2 〈 p 〈 1, and when 0 〈 p 〈 1/2, the Lp penalty exhibits the similar properties as the L1/2 penalty. This suggests that the L1/2 regularization scheme can be accepted as the best and therefore the representative of all the Lp （0 〈 p 〈 1） regularization schemes.

THE L^2-NORM ERROR ESTIMATE OF NONCONFORMING FINITE ELEMENT METHOD FOR THE 2ND ORDER ELLIPTIC PROBLEM WITH THE LOWEST REGULARITY

Presents the abstract L...-norm error estimate of nonconforming finite element method. Use of the Aubin Nitsche Lemma in estimating nonconforming finite element methods; Details on the equations.

LMI Approach to Observer-based FD Systems Designing

Increasing the robustness to the unknown uncertainty and simultaneously enhancing the sensibility to the faults is one of the important issues considered in the fault detection development. Considering the L2-gain of residual system, this paper deals the observer-based fault detection problem. By using of H∞ control theory,an LMI approach to design fault detection observer is given. A numerical example is used to illustrate the effectiveness of the proposed approach.

Towards Securing Machine Learning Models Against Membership Inference Attacks

From fraud detection to speech recognition,including price prediction,Machine Learning(ML)applications are manifold and can significantly improve different areas.Nevertheless,machine learning models are vulnerable and are exposed to different security and privacy attacks.Hence,these issues should be addressed while using ML models to preserve the security and privacy of the data used.There is a need to secure ML models,especially in the training phase to preserve the privacy of the training datasets and to minimise the information leakage.In this paper,we present an overview of ML threats and vulnerabilities,and we highlight current progress in the research works proposing defence techniques againstML security and privacy attacks.The relevant background for the different attacks occurring in both the training and testing/inferring phases is introduced before presenting a detailed overview of Membership Inference Attacks(MIA)and the related countermeasures.In this paper,we introduce a countermeasure against membership inference attacks(MIA)on Conventional Neural Networks(CNN)based on dropout and L2 regularization.Through experimental analysis,we demonstrate that this defence technique can mitigate the risks of MIA attacks while ensuring an acceptable accuracy of the model.Indeed,using CNN model training on two datasets CIFAR-10 and CIFAR-100,we empirically verify the ability of our defence strategy to decrease the impact of MIA on our model and we compare results of five different classifiers.Moreover,we present a solution to achieve a trade-off between the performance of themodel and the mitigation of MIA attack.

CONVERGENCE ANALYSIS OF MIXED VOLUME ELEMENT-CHARACTERISTIC MIXED VOLUME ELEMENT FOR THREE-DIMENSIONAL CHEMICAL OIL-RECOVERY SEEPAGE COUPLED PROBLEM

The physical model is described by a seepage coupled system for simulating numerically three-dimensional chemical oil recovery, whose mathematical description includes three equations to interpret main concepts. The pressure equation is a nonlinear parabolic equation, the concentration is defined by a convection-diffusion equation and the saturations of different components are stated by nonlinear convection-diffusion equations. The transport pressure appears in the concentration equation and saturation equations in the form of Darcy velocity, and controls their processes. The flow equation is solved by the conservative mixed volume element and the accuracy is improved one order for approximating Darcy velocity. The method of characteristic mixed volume element is applied to solve the concentration, where the diffusion is discretized by a mixed volume element method and the convection is treated by the method of characteristics. The characteristics can confirm strong computational stability at sharp fronts and it can avoid numerical dispersion and nonphysical oscillation. The scheme can adopt a large step while its numerical results have small time-truncation error and high order of accuracy. The mixed volume element method has the law of conservation on every element for the diffusion and it can obtain numerical solutions of the concentration and adjoint vectors. It is most important in numerical simulation to ensure the physical conservative nature. The saturation different components are obtained by the method of characteristic fractional step difference. The computational work is shortened greatly by decomposing a three-dimensional problem into three successive one-dimensional problems and it is completed easily by using the algorithm of speedup. Using the theory and technique of a priori estimates of differential equations, we derive an optimal second order estimates in 12 norm. Numerical examples are given to show the effectiveness and practicability and the method is testified as a powerful tool to solve the important problems.

不平衡转子系统弯扭耦合复杂故障智能诊断

弯曲振动与扭转振动耦合在旋转机械实际运行中往往不可避免。考虑不平衡转子不同复杂工况的弯扭耦合情况,利用深度学习技术的优势,构建了基于一维卷积神经网络的诊断模型,提出了一种用于处理不平衡转子发生弯曲,扭转以及弯扭耦合振动情况的智能故障诊断方法。分析了数据输入类型和L 2正则化对诊断的影响,优化了诊断模型以提高诊断精度,并进行了试验验证。研究结果表明,该方法可以实现不同转速下,发生弯扭耦合振动时单种或多种复合故障的智能诊断,获得比其他方法更好的诊断效果。

A pruning algorithm with L_(1/2) regularizer for extreme learning machine

Compared with traditional learning methods such as the back propagation(BP)method,extreme learning machine provides much faster learning speed and needs less human intervention,and thus has been widely used.In this paper we combine the L1/2regularization method with extreme learning machine to prune extreme learning machine.A variable learning coefcient is employed to prevent too large a learning increment.A numerical experiment demonstrates that a network pruned by L1/2regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L2regularization.

一种基于L_(1/2)正则约束的超分辨率重建算法

为了提高重建图像质量,减少处理时间,提出一种基于L_(1/2)正则约束的单帧图像超分辨率重建算法.该算法在稀疏重建字典对训练阶段,为了有效提取低分辨率图像边缘、纹理等特征细节信息,采用小波系数单支重构方法对低分辨率图像进行特征提取;而在图像重建阶段,为了解决基于L1正则模型得到的解时常不够稀疏,重建图像质量有待进一步提高的问题,采用L_(1/2)范数代替L1范数构建超分辨率重建模型,并且采用一种快速求解的L_(1/2)正则化算法进行稀疏求解.实验结果表明:与现有算法相比较,该算法在重建图像主观和客观评价指标、算法运行速度等方面均更优.

非线性抛物椭圆方程组的正则解和奇异解

该文用单模方法在Lorentz空间研究了抛物椭圆方程组奇异解和正则解的存在性,其中初值属于Lorentz空间,L^(n/2,∞)(R^n),n≥3.利用时间加权的Lorentz空间,还得到了其正则解.此外,如果初值满足自相似结构,也得到了自相似解的存在性.

具有VMO系数的拟线性椭圆方程的L^(2,λ)正则性

得到了一类拟线性一致椭圆型方程的弱解梯度在系数矩阵满足VMO条件下的局部Morrey空间正则性结果．

深度学习中的正则化方法研究

带有百万个参数的神经网络在大量训练集的训练下,很容易产生过拟合现象。一些正则化方法被学者提出以期达到对参数的约束求解。本文总结了深度学习中的L1,L2和Dropout正则化方法。最后基于上述正则化方法,进行了MNIST手写体识别对比数值试验。