A Low-Power 12-Bit SAR ADC for Analog Convolutional Kernel of Mixed-Signal CNN Accelerator

As deep learning techniques such as Convolutional Neural Networks(CNNs)are widely adopted,the complexity of CNNs is rapidly increasing due to the growing demand for CNN accelerator system-on-chip(SoC).Although conventional CNN accelerators can reduce the computational time of learning and inference tasks,they tend to occupy large chip areas due to many multiply-and-accumulate(MAC)operators when implemented in complex digital circuits,incurring excessive power consumption.To overcome these drawbacks,this work implements an analog convolutional filter consisting of an analog multiply-and-accumulate arithmetic circuit along with an analog-to-digital converter(ADC).This paper introduces the architecture of an analog convolutional kernel comprised of low-power ultra-small circuits for neural network accelerator chips.ADC is an essential component of the analog convolutional kernel used to convert the analog convolutional result to digital values to be stored in memory.This work presents the implementation of a highly low-power and area-efficient 12-bit Successive Approximation Register(SAR)ADC.Unlink most other SAR-ADCs with differential structure;the proposed ADC employs a single-ended capacitor array to support the preceding single-ended max-pooling circuit along with minimal power consumption.The SARADCimplementation also introduces a unique circuit that reduces kick-back noise to increase performance.It was implemented in a test chip using a 55 nm CMOS process.It demonstrates that the proposed ADC reduces Kick-back noise by 40%and consequently improves the ADC’s resolution by about 10%while providing a near rail-to-rail dynamic rangewith significantly lower power consumption than conventional ADCs.The ADC test chip shows a chip size of 4600μm^(2)with a power consumption of 6.6μW while providing an signal-to-noise-and-distortion ratio(SNDR)of 68.45 dB,corresponding to an effective number of bits(ENOB)of 11.07 bits.

A Kernel-Based Convolution Method to Calculate Sparse Aerial Image Intensity for Lithography Simulation

Optical proximity correction (OPC) systems require an accurate and fast way to predict how patterns will be transferred to the wafer.Based on Gabor's 'reduction to principal waves',a partially coherent imaging system can be represented as a superposition of coherent imaging systems,so an accurate and fast sparse aerial image intensity calculation algorithm for lithography simulation is presented based on convolution kernels,which also include simulating the lateral diffusion and some mask processing effects via Gaussian filter.The simplicity of this model leads to substantial computational and analytical benefits.Efficiency of this method is also shown through simulation results.

A multi-scale convolutional auto-encoder and its application in fault diagnosis of rolling bearings

Aiming at the difficulty of fault identification caused by manual extraction of fault features of rotating machinery,a one-dimensional multi-scale convolutional auto-encoder fault diagnosis model is proposed,based on the standard convolutional auto-encoder.In this model,the parallel convolutional and deconvolutional kernels of different scales are used to extract the features from the input signal and reconstruct the input signal;then the feature map extracted by multi-scale convolutional kernels is used as the input of the classifier;and finally the parameters of the whole model are fine-tuned using labeled data.Experiments on one set of simulation fault data and two sets of rolling bearing fault data are conducted to validate the proposed method.The results show that the model can achieve 99.75%,99.3%and 100%diagnostic accuracy,respectively.In addition,the diagnostic accuracy and reconstruction error of the one-dimensional multi-scale convolutional auto-encoder are compared with traditional machine learning,convolutional neural networks and a traditional convolutional auto-encoder.The final results show that the proposed model has a better recognition effect for rolling bearing fault data.

Fast Image Segmentation Algorithm Based on Salient Features Model and Spatial-frequency Domain Adaptive Kernel

A fast image segmentation algorithm based on salient features model and spatial-frequency domain adaptive kernel is proposed to solve the accurate discriminate objects problem of online visual detection in such scenes of variable sample morphological characteristics,low contrast and complex background texture.Firstly,by analyzing the spectral component distribution and spatial contour feature of the image,a salient feature model is established in spatial-frequency domain.Then,the salient object detection method based on Gaussian band-pass filter and the design criterion of adaptive convolution kernel are proposed to extract the salient contour feature of the target in spatial and frequency domain.Finally,the selection and growth rules of seed points are improved by integrating the gray level and contour features of the target,and the target is segmented by seeded region growing.Experiments have been performed on Berkeley Segmentation Data Set,as well as sample images of online detection,to verify the effectiveness of the algorithm.The experimental results show that the Jaccard Similarity Coefficient of the segmentation is more than 90%,which indicates that the proposed algorithm can availably extract the target feature information,suppress the background texture and resist noise interference.Besides,the Hausdorff Distance of the segmentation is less than 10,which infers that the proposed algorithm obtains a high evaluation on the target contour preservation.The experimental results also show that the proposed algorithm significantly improves the operation efficiency while obtaining comparable segmentation performance over other algorithms.

LKAW: A Robust Watermarking Method Based on Large Kernel Convolution and Adaptive Weight Assignment

Robust watermarking requires finding invariant features under multiple attacks to ensure correct extraction.Deep learning has extremely powerful in extracting features,and watermarking algorithms based on deep learning have attracted widespread attention.Most existing methods use 3×3 small kernel convolution to extract image features and embed the watermarking.However,the effective perception fields for small kernel convolution are extremely confined,so the pixels that each watermarking can affect are restricted,thus limiting the performance of the watermarking.To address these problems,we propose a watermarking network based on large kernel convolution and adaptive weight assignment for loss functions.It uses large-kernel depth-wise convolution to extract features for learning large-scale image information and subsequently projects the watermarking into a highdimensional space by 1×1 convolution to achieve adaptability in the channel dimension.Subsequently,the modification of the embedded watermarking on the cover image is extended to more pixels.Because the magnitude and convergence rates of each loss function are different,an adaptive loss weight assignment strategy is proposed to make theweights participate in the network training together and adjust theweight dynamically.Further,a high-frequency wavelet loss is proposed,by which the watermarking is restricted to only the low-frequency wavelet sub-bands,thereby enhancing the robustness of watermarking against image compression.The experimental results show that the peak signal-to-noise ratio(PSNR)of the encoded image reaches 40.12,the structural similarity(SSIM)reaches 0.9721,and the watermarking has good robustness against various types of noise.

A Lightweight Network with Dual Encoder and Cross Feature Fusion for Cement Pavement Crack Detection

Automatic crack detection of cement pavement chiefly benefits from the rapid development of deep learning,with convolutional neural networks(CNN)playing an important role in this field.However,as the performance of crack detection in cement pavement improves,the depth and width of the network structure are significantly increased,which necessitates more computing power and storage space.This limitation hampers the practical implementation of crack detection models on various platforms,particularly portable devices like small mobile devices.To solve these problems,we propose a dual-encoder-based network architecture that focuses on extracting more comprehensive fracture feature information and combines cross-fusion modules and coordinated attention mechanisms formore efficient feature fusion.Firstly,we use small channel convolution to construct shallow feature extractionmodule(SFEM)to extract low-level feature information of cracks in cement pavement images,in order to obtainmore information about cracks in the shallowfeatures of images.In addition,we construct large kernel atrous convolution(LKAC)to enhance crack information,which incorporates coordination attention mechanism for non-crack information filtering,and large kernel atrous convolution with different cores,using different receptive fields to extract more detailed edge and context information.Finally,the three-stage feature map outputs from the shallow feature extraction module is cross-fused with the two-stage feature map outputs from the large kernel atrous convolution module,and the shallow feature and detailed edge feature are fully fused to obtain the final crack prediction map.We evaluate our method on three public crack datasets:DeepCrack,CFD,and Crack500.Experimental results on theDeepCrack dataset demonstrate the effectiveness of our proposed method compared to state-of-the-art crack detection methods,which achieves Precision(P)87.2%,Recall(R)87.7%,and F-score(F1)87.4%.Thanks to our lightweight crack detectionmodel,the parameter count of the model in real-world detection scenarios has been significantly reduced to less than 2M.This advancement also facilitates technical support for portable scene detection.

A Multi-Category Brain Tumor Classification Method Bases on Improved ResNet50

Brain tumor is one of the most common tumors with high mortality.Early detection is of great significance for the treatment and rehabilitation of patients.The single channel convolution layer and pool layer of traditional convolutional neural network(CNN)structure can only accept limited local context information.And most of the current methods only focus on the classification of benign and malignant brain tumors,multi classification of brain tumors is not common.In response to these shortcomings,considering that convolution kernels of different sizes can extract more comprehensive features,we put forward the multi-size convolutional kernel module.And considering that the combination of average-pooling with max-pooling can realize the complementary of the high-dimensional information extracted by the two structures,we proposed the dual-channel pooling layer.Combining the two structures with ResNet50,we proposed an improved ResNet50 CNN for the applications in multi-category brain tumor classification.We used data enhancement before training to avoid model over fitting and used five-fold cross-validation in experiments.Finally,the experimental results show that the network proposed in this paper can effectively classify healthy brain,meningioma,diffuse astrocytoma,anaplastic oligodendroglioma and glioblastoma.

Super-Resolution Using Enhanced U-Net for Brain MRI Images

Super-resolution is an important technique in image processing. It overcomes some hardware limitations failing to get high-resolution image. After machine learning gets involved, the super-resolution technique gets more efficient in improving the image quality. In this work, we applied super-resolution to the brain MRI images by proposing an enhanced U-Net. Firstly, we used U-Net to realize super-resolution on brain Magnetic Resonance Images (MRI). Secondly, we expanded the functionality of U-Net to the MRI with different contrasts by edge-to-edge training. Finally, we adopted transfer learning and employed convolutional kernel loss function to improve the performance of the U-Net. Experimental results have shown the superiority of the proposed method, e.g., the resolution on rate was boosted from 81.49% by U-Net to 94.22% by our edge-to-edge training.

GA-1DLCNN method and its application in bearing fault diagnosis

Due to the fact that the vibration signal of the rotating machine is one-dimensional and the large-scale convolution kernel can obtain a better perception field, on the basis of the classical convolution neural network model(LetNet-5), one-dimensional large-kernel convolution neural network(1 DLCNN) is designed. Since the hyper-parameters of 1 DLCNN have a greater impact on network performance, the genetic algorithm(GA) is used to optimize the hyper-parameters, and the method of optimizing the parameters of 1 DLCNN by the genetic algorithm is named GA-1 DLCNN. The experimental results show that the optimal network model based on the GA-1 DLCNN method can achieve 99.9% fault diagnosis accuracy, which is much higher than those of other traditional fault diagnosis methods. In addition, the 1 DLCNN is compared with one-dimencional small-kernel convolution neural network(1 DSCNN) and the classical two-dimensional convolution neural network model. The input sample lengths are set to be 128, 256, 512, 1 024, and 2 048, respectively, and the final diagnostic accuracy results and the visual scatter plot show that the effect of 1 DLCNN is optimal.

Gabor-CNN for object detection based on small samples

Object detection models based on convolutional neural networks(CNN)have achieved state-of-the-art performance by heavily rely on large-scale training samples.They are insufficient when used in specific applications,such as the detection of military objects,as in these instances,a large number of samples is hard to obtain.In order to solve this problem,this paper proposes the use of Gabor-CNN for object detection based on a small number of samples.First of all,a feature extraction convolution kernel library composed of multi-shape Gabor and color Gabor is constructed,and the optimal Gabor convolution kernel group is obtained by means of training and screening,which is convolved with the input image to obtain feature information of objects with strong auxiliary function.Then,the k-means clustering algorithm is adopted to construct several different sizes of anchor boxes,which improves the quality of the regional proposals.We call this regional proposal process the Gabor-assisted Region Proposal Network(Gabor-assisted RPN).Finally,the Deeply-Utilized Feature Pyramid Network(DU-FPN)method is proposed to strengthen the feature expression of objects in the image.A bottom-up and a topdown feature pyramid is constructed in ResNet-50 and feature information of objects is deeply utilized through the transverse connection and integration of features at various scales.Experimental results show that the method proposed in this paper achieves better results than the state-of-art contrast models on data sets with small samples in terms of accuracy and recall rate,and thus has a strong application prospect.

Kernel-Based Semantic Relation Detection and Classification via Enriched Parse Tree Structure

This paper proposes a tree kernel method of semantic relation detection and classification （RDC） between named entities. It resolves two critical problems in previous tree kernel methods of RDC. First, a new tree kernel is presented to better capture the inherent structural information in a parse tree by enabling the standard convolution tree kernel with context-sensitiveness and approximate matching of sub-trees. Second, an enriched parse tree structure is proposed to well derive necessary structural information, e.g., proper latent annotations, from a parse tree. Evaluation on the ACE RDC corpora shows that both the new tree kernel and the enriched parse tree structure contribute significantly to RDC and our tree kernel method much outperforms the state-of-the-art ones.

An efficient projection defocus algorithm based on multi-scale convolution kernel templates

The focal problems of projection include out-of-focus projection images from the projector caused by incomplete mechanical focus and screen-door effects produced by projection pixilation. To eliminate these defects and enhance the imaging quality and clarity of projectors, a novel adaptive projection defocus algorithm is proposed based on multi-scale convolution kernel templates. This algorithm applies the improved Sobel-Tenengrad focus evaluation function to calculate the sharpness degree of intensity equalization and then constructs multi-scale defocus convolution kernels to remap and render the defocus projection image. The resulting projection defocus corrected images can eliminate out-of-focus effects and improve the sharpness of uncorrected images. Experiments show that the algorithm works quickly and robustly and that it not only effectively eliminates visual artifacts and can run on a self-designed smart projection system in real time but also significantly improves the resolution and clarity of the observer's visual perception.

A Generalization of Haussler's Convolution Kernel——Mapping Kernel and Its Application to Tree Kernels

Haussler＇s convolution kernel provides an effective framework for engineering positive semidefinite kernels, and has a wide range of applications.On the other hand,the mapping kernel that we introduce in this paper is its natural generalization,and will enlarge the range of application significantly.Our main theorem with respect to positive semidefiniteness of the mapping kernel（1） implies Haussler＇s theorem as a corollary,（2） exhibits an easy-to-check necessary and sufficient condition for mapping kernels to be positive semidefinite,and（3） formalizes the mapping kernel so that significant flexibility is provided in engineering new kernels.As an evidence of the effectiveness of our results,we present a framework to engineer tree kernels.The tree is a data structure widely used in many applications,and tree kernels provide an effective method to analyze tree-type data.Thus,not only is the framework important as an example but also as a practical research tool.The description of the framework accompanies a survey of the tree kernels in the literature,where we see that 18 out of the 19 surveyed tree kernels of different types are instances of the mapping kernel,and examples of novel interesting tree kernels.

Error Estimates for the Time Discretization of a Semilinear Integrodifferential Parabolic Problem with Unknown Memory Kernel

This paper is devoted to the study of an inverse problem containing a semilinear integrodifferential parabolic equation with an unknown memory kernel.This equation is accompanied by a Robin boundary condition.The missing kernel can be recovered from an additional global measurement in integral form.In this contribution,an error analysis is performed for a time-discrete numerical scheme based on Backward Euler’s Method.The theoretical results are supported by some numerical experiments.

Convolution Kernels Implementation of Cardinalized Probability Hypothesis Density Filter

The probability hypothesis density （PHD） propagates the posterior intensity in place of the poste- rior probability density of the multi-target state. The cardinalized PHD （CPHD） recursion is a generalization of PHD recursion, which jointly propagates the posterior intensity function and posterior cardinality distribution. A number of sequential Monte Carlo （SMC） implementations of PHD and CPHD filters （also known as SMC- PHD and SMC-CPHD filters, respectively） for general non-linear non-Gaussian models have been proposed. However, these approaches encounter the limitations when the observation variable is analytically unknown or the observation noise is null or too small. In this paper, we propose a convolution kernel approach in the SMC-CPHD filter. The simuIation results show the performance of the proposed filter on several simulated case studies when compared to the SMC-CPHD filter.

SINGULAR INTEGRAL DIFFERENTIAL EQUATIONS WITH BOTH CAUCHY KERNEL AND CONVOLUTION KERNEL AND RIEMANN BOUNDARY VALUE PROBLEM

In this paper, we set up and discuss a kind of singular integral differential equation with convolution kernel and Canchy kernel. By Fourier transform and some lemmas, we turn this class of equations into Riemann boundary value problems, and obtain the general solution and the condition of solvability in class {0}.

Positive definiteness of real quadratic forms resulting from the variable-step L1-type approximations of convolution operators

The positive definiteness of real quadratic forms with convolution structures plays an important rolein stability analysis for time-stepping schemes for nonlocal operators. In this work, we present a novel analysistool to handle discrete convolution kernels resulting from variable-step approximations for convolution operators.More precisely, for a class of discrete convolution kernels relevant to variable-step L1-type time discretizations, weshow that the associated quadratic form is positive definite under some easy-to-check algebraic conditions. Ourproof is based on an elementary constructing strategy by using the properties of discrete orthogonal convolutionkernels and discrete complementary convolution kernels. To our knowledge, this is the first general result onsimple algebraic conditions for the positive definiteness of variable-step discrete convolution kernels. Using theunified theory, we obtain the stability for some simple nonuniform time-stepping schemes straightforwardly.

Analysis of the second-order BDF scheme with variable steps for the molecular beam epitaxial model without slope selection

In this work,we are concerned with the stability and convergence analysis of the second-order backward difference formula(BDF2)with variable steps for the molecular beam epitaxial model without slope selection.We first show that the variable-step BDF2 scheme is convex and uniquely solvable under a weak time-step constraint.Then we show that it preserves an energy dissipation law if the adjacent time-step ratios satisfy r_(k):=τ_(k)/τ_(k-1)<3.561.Moreover,with a novel discrete orthogonal convolution kernels argument and some new estimates on the corresponding positive definite quadratic forms,the L^(2)norm stability and rigorous error estimates are established,under the same step-ratio constraint that ensures the energy stability,i.e.,0<r_(k)<3.561.This is known to be the best result in the literature.We finally adopt an adaptive time-stepping strategy to accelerate the computations of the steady state solution and confirm our theoretical findings by numerical examples.

DISCRETE ENERGY ANALYSIS OF THE THIRD-ORDER VARIABLE-STEP BDF TIME-STEPPING FOR DIFFUSION EQUATIONS

This is one of our series works on discrete energy analysis of the variable-step BDF schemes.In this part,we present stability and convergence analysis of the third-order BDF(BDF3)schemes with variable steps for linear diffusion equations,see,e.g.,[SIAM J.Numer.Anal.,58:2294-2314]and[Math.Comp.,90:1207-1226]for our previous works on the BDF2 scheme.To this aim,we first build up a discrete gradient structure of the variable-step BDF3 formula under the condition that the adjacent step ratios are less than 1.4877,by which we can establish a discrete energy dissipation law.Mesh-robust stability and convergence analysis in the L^(2) norm are then obtained.Here the mesh robustness means that the solution errors are well controlled by the maximum time-step size but independent of the adjacent time-step ratios.We also present numerical tests to support our theoretical results.

Learning Noun Phrase Anaphoricity in Coreference Resolution via Label Propagation

Knowledge of noun phrase anaphoricity might be profitably exploited in coreference resolution to bypass the resolution of non-anaphoric noun phrases. However, it is surprising to notice that recent attempts to incorporate automatically acquired anaphoricity information into coreferenee resolution systems have been far from expectation. This paper proposes a global learning method in determining the anaphoricity of noun phrases via a label propagation algorithm to improve learning-based coreference resolution. In order to eliminate the huge computational burden in the label propagation algorithm, we employ the weighted support vectors as the critical instances in the training texts. In addition, two kinds of kernels, i.e instances to represent all the anaphoricity-labeled NP , the feature-based RBF （Radial Basis Function） kernel and the convolution tree kernel with approximate matching, are explored to compute the anaphoricity similarity between two noun phrases. Experiments on the ACE2003 corpus demonstrate the great effectiveness of our method in anaphoricity determination of noun phrases and its application in learning-based coreference resolution.