A Novel Quantization and Model Compression Approach for Hardware Accelerators in Edge Computing

Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.

Effective Model Compression via Stage-wise Pruning

Automated machine learning(AutoML)pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks(deep CNNs).However,some previous work found that the results of many Auto-ML pruning methods cannot even surpass the results of the uniformly pruning method.In this paper,the ineffectiveness of Auto-ML pruning,which is caused by unfull and unfair training of the supernet,is shown.A deep supernet suffers from unfull training because it contains too many candidates.To overcome the unfull training,a stage-wise pruning(SWP)method is proposed,which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage training.Besides,a wide supernet is hit by unfair training since the sampling probability of each channel is unequal.Therefore,the fullnet and the tinynet are sampled in each training iteration to ensure that each channel can be overtrained.Remarkably,the proxy performance of the subnets trained with SWP is closer to the actual performance than that of most of the previous AutoML pruning work.Furthermore,experiments show that SWP achieves the state-of-the-art in both CIFAR-10 and ImageNet under the mobile setting.

A pruning-then-quantization model compression framework for facial emotion recognition

Facial emotion recognition achieves great success with the help of large neural models but also fails to be applied in practical situations due to the large model size of neural methods.To bridge this gap,in this paper,we combine two mainstream model compression methods(pruning and quantization)together,and propose a pruningthen-quantization framework to compress the neural models for facial emotion recognition tasks.Experiments on three datasets show that our model could achieve a high model compression ratio and maintain the model’s high performance well.Besides,We analyze the layer-wise compression performance of our proposed framework to explore its effect and adaptability in fine-grained modules.

DPAL-BERT:A Faster and Lighter Question Answering Model

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.

Failure behavior and strength model of blocky rock mass with and without rockbolts

To better understand the failure behaviours and strength of bolt-reinforced blocky rocks,large scale extensive laboratory experiments are carried out on blocky rock-like specimens with and without rockbolt reinforcement.The results show that both shear failure and tensile failure along joint surfaces are observed but the shear failure is a main controlling factor for the peak strength of the rock mass with and without rockbolts.The rockbolts are necked and shear deformation simultaneously happens in bolt reinforced rock specimens.As the joint dip angle increases,the joint shear failure becomes more dominant.The number of rockbolts has a significant impact on the peak strain and uniaxial compressive strength(UCS),but little influence on the deformation modulus of the rock mass.Using the Winkler beam model to represent the rockbolt behaviours,an analytical model for the prediction of the strength of boltreinforced blocky rocks is proposed.Good agreement between the UCS values predicted by proposed model and obtained from experiments suggest an encouraging performance of the proposed model.In addition,the performance of the proposed model is further assessed using published results in the literature,indicating the proposed model can be used effectively in the prediction of UCS of bolt-reinforced blocky rocks.

Constitutive Model for Rubberlike Materials

The isotropic continuum stored energy (CSE) functional, fully combined with the Poisson function, has been applied to constitutively model compressible as well as incompressible rubberlike materials. The isotropic CSE constitutive model fits uniaxial tension test and predicts unfitted pure shear and equibiaxial tension tests of incompressible S4035A thermoplastic elastomer (TPE). Furthermore, the isotropic CSE model along with the Poisson function fits uniaxial tension test along with the kinematic relation test and predicts unfitted pure shear and equibiaxial tension tests along with the corresponding kinematic relation tests of a compressible synthetic rubber. The detailed procedures for uniquely identifying constitutive parameters are emphasized, the experimental characterization results are discussed, and the principles for constitutive models are summarized and augmented.

Effect of warm acupuncture on nitric oxide synthase and calcitonin gene-related peptide in a rat model of lumbar nerve root compression

BACKGROUND： Varying degrees of inflammatory responses occur during lumbar nerve root compression. Studies have shown that nitric oxide synthase （NOS） and calcitonin gene-related peptide （CGRP） are involved in secondary disc inflammation. OBJECTIVE： To observe the effects of warm acupuncture on the ultrastructure of inflammatory mediators in a rat model of lumbar nerve root compression, including NOS and CGRP contents. DESIGN, TIME AND SETTING： Randomized, controlled study, with molecular biological analysis, was performed at the Experimental Center, Sixth People＇s Hospital Affiliated to Shanghai Jiao Tong University, between September 2006 and April 2007. MATERIALS： Acupuncture needles and refined Moxa grains were purchased from Shanghai Taicheng Technology Development Co., Ltd., China; Mobic tablets were purchased from Shanghai Boehringer Ingelheim Pharmaceuticals Co., Ltd., China; enzyme linked immunosorbent assay （ELISA） kits for NOS and CGRP were purchased from ADL Biotechnology, Inc., USA. METHODS： A total of 50, healthy, adult Sprague-Dawley rats, were randomly divided into five groups normal, model, warm acupuncture, acupuncture, and drug, with 10 rats in each group. Rats in the four groups, excluding the normal group, were used to establish models of lumbar nerve root compression. After 3 days, Jiaji points were set using reinforcing-reducing manipulation in the warm acupuncture group. Moxa grains were burned on each needle, with 2 grains each daily. The acupuncture group was the same as the warm acupuncture group, with the exception of non-moxibustion. Mobic suspension （3.75 mg/kg） was used in the oral drug group, once a day. Treatment of each group lasted for 14 consecutive days. Modeling and medication were not performed in the normal group. MAIN OUTCOME MEASURES： The ultrastructure of damaged nerve roots was observed with transmission electron microscopy; NOS and CGRP contents were measured using ELISA. RESULTS： The changes of the radicular ultramicrostructure were characterized by Wallerian degeneration; nerve fibers were clearly demyelinated; axons collapsed or degenerated; outer Schwann cell cytoplasm was swollen and its nucleus was compacted. Compared with the normal group, NOS and CGRP contents in the nerve root compression zone in the model group were significantly increased （P 〈 0.01）. Nerve root edema was improved in the drug, acupuncture and the warm acupuncture groups over the model group. NOS and CGRP expressions were also decreased with the warm acupuncture group having the lowest concentration （P 〈 0.01）. CONCLUSION： In comparison to the known effects of Mobic drug and acupuncture treatments, the warm acupuncture significantly decreased NOS and CGRP expression which helped improve the ultrastructure of the compressed nerve root.

Deep Model Compression for Mobile Platforms:A Survey

Despite the rapid development of mobile and embedded hardware, directly executing computationexpensive and storage-intensive deep learning algorithms on these devices’ local side remains constrained for sensory data analysis. In this paper, we first summarize the layer compression techniques for the state-of-theart deep learning model from three categories: weight factorization and pruning, convolution decomposition, and special layer architecture designing. For each category of layer compression techniques, we quantify their storage and computation tunable by layer compression techniques and discuss their practical challenges and possible improvements. Then, we implement Android projects using TensorFlow Mobile to test these 10 compression methods and compare their practical performances in terms of accuracy, parameter size, intermediate feature size,computation, processing latency, and energy consumption. To further discuss their advantages and bottlenecks,we test their performance over four standard recognition tasks on six resource-constrained Android smartphones.Finally, we survey two types of run-time Neural Network(NN) compression techniques which are orthogonal with the layer compression techniques, run-time resource management and cost optimization with special NN architecture,which are orthogonal with the layer compression techniques.

A Novel Deep Neural Network Compression Model for Airport Object Detection

A novel deep neural network compression model for airport object detection has been presented.This novel model aims at disadvantages of deep neural network,i.e.the complexity of the model and the great cost of calculation.According to the requirement of airport object detection,the model obtains temporal and spatial semantic rules from the uncompressed model.These spatial semantic rules are added to the model after parameter compression to assist the detection.The rules can improve the accuracy of the detection model in order to make up for the loss caused by parameter compression.The experiments show that the effect of the novel compression detection model is no worse than that of the uncompressed original model.Even some of the original model false detection can be eliminated through the prior knowledge.

BLOWUP CRITERION FOR THE COMPRESSIBLE FLUID-PARTICLE INTERACTION MODEL IN 3D WITH VACUUM

In this article, we consider the blowup criterion for the local strong solution to the compressible fluid-particle interaction model in dimension three with vacuum. We establish a BKM type criterion for possible breakdown of such solutions at critical time in terms of both the L^∞ （0, T; L^6）-norm of the density of particles and the ^L1（0, T; L^∞）-norm of the deformation tensor of velocity gradient.

Acupuncture at the “Huatuojiaji” point affects nerve root regional interleukin-1 level in a rat model of lumbar nerve root compression

BACKGROUND：It has been shown that interleukin-1 （IL-1） may cause inflammatory reactions, which stimulate the nerve root of patients with lumbar intervertebral disc protrusion and leads to pain. Whether the clinical curative effects of acupuncture in the treatment of lumbar and leg pain are linked to an inhibition of local IL-1 secretion is unknown. OBJECTIVE： To assess the influence of acupuncture on IL-1, this study was designed to verify the effects of acupuncture at the ＂Huatuojiaji （Extra）＂ point on the nerve root in a rat model of lumbar nerve root compression, compared with administration of meloxicam, a non-steroidal anti-inflammatory drug. DESIGN, TIME AND SETTING： Randomized, controlled, molecular biology experiment, performed at the Experimental Center, Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University between September 2005 and April 2006. MATERIALS： Forty healthy adult Sprague Dawley rats of either gender were included in this study. The rats were randomly and evenly divided into the following four groups： normal control, model, acupuncture and meloxicam groups. Lumbar nerve root compression was induced in rats in the model, acupuncture, and meloxicam groups by inserting a specially made silicon rubber slice at the juncture of the L5 nerve root and the dural sac. The acupuncture needle （pattern number N3030, 30#, 1.5 inch） was purchased from Suzhou Medical Appliance Factory, China. IL-1 enzyme linked immunosorbent assay （ELISA） kit was purchased from Santa Cruz Biotechnology, Inc., USA. METHODS： The acupuncture group was acupunctured at the ＂Huatuojiaji＂ point, which is lateral to the compressed L5-6 nerve root, with an acupuncture depth of 0.5 cm. There were two treatment courses, each of involved seven 20-minute acupuncture sessions, one session a day. The meloxicam group was administered intragastrically 3.75 mg/kg meloxicam （5 mg meloxicam /10 mL physiological saline）. Rats in the normal control group and model group received an intragastric administration of 10 mL/kg physiological saline. All administrations were performed once a day. MAIN OUTCOME MEASURES： At day 14 post-surgery, the IL-1 level in the compressed nerve root was determined by a streptavidin-peroxidase （S-P） immunohistochemical method, and IL-1β mRNA expression in the compressed nerve root was simultaneously detected by real-time reverse transcription-polymerase chain reaction. RESULTS： The expression levels of IL-1 and IL-1β mRNA in the L5 nerve root were significantly higher in the model group than in the control group （P 〈 0.01）. However, the expression levels of IL-1 and IL-1β mRNA were significantly lower in the acupuncture and meloxicam groups than in the model group （P 〈 0.05–0.01）. Expression levels of IL-1 and IL-1β mRNA were significantly higher in the acupuncture group than in the meloxicam group （P 〈 0.01）. CONCLUSION： Acupuncture at the ＂Huatuojiaji＂ point decreases the IL-1 level by inhibiting IL-1β mRNA expression to a greater extent than meloxicam administration.

An analytical p-y curve method based on compressive soil pressure model in sand soil

With the high-quality development of urban buildings,higher requirements are come up with for lateral bearing capacity of laterally loaded piles.Consequently,a more accurate analysis to predict the lateral response of the pile within an allowable displacement is an important issue.However,the current p-y curve methods cannot fully take into account the pile-soil interaction,which will lead to a large calculation difference.In this paper,a new analytical p-y curve is established and a finite difference method for determining the lateral response of pile is proposed,which can consider the separation effect of pile-soil interface and the coefficient of circumferential friction resistance.In particular,an analytical expression is developed to determine the compressive soil pressure by dividing the compressive soil pressure into two parts:initial compressive soil pressure and increment of compressive soil pressure.In addition,the relationship between compressive soil pressure and horizontal displacement of the pile is established based on the reasonable assumption.The correctness of the proposed method is verified through four examples.Based on the verified method,a parametric analysis is also conducted to investigate the influences of factors on lateral response of the pile,including internal friction angle,pile length and elastic modulus of pile.

Statistical damage constitutive model for concrete materials under uniaxial compression

According to the damage mechanism of concrete material during the uniaxial compressive failure process,this paper further establishes the statistical damage constitutive model of concrete subjected to uniaxial compressive stress based on the statistical damage model under uniaxial tension. The damage evolution law in the direction subjected to pressure is confirmed by the tensile damage evolution process of lateral deformation due to the Poisson effect,and then the compressive stress-strain relationship is defined. The peak nominal stress state and the critical state occurring in the macro longitudinal distributed splitting cracks are distinguished. The whole loading process can be divided into the even damage phase and the local breakage phase. The concrete specimen is divided into the failure process zone and the resting unloading zone. The size effects during the local breakage phase under the uniaxial monotonic compressive process and the hysteretic phenomenon under the cyclic compressive loading process are analyzed. Finally,the comparison between theoretical results and experimental results preliminarily verifies the rationality and feasibility of understanding the failure mechanism of concrete through the statistical damage constitutional law.

A composite material model for investigation of micro-fracture mechanism of brittle rock subjected to uniaxial compression

A two phase model of rock was proposed in order to investigate the mechanism of brittle fracture due to uniaxial compression, in which rock was considered to be a composite material consisting of hard grains and colloids. The stress state in colloid region near grains was calculated using Finite Element Method (FEM). The influence of the tensile stresses on the crack initiation and failure process of brittle rock subjected to uniaxial compression was investigated by numerical experiments. The FE results show that tensile stresses are induced easily in the neighboring area of hard grains with the maximum value near grain boundaries. The distribution of tensile stresses depends on the relative position of hard grains. The cracks initiated just near the boundary area of hard grains, which was governed by tensile stress. These results clearly reveal the micro fracture mechanism of brittle rock loaded by uniaxial compression. It can be concluded that the failure mode of brittle rock under uniaxial compression is still tensile fracture from the point view of microstructure. However, since the wide colloid region is still under compressive stress state, further propagation of boundary cracks through this region obviously needs more external load, thus causing the uniaxial compressive strength of rock much higher than its tensile strength obtained via Brazilian (splitting)

Numerical model of compressible gas flow in soil pollution control

Based on the theory of fluid dynamics in porous media, a numerical model of gas flow in unsaturated zone is developed with the consideration of gas density change due to variation of air pressure. This model is characterized of its wider range of availability. The accuracy of this numerical model is analyzed through comparison with modeling results by previous model with presumption of little pressure variation and the validity of this numerical model is shown. Thus it provides basis for the designing and management of landfill gas control system or soil vapor extraction system in soil pollution control.

Novel magnetic compression technique for establishment of a canine model of tracheoesophageal fistula

BACKGROUND Clinically,tracheoesophageal fistula(TEF)is lack of effective surgical strategies.One reason is due to the lack of appropriate animal models of acquired TEF,which is usually complex and difficult.Recently,the magnetic compression technique has been applied for digestive tract anastomosis or vascular anastomosis in animals.In this study,an animal model of TEF in dogs was developed by using the magnetic compression technique,hoping to provide a new method for mimicking TEF.AIM To establish a TEF model in dogs by using the magnetic compression technique.METHODS Six male beagles were used as models with two Nd-Fe-B permanent magnets for TEF.The parent magnet and the daughter magnet were placed in the cervical esophagus and trachea,respectively.The anterior wall of the esophagus and the posterior wall of the trachea were compressed when the two magnets coupled.After 4-6 d,the necrotic tissue between the two magnets fell off and the parent and daughter magnets disengaged from the target location,leaving a fistula.Gastroscopy/bronchoscopy,upper gastrointestinal contrast study,and histological analysis were performed.RESULTS The establishment of the TEF model in all six beagles was successful.The average time of magnet placement was 4.33±1.11 min(range,3-7 min).Mean time for the magnets to disengage from the target location was 4.67±0.75 d(range,4-6 d).TEFs were observed by gastroscopy/bronchoscopy and esophageal angiography.The gross anatomical structure of the esophagus and the trachea was in good condition.There was no esophageal mucosa or pseudostratified ciliated columnar epithelium at the site of the fistula according to histological analysis.CONCLUSION It is simple,feasible,and minimally invasive to use the magnetic compression technique for the establishment of the TEF model in dogs.

聚氨酯泡沫注浆修复材料泡孔结构特征及抗压性能研究进展

聚氨酯泡沫(PUF)注浆修复材料以轻质高强、快凝高膨胀等优势,在基础设施修复加固领域得到了广泛的应用。抗压性能是工程修复材料十分重要的力学指标之一,而材料的微观结构对其宏观力学行为有决定性影响。近年来,国内外专家学者对PUF注浆修复材料的微观结构特性和抗压性能进行了大量研究,但尚未见到系统性总结。本文介绍了PUF注浆修复材料的定义及反应过程,综述了其微观泡孔结构特性、压缩应力-应变关系和本构模型,从宏微观角度讨论了其变形和破坏机理,总结了密度、试件尺寸、应变率和特殊环境工况对压缩性能的影响机理及规律,最后展望了PUF注浆修复材料未来的研究方向。

Effectively Lossless Subspace Appearance Model Compression Using Prior Information

Subspace appearance models are widely used in computer vision and image processing tasks to compactly represent the appearance variations of target objects. In order to ensure algorithm performance, they are typically stored in high-precision formats; this results in a large storage footprint, rendering redistribution costly and difficult. Since for most image and vision applications, pixel values are quantized to 8 bits by the acquisition apparatuses, we show that it is possible to construct a fixed-width, effectively Iossless representation of the bases vectors, in the sense that reconstructions from the original bases and from the quantized bases never deviate by more than half of the quantization step-size. In addition to directly applying this result to Iosslessly compress individual models, we also propose an algorithm to compress appearance models by utilizing prior information on the modeled objects in the form of prior appearance subspaces. Experiments conducted on the compression of person-specific face appearance models demonstrate the effectiveness of the proposed algorithms.

Mathematical model and general laws of wet compression

Wet compression is an effective way to enhance the performance of gas turbines and it has attracted a good deal of attention in recent years. The one sidedness and inaccuracy of previous studies,which took concentration gradient as mass transfer potential are discussed. The mass transfer process is analyzed from the viewpoint of non equilibrium thermodynamics,by taking generalized thermodynamic driving force as mass transfer potential,and the corresponding mass transfer coefficient is obtained using the heat and mass transfer equilibrium occurring between moist air and water droplets at wet bulb temperature for the sake of avoiding complex tests and providing more accurate formulas. A mathematical model of wet compression is therefore established,and the general laws of wet compression are investigated. The results show that the performance of atomizer is critical for wet compression and wet compression is more suitable for compressors with higher pressure ratio and longer compression time.

Experimental Study on the Repair Effect of Xianlinggubao Capsule on Osteoporotic Vertebral Compression Fracture in Rabbits

[Objectives]To observe the effect of Xianlinggubao Capsule on osteoporotic vertebral compression fracture(OVCF)in rabbits and the influence mechanism of the repair of fractures.[Methods]Female June age 30 rabbits were randomly divided into control group,model control group and Xianlinggubao group.After bilateral ovariectomy,the model control group and Xianlinggubao group were injected with dexamethasone continuously for 4 weeks,and then the OVCF compound model was established by surgery.The Xianlinggubao group was treated with Xianlinggubao at a dose of 300 mg/(kg·d)for 60 d,while the blank control group and the model control group were treated with the same amount of normal saline for 60 d.The number of blood vessels and the expression of bone morphogenetic protein-2(BMP-2)were detected by immunohistochemical staining and the bone mineral density(BMD)in the callus of the third lumbar fracture area of rabbits was measured.The content of serum phosphorus(P),alkaline phosphatase(ALP)and total calcium(TCa)in rabbit venous blood were measured by automatic biochemical analyzer.The content of vascular endothelial growth factor(VEGF)and platelet-derived growth factor(PDGF)in rabbit venous blood were measured by ELISA kit.[Results]The number of blood vessels and the expression of BMP-2 in the callus of the third lumbar fracture area of rabbits was high in Xianlinggubao group,the content of serum P,ALP,TCa,VEGF and PDGF was obviously increased,BMD was obviously increased,the bone microstructure of the third lumbar vertebrae fracture area of rabbits was basically restored.Compared with the model control group(P<0.05),the difference was statistically significant.[Conclusions]Xianlinggubao Capsule can increase calcium and phosphorus deposition,promote the formation of blood vessels in the fracture area of OVCF in rabbits,and have a strong repair effect on OVCF in rabbits.