Low rank optimization for efficient deep learning:making a balance between compact architecture and fast training

Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.

Low-Rank Multi-View Subspace Clustering Based on Sparse Regularization

Multi-view Subspace Clustering (MVSC) emerges as an advanced clustering method, designed to integrate diverse views to uncover a common subspace, enhancing the accuracy and robustness of clustering results. The significance of low-rank prior in MVSC is emphasized, highlighting its role in capturing the global data structure across views for improved performance. However, it faces challenges with outlier sensitivity due to its reliance on the Frobenius norm for error measurement. Addressing this, our paper proposes a Low-Rank Multi-view Subspace Clustering Based on Sparse Regularization (LMVSC- Sparse) approach. Sparse regularization helps in selecting the most relevant features or views for clustering while ignoring irrelevant or noisy ones. This leads to a more efficient and effective representation of the data, improving the clustering accuracy and robustness, especially in the presence of outliers or noisy data. By incorporating sparse regularization, LMVSC-Sparse can effectively handle outlier sensitivity, which is a common challenge in traditional MVSC methods relying solely on low-rank priors. Then Alternating Direction Method of Multipliers (ADMM) algorithm is employed to solve the proposed optimization problems. Our comprehensive experiments demonstrate the efficiency and effectiveness of LMVSC-Sparse, offering a robust alternative to traditional MVSC methods.

Robust Principal Component Analysis Integrating Sparse and Low-Rank Priors

Principal Component Analysis (PCA) is a widely used technique for data analysis and dimensionality reduction, but its sensitivity to feature scale and outliers limits its applicability. Robust Principal Component Analysis (RPCA) addresses these limitations by decomposing data into a low-rank matrix capturing the underlying structure and a sparse matrix identifying outliers, enhancing robustness against noise and outliers. This paper introduces a novel RPCA variant, Robust PCA Integrating Sparse and Low-rank Priors (RPCA-SL). Each prior targets a specific aspect of the data’s underlying structure and their combination allows for a more nuanced and accurate separation of the main data components from outliers and noise. Then RPCA-SL is solved by employing a proximal gradient algorithm for improved anomaly detection and data decomposition. Experimental results on simulation and real data demonstrate significant advancements.

A Perturbation Analysis of Low-Rank Matrix Recovery by Schatten p-Minimization

A number of previous papers have studied the problem of recovering low-rank matrices with noise, further combining the noisy and perturbed cases, we propose a nonconvex Schatten p-norm minimization method to deal with the recovery of fully perturbed low-rank matrices. By utilizing the p-null space property (p-NSP) and the p-restricted isometry property (p-RIP) of the matrix, sufficient conditions to ensure that the stable and accurate reconstruction for low-rank matrix in the case of full perturbation are derived, and two upper bound recovery error estimation ns are given. These estimations are characterized by two vital aspects, one involving the best r-approximation error and the other concerning the overall noise. Specifically, this paper obtains two new error upper bounds based on the fact that p-RIP and p-NSP are able to recover accurately and stably low-rank matrix, and to some extent improve the conditions corresponding to RIP.

基于Low rank近似的黏弹性介质衰减补偿微地震逆时定位与裂缝成像

真实地下介质具有黏弹性,地震波在传播过程中会发生耗散与频散.忽视黏弹性介质的吸收衰减效应,逆时延拓过程中地震波将会出现振幅减弱、相位失真等现象,无法准确定位震源真实位置,因此需要对黏弹性介质中传播的波场进行衰减补偿,并通过采用合适的成像算子对微地震震源进行定位与裂缝成像.本文基于耗散与频散解耦的分数阶黏弹性波动方程模拟波场,采用low rank分解近似混合域算子,分离衰减相关项并反转耗散项符号,并在补偿的衰减项波场的波数域中进行低通滤波,压制噪声的影响;使用优化后的成像算子进行微地震震源定位,并通过分离散射波场,对散射波进行逆时反传寻找裂缝.数值实验证明,本文方法通过low rank近似有效提高了计算效率,衰减补偿算子在滤波器约束下能够稳定地补偿反向延拓的波场,优化后的成像算子能够在压制随机噪声的同时进一步提高计算效率和定位分辨率.

基于Low-rank一步法波场延拓的黏声各向异性介质纯qP波正演模拟

各向异性介质纯qP波正演模拟及逆时偏移近年受到广泛关注,但它虽考虑了地下介质的各向异性特征,却忽略了黏滞性特征,使得最终偏移结果中噪声增加、分辨率降低。常规拟声波方程存在伪横波干扰、受模型参数限制(ε≥δ)、传播不稳定等因素影响,极大地限制了其应用。为此,引入一步法波场延拓方法,推导了黏声介质方程在空间—波数域的表达形式;结合空间—波数域各向异性介质延拓算子,构建一种适用于黏声各向异性介质的空间—波数域纯qP波波场延拓算子;引入Low-rank分解算法,实现基于Low-rank一步法波场延拓的黏声各向异性介质纯qP波正演模拟。数值模拟结果表明:①地震波场能同时表现出各向异性特征和黏滞性特征,更符合实际地下介质情况;②该方法克服了拟声波方程的局限性,消除了伪横波干扰,不受模型参数限制且地震波场能稳定传播;③在适当增大时间步长情形下无数值频散现象,所提算法能同时兼顾计算效率和计算精度,是一种稳定、高效的正演模拟方法,为基于Q补偿的各向异性介质逆时偏移提供了理论依据。

基于混沌吸引子重构和Low-rank聚类的跳频信号电台分选

辐射源无调制信息的暂态信号能够表征辐射源发射机的无意调制特性,对该暂态信号分析可实现辐射源识别。而跳频电台在开机以及频率转换瞬间,都存在一个无信息传送的暂态调整时间,该暂态调整瞬间,电台发射的信号是无调制信息的非线性、非平稳和非高斯信号。该暂态时间序列可反映跳频电台的器件特性,同时该序列往往呈现复杂的混沌特性。因此,借鉴混沌时间序列分析的思想,同时利用暂态信号的Low-rank特性,该文提出了一种基于暂态信号混沌吸引子重构和Low-rank聚类的跳频信号电台分选算法。实验测试表明:跳频电台的暂态信号时间序列属于混沌时间序列,同时实测多跳频信号的电台分选结果证明了Low-rank聚类算法在跳频电台分选上的可行性。

Improving bubble-particle attachment during the flotation of low rank coal by surface modification

Based on the problems caused by many oxygen-containing functional groups and poor floatability on the surface of low rank coal,the characteristics of low rank coal were analyzed systematically by means of scanning electron microscopy(SEM),X-ray diffraction(XRD)and X-Ray photoelectron spectroscopy(XPS)techniques.The bubble-particle induction time was used to determine the characterization of the bubble-particle attachment,and the bubble-particle attachment of low rank coal modified by soaking the coal samples in an acid or alkaline solution was analyzed.The floatability of the modified coal surface was verified by flotation tests.The results show that the particle size of 0.125–0.074 mm of the coal sample exhibited better bubble-particle attachment characteristics.The small bubble,the larger approach velocity of bubble and the larger bubble deformation were more helpful to enhance the bubbleparticle attachment.For an acid solution,the smaller the p H was and the longer the soaking time was,the better the floatability of the coal sample and the higher the combustible material recovery were.The combustible material recovery of low rank coal was increased to 78.79%by soaking the sample in an acid solution of pH=0 for 180 min.On the contrary there was a best concentration for the alkaline solution.

基于Low-rank分解的复杂TI介质纯qP波正演模拟与逆时偏移

近年来,面向实际应用的TI介质准P波正演模拟与逆时偏移成像技术受到空前的关注.基于常规耦合型传播方程的正演模拟方法不仅存在伪横波及频散假象干扰,而且还遭受模型参数限制(η>0)和不稳定影响;而纯qP波方程的推导繁琐,且由于方程中包含拟微分算子造成求解难度大且精度有限.为此,本文首先构建了一种适用于任意TI介质的纯qP波传播算子,然后借助Low-rank分解求取该算子中的空间-波数域矩阵,同时引入Cerjan衰减边界条件来压制边界反射干扰,最终实现了一种间接的纯qP波波场外推方案,并将其成功应用于复杂TI介质正演模拟与逆时偏移成像中.通过开展数值模拟,并与其他方法对比表明:①该方法既避免了纯qP波方程的繁琐推导,又克服了耦合型方程对模型参数的限制;②还彻底消除了残余伪横波噪音及数值频散;③且能适应较大时间或空间步长及高频震源,是一种相对准确且稳定的各向异性纵波正演与成像策略.

TTI介质Low-rank有限差分法高效正演模拟及逆时偏移

计算效率是制约各向异性逆时偏移实用化的关键因素,此外,伪横波假象、数值频散以及不稳定问题也是TTI介质qP波正演模拟及逆时偏移的固有难题。Low-rank波场延拓算法虽能解决上述三方面问题,但其运算速度受模型参数限制,计算效率较低。为此,本文基于混合网格有限差分思想,给出一种新的紧致差分模板,并借助Low-rank分解求取与模型匹配的自适应差分系数,进而实现一种针对TTI介质的Low-rank有限差分法高效正演模拟及逆时偏移成像策略。数值模型测试结果表明:本文方法既继承了有限差分法高效灵活的特点,又拥有Low-rank波场延拓方法准确计算纯qP波波场的优势,即在提高计算效率的同时避免了伪横波假象和数值不稳定,是一种兼顾成像精度与计算效率的各向异性逆时偏移实用方法。

各向异性介质Low-rank有限差分法纯qP波叠前平面波最小二乘逆时偏移

拟声波最小二乘逆时偏移是一种极具潜力的地震波成像工具,但该方法遭受各向异性拟声波近似的限制,TTI介质正演模拟不稳定、反偏移记录中遭受伪横波二次扰动及数值频散假象,另外拟声波最小二乘逆时偏移还面临计算效率低、收敛速度慢、对速度等模型参数依赖性高等问题.为了克服各向异性拟声波最小二乘逆时偏移的缺陷,在反演框架下,本文借助Low-rank有限差分算法首次提出并实现了TTI介质纯qP波线性正演模拟及纯qP波最小二乘逆时偏移;为了进一步提升反演成像效率,同时改善反演成像方法对模型参数误差的依赖性及对地震数据噪声的适应性,通过引入叠前平面波优化策略,发展了TTI介质纯qP波叠前平面波最小二乘逆时偏移成像方法.在编程实现方法的基础上,通过开展模型成像测试,展示了本方法的优势和潜力:一方面加快了反演成像效率,另一方面也提升了方法的抗噪性,同时还降低了方法对模型参数的依赖性.

The mechanism and products for co-thermal extraction of biomass and low-rank coal with NMP

The high-value utilization of low-rank coal would allow for expanding energy sources,improving energy efficiencies,and alleviating environmental issues.In order to use low-rank coal effectively,the hypercoals(HPCs)were co-extracted from two types of low-rank coal and biomass via N-methyl-2-purrolidinone(NMP)under mild conditions.The structures of the HPCs and residues were characterized by proximate and ultimate analysis,Raman spectra,and Fourier transform infrared(FT-IR)spectra.The carbon structure changes within the raw coals and HPCs were discussed.The individual thermal dissolution of Xibu(XB)coal,Guandi(GD)coal,and the biomass demonstrated that the biomass provided the lowest thermal dissolution yield Y1 and the highest thermal soluble yield Y2 at 280℃,and the ash content of three HPCs decreased as the extraction temperature rose.Co-thermal extractions in NMP at various coal/biomass mass ratios were performed,demonstrating a positive synergic effect for Y2 in the whole coal/biomass mass ratios.The maximum value of Y2 was 52.25wt% for XB coal obtained with a XB coal/biomass of 50wt% biomass.The maximum value of Y2 was 50.77wt% for GD coal obtained with a GD coal/biomass of 1:4.The difference for the optimal coal/biomass mass ratios between XB and GD coals could be attributed to the different co-extraction mechanisms for this two type coals.

Influence of heating rate on reactivity and surface chemistry of chars derived from pyrolysis of two Chinese low rank coals

A series of char samples were derived from pyrolysis of two typical low-rank coals in China(Shengli lignite and Shenmu bituminous coal) at low, medium and fast heating rates, respectively, to the same pyrolysis temperature 750 °C. Then these chars were characterized by means of thermogravimetric analysis and Fourier transform infrared spectrometer with the aim to investigate the influence of heating rate in pyrolysis process on gasification reactivity and surface chemistry of them. Besides, a homogeneous model was used to quantitatively analyze the activation energy of gasification reaction. The results reveal that Shengli lignite and its derived chars behave higher gasification reactivity and have less content of oxygen functional groups than Shenmu coal and chars. Meanwhile, chars derived from Shengli lignite at 50 °C/min and Shenmu coal at 200 °C/min have the greatest gasification reactivity, respectively. The oxygen functional groups in Shengli lignite are easily thermo-decomposed, and they are less affected by the heating rate, while that in Shenmu coal have a significant change with the variation of heating rate.In addition, there is no good correlation between the change of oxygen functional groups and that of the gasification reactivity of the derived chars from pyrolysis at different heating rates.

Resistivity response to the porosity and permeability of low rank coal

Laojunmiao coal samples from the eastern Junggar basin were studied to understand the relationship between coal resistivity and the physical parameters of coal reservoirs under high temperatures and pressures.Specifically,we analysed the relationship of coal resistivity to porosity and permeability via heating and pressurization experiments.The results indicated that coal resistivity decreases exponentially with increasing pressure.Increasing the temperature decreases the resistivity.The sensitivity of coal resistivity to the confining pressure is worse when the temperature is higher.The resistivity of dry coal samples was linearly related to φ~m.Increasing the temperature decreased the cementation exponent(m).Increasing the confining pressure exponentially decreases the porosity.Decreasing the pressure increases the resistivity and porosity for a constant temperature.Increasing the temperature yields a quadratic relationship between the resistivity and permeability for a constant confining pressure.Based on the Archie formula,we obtained the coupling relationship between coal resistivity and permeability for Laojunmiao coal samples at different temperatures and confining pressures.

Co-pyrolysis characteristics and interaction route between low-rank coals and Shenhua coal direct liquefaction residue

To reasonably utilize the coal direct liquefaction residue(DLR), contrasting research on the co-pyrolysis between different low-rank coals and DLR was investigated using a TGA coupled with an FT-IR spectrophotometer and a fixed-bed reactor. GC–MS, FTIR, and XRD were used to explore the reaction mechanisms of the various co-pyrolysis processes. Based on the TGA results, it was confirmed that the tetrahydrofuran insoluble fraction of DLR helped to catalyze the conversion reaction of lignite. Also, the addition of DLR improved the yield of tar in the fixed-bed, with altering the composition of the tar. Moreover, a kinetic analysis during the co-pyrolysis was conducted using a distributed activation energy model. The co-pyrolysis reactions showed an approximate double-Gaussian distribution.

Pore Characteristics of Vitrain and Durain in Low Rank Coal Area

The low rank coalbed methane (CBM) has great potential for exploration and development in China, but its exploitation level is low at present stage. The pores are the storage space of CBM, so recognizing its structural characteristics has very important practical significance for the development of CBM. The samples of No. 4 and upper No. 4 coalbed in Dafosi were selected to carry out the analysis of mercury injection test, nitrogen adsorption test and scanning electron microscopy to study the different lithotypes of the pore structure, pore throat distribution and fracture character of low rank coal reservoir. The results showed that micropore of low rank coal in Dafosi relatively developed and the pore volume of vitrain was equivalent to durain. The pore throat of durain was larger than vitrain, the connectivity was better and the fissures were more developed. The percolation capacity and reservoir performance of upper No. 4 coal was better than No. 4 coal. Generally, the potential of exploration and development of upper No. 4 coal in the study area was better than that of No. 4, and the developed area of durain was more beneficial for the development of CBM.

Swelling Measurements of a Low Rank Coal in Supercritical CO₂

Coal swelling in the presence of water as well as CO2 is a well-known phenomenon, and these may affect the permeability of coal. Quantifying swelling effects is becoming an important issue to verify the suitability of particular coal seams for CO2-enhanced coal bed methane recovery projects. In this report, coal swelling experiments using a visualization method in the CO2 supercritical conditions were conducted on crushed coal samples. The measurement apparatus was designed specifically for the present swelling experiment using a visualization method. Crushed coal samples were used instead of block coal samples to shorten equilibrium time and to solve the problem of limited availability of core coal samples. Dry and wet coal samples were used in the experiments because there is relatively limited information about how the swelling of coal by CO2 is affected by water saturation. Moreover, some coal seams are saturated with water in initial reservoir conditions. The maximum volumetric swelling was around 3% at 10 MPa for dry samples and almost half that at the same pressure for wet samples. The wet samples showed lower volumetric swelling than dry ones because the wet coal samples were already swollen by water. Experimental results obtained for swelling were comparable with other reports. Our visualization method using crushed samples has advantages in terms of sample preparation and experimental execution compared with the other methods used to measure coal swelling using block samples.