Depth-Guided Vision Transformer With Normalizing Flows for Monocular 3D Object Detection

Monocular 3D object detection is challenging due to the lack of accurate depth information.Some methods estimate the pixel-wise depth maps from off-the-shelf depth estimators and then use them as an additional input to augment the RGB images.Depth-based methods attempt to convert estimated depth maps to pseudo-LiDAR and then use LiDAR-based object detectors or focus on the perspective of image and depth fusion learning.However,they demonstrate limited performance and efficiency as a result of depth inaccuracy and complex fusion mode with convolutions.Different from these approaches,our proposed depth-guided vision transformer with a normalizing flows(NF-DVT)network uses normalizing flows to build priors in depth maps to achieve more accurate depth information.Then we develop a novel Swin-Transformer-based backbone with a fusion module to process RGB image patches and depth map patches with two separate branches and fuse them using cross-attention to exchange information with each other.Furthermore,with the help of pixel-wise relative depth values in depth maps,we develop new relative position embeddings in the cross-attention mechanism to capture more accurate sequence ordering of input tokens.Our method is the first Swin-Transformer-based backbone architecture for monocular 3D object detection.The experimental results on the KITTI and the challenging Waymo Open datasets show the effectiveness of our proposed method and superior performance over previous counterparts.

A Normalizing Flow-Based Bidirectional Mapping Residual Network for Unsupervised Defect Detection

Unsupervised methods based on density representation have shown their abilities in anomaly detection,but detection performance still needs to be improved.Specifically,approaches using normalizing flows can accurately evaluate sample distributions,mapping normal features to the normal distribution and anomalous features outside it.Consequently,this paper proposes a Normalizing Flow-based Bidirectional Mapping Residual Network(NF-BMR).It utilizes pre-trained Convolutional Neural Networks(CNN)and normalizing flows to construct discriminative source and target domain feature spaces.Additionally,to better learn feature information in both domain spaces,we propose the Bidirectional Mapping Residual Network(BMR),which maps sample features to these two spaces for anomaly detection.The two detection spaces effectively complement each other’s deficiencies and provide a comprehensive feature evaluation from two perspectives,which leads to the improvement of detection performance.Comparative experimental results on the MVTec AD and DAGM datasets against the Bidirectional Pre-trained Feature Mapping Network(B-PFM)and other state-of-the-art methods demonstrate that the proposed approach achieves superior performance.On the MVTec AD dataset,NF-BMR achieves an average AUROC of 98.7%for all 15 categories.Especially,it achieves 100%optimal detection performance in five categories.On the DAGM dataset,the average AUROC across ten categories is 98.7%,which is very close to supervised methods.

Coarse-to-Fine Video Instance Segmentation With Factorized Conditional Appearance Flows

We introduce a novel method using a new generative model that automatically learns effective representations of the target and background appearance to detect,segment and track each instance in a video sequence.Differently from current discriminative tracking-by-detection solutions,our proposed hierarchical structural embedding learning can predict more highquality masks with accurate boundary details over spatio-temporal space via the normalizing flows.We formulate the instance inference procedure as a hierarchical spatio-temporal embedded learning across time and space.Given the video clip,our method first coarsely locates pixels belonging to a particular instance with Gaussian distribution and then builds a novel mixing distribution to promote the instance boundary by fusing hierarchical appearance embedding information in a coarse-to-fine manner.For the mixing distribution,we utilize a factorization condition normalized flow fashion to estimate the distribution parameters to improve the segmentation performance.Comprehensive qualitative,quantitative,and ablation experiments are performed on three representative video instance segmentation benchmarks(i.e.,YouTube-VIS19,YouTube-VIS21,and OVIS)and the effectiveness of the proposed method is demonstrated.More impressively,the superior performance of our model on an unsupervised video object segmentation dataset(i.e.,DAVIS19)proves its generalizability.Our algorithm implementations are publicly available at https://github.com/zyqin19/HEVis.

Efficient parameter inference for gravitational wave signals in the presence of transient noises using temporal and time-spectral fusion normalizing flow

Glitches represent a category of non-Gaussian and transient noise that frequently intersects with gravitational wave(GW)signals,thereby exerting a notable impact on the processing of GW data.The inference of GW parameters,crucial for GW astronomy research,is particularly susceptible to such interference.In this study,we pioneer the utilization of a temporal and time-spectral fusion normalizing flow for likelihood-free inference of GW parameters,seamlessly integrating the high temporal resolution of the time domain with the frequency separation characteristics of both time and frequency domains.Remarkably,our findings indicate that the accuracy of this inference method is comparable to that of traditional non-glitch sampling techniques.Furthermore,our approach exhibits a greater efficiency,boasting processing times on the order of milliseconds.In conclusion,the application of a normalizing flow emerges as pivotal in handling GW signals affected by transient noises,offering a promising avenue for enhancing the field of GW astronomy research.

Inverse Molecule Design with Invertible Neural Networks as Generative Models

Using neural networks for supervised learning means learning a function that maps input x to output y. However, in many applications, the inverse learning is also wanted, i.e., inferring y from x, which requires invertibility of the learning. Since the dimension of input is usually much higher than that of the output, there is information loss in the forward learning from input to output. Thus, creating invertible neural networks is a difficult task. However, recent development of invertible learning techniques such as normalizing flows has made invertible neural networks a reality. In this work, we applied flow-based invertible neural networks as generative models to inverse molecule design. In this context, the forward learning is to predict chemical properties given a molecule, and the inverse learning is to infer the molecules given the chemical properties. Trained on 100 and 1000 molecules, respectively, from a benchmark dataset QM9, our model identified novel molecules that had chemical property values well exceeding the limits of the training molecules as well as the limits of the whole QM9 of 133,885 molecules, moreover our generative model could easily sample many molecules (x values) from any one chemical property value (y value). Compared with the previous method in the literature that could only optimize one molecule for one chemical property value at a time, our model could be trained once and then be sampled any multiple times and for any chemical property values without the need of retraining. This advantage comes from treating inverse molecule design as an inverse regression problem. In summary, our main contributions were two: 1) our model could generalize well from the training data and was very data efficient, 2) our model could learn bidirectional correspondence between molecules and their chemical properties, thereby offering the ability to sample any number of molecules from any y values. In conclusion, our findings revealed the efficiency and effectiveness of using invertible neural networks as generative models in inverse molecule design.

Sap flow characteristics and physiological adjustments of two dominant tree species in pure and mixed plantations in the semi-arid Loess Plateau of China

It is essential to understand the water consumption characteristics and physiological adjustments of tree species under drought conditions,as well as the effects of pure and mixed plantations on these characteristics in semi-arid regions.In this study,the normalized sap flow(SFn),leaf water potential,stomatal conductance(gs),and photosynthetic rate(Pr)were monitored for two dominant species,i.e.,Pinus tabuliformis and Hippophae rhamnoides,in both pure and mixed plantations in a semi-arid region of Chinese Loess Plateau.A threshold-delay model showed that the lower rainfall thresholds(RL)for P.tabuliformis and H.rhamnoides in pure plantations were 9.6 and 11.0 mm,respectively,and the time lags(τ)after rainfall were 1.15 and 1.76 d for corresponding species,respectively.The results indicated that P.tabuliformis was more sensitive to rainfall pulse than H.rhamnoides.In addition,strong stomatal control allowed P.tabuliformis to experience low gsand Prin response to drought,while maintaining a high midday leaf water potential(Ψm).However,H.rhamnoides maintained high gsand Prat a lowΨmexpense.Therefore,P.tabuliformis and H.rhamnoides can be considered as isohydric and anisohydric species,respectively.In mixed plantation,the values of RLfor P.tabuliformis and H.rhamnoides were 6.5 and 8.9 mm,respectively;and the values ofτwere 0.86 and 1.61 d for corresponding species,respectively,which implied that mixed afforestation enhanced the rainfall pulse sensitivity for both two species,especially for P.tabuliformis.In addition,mixed afforestation significantly reduced SFn,gs,and Prfor P.tabuliformis(P<0.05),while maintaining a high leaf water potential status.However,no significant effect of mixed afforestation of H.rhamnoides was observed at the expense of leaf water potential status in response to drought.Although inconsistent physiological responses were adopted by these species,the altered water consumption characteristics,especially for P.tabuliformis indicated that the mixed afforestation requires further investigation.

Active contours with normally generalized gradient vector flow external force

Gradient vector flow （GVF） is an effective external force for active contours, but its iso- tropic nature handicaps its performance. The recently proposed gradient vector flow in the normal direction （NGVF） is anisotropic since it only keeps the diffusion along the normal direction of the isophotes; however, it has difficulties forcing a snake into long, thin boundary indentations. In this paper, a novel external force for active contours called normally generalized gradient vector flow （NGGVF） is proposed, which generalizes the NGVF formulation to include two spatially varying weighting functions. Consequently, the proposed NGGVF snake is anisotropic and would improve ac- tive contour convergence into long, thin boundary indentations while maintaining other desirable properties of the NGVF snake, such as enlarged capture range, initialization insensitivity and good convergence at concavities. The advantages on synthetic and real images are demonstrated.

STUDY OF REGIONAL STABILITY OF ECD DISTRIBUTION IN NORMAL BRAIN

Objective To evaluate in vivo stability of ethylenedylbis cysteine diethylester (ECD) brain SPECT. Methods Each of 13 normal volunteers (31. 2 ± 11. 8 years) has 12 dynamic SPECK scans ac-quired in 60min 1h after an injection of 99mTc-ECD using a triple headed gamma camera equipped with ultra high resolution fan beam collimators. Average counts per pixel were measured from frontal, temporal, parie-tal, occipital regions, cerebellum, basal ganglia, thalamus and white matter. Regional ECD clearance rates, regional gray-to-white matter (G/W) ratios and the change of the G/W ratio were calculated. Results The average ECD clearance rate was 4. 2% /h, ranged from 3. 03% /h to 5. 41% /h corresponding to white matter and occipital. There was no significant difference between regional ECD clearance rates. Regional G 7W ratio was between 1.27 to 1.75. The G/W ratio of temporal lobe was lower than the occipital ( P <0.05). The change of regional G/W ratio with time is slow. Conclusion Regional ECD distribution is stable in normal brain. ECD clearance from brain is slow and no significant regional difference.

Invariant hypersurface flows in centro-affine geometry

In this paper,the invariant geometric flows for hypersurfaces in centro-affine geometry are explored.We first present evolution equations of the centro-affine invariants corresponding to the geometric flows.Based on these fundamental evolution equations,we show that the centro-affine heat flow for hypersurfaces is equivalent to a system of ordinary differential equations,which can be solved explicitly.Finally,the centro-affine invariant normal flows for hypersurfaces are investigated,and two specific flows are provided to illustrate the behaviour of the flows.

Solving Time Dependent Fokker-Planck Equations via Temporal Normalizing Flow

In this work,we propose an adaptive learning approach based on temporal normalizing flows for solving time-dependent Fokker-Planck(TFP)equations.It is well known that solutions of such equations are probability density functions,and thus our approach relies on modelling the target solutions with the temporal normalizing flows.The temporal normalizing flow is then trained based on the TFP loss function,without requiring any labeled data.Being a machine learning scheme,the proposed approach is mesh-free and can be easily applied to high dimensional problems.We present a variety of test problems to show the effectiveness of the learning approach.

Sampling with Prior Knowledge for High-dimensional Gravitational Wave Data Analysis

Extracting knowledge from high-dimensional data has been notoriously difficult,primarily due to the so-called"curse of dimensionality"and the complex joint distributions of these dimensions.This is a particularly profound issue for high-dimensional gravitational wave data analysis where one requires to conduct Bayesian inference and estimate joint posterior distributions.In this study,we incorporate prior physical knowledge by sampling from desired interim distributions to develop the training dataset.Accordingly,the more relevant regions of the high-dimensional feature space are covered by additional data points,such that the model can learn the subtle but important details.We adapt the normalizing flow method to be more expressive and trainable,such that the information can be effectively extracted and represented by the transformation between the prior and target distributions.Once trained,our model only takes approximately 1 s on one V100 GPU to generate thousands of samples for probabilistic inference purposes.The evaluation of our approach confirms the efficacy and efficiency of gravitational wave data inferences and points to a promising direction for similar research.The source code,specifications,and detailed procedures are publicly accessible on GitHub.