DPT‐tracker:Dual pooling transformer for efficient visual tracking

Transformer tracking always takes paired template and search images as encoder input and conduct feature extraction and target‐search feature correlation by self and/or cross attention operations,thus the model complexity will grow quadratically with the number of input images.To alleviate the burden of this tracking paradigm and facilitate practical deployment of Transformer‐based trackers,we propose a dual pooling transformer tracking framework,dubbed as DPT,which consists of three components:a simple yet efficient spatiotemporal attention model(SAM),a mutual correlation pooling Trans-former(MCPT)and a multiscale aggregation pooling Transformer(MAPT).SAM is designed to gracefully aggregates temporal dynamics and spatial appearance information of multi‐frame templates along space‐time dimensions.MCPT aims to capture multi‐scale pooled and correlated contextual features,which is followed by MAPT that aggregates multi‐scale features into a unified feature representation for tracking prediction.DPT tracker achieves AUC score of 69.5 on LaSOT and precision score of 82.8 on Track-ingNet while maintaining a shorter sequence length of attention tokens,fewer parameters and FLOPs compared to existing state‐of‐the‐art(SOTA)Transformer tracking methods.Extensive experiments demonstrate that DPT tracker yields a strong real‐time tracking baseline with a good trade‐off between tracking performance and inference efficiency.

A deep Koopman operator-based modelling approach for long-term prediction of dynamics with pixel-level measurements

Although previous studies have made some clear leap in learning latent dynamics from high-dimensional representations,the performances in terms of accuracy and inference time of long-term model prediction still need to be improved.In this study,a deep convolutional network based on the Koopman operator(CKNet)is proposed to model non-linear systems with pixel-level measurements for long-term prediction.CKNet adopts an autoencoder network architecture,consisting of an encoder to generate latent states and a linear dynamical model(i.e.,the Koopman operator)which evolves in the latent state space spanned by the encoder.The decoder is used to recover images from latent states.According to a multi-step ahead prediction loss function,the system matrices for approximating the Koopman operator are trained synchronously with the autoencoder in a mini-batch manner.In this manner,gradients can be synchronously transmitted to both the system matrices and the autoencoder to help the encoder self-adaptively tune the latent state space in the training process,and the resulting model is time-invariant in the latent space.Therefore,the proposed CKNet has the advantages of less inference time and high accuracy for long-term prediction.Experiments are per-formed on OpenAI Gym and Mujoco environments,including two and four non-linear forced dynamical systems with continuous action spaces.The experimental results show that CKNet has strong long-term prediction capabilities with sufficient precision.

Reference-driven method for MR image reconstruction based on wavelet sparsity and nonlocal total variation

A novel reference-driven method for MR image reconstruction based on wavelet sparsity and nonlocal total variation（NLTV）is proposed.Utilizing the sparsity of the difference image between the target image and the motion-compensated reference image in wavelet transform domain,the proposed method does not need to estimate contrast changes and therefore increases computational efficiency.Additionally,NLTV regularization is applied to preserve image details and features without blocky effects.An efficient alternating iterative algorithm is used to estimate motion effects and reconstruct the difference image.Experimental results demonstrate that the proposed method can significantly reduce sampling rate or improve the quality of the reconstructed image alternatively.

Image Motion Compensation of Off-Axis TMA Three-Line ArrayAerospace Mapping Cameras

To enhance the image motion compensation accuracy of off-axis three-mirror anastigmatic( TMA)three-line array aerospace mapping cameras,a new method of image motion velocity field modeling is proposed in this paper. Firstly,based on the imaging principle of mapping cameras,an analytical expression of image motion velocity of off-axis TMA three-line array aerospace mapping cameras is deduced from different coordinate systems we established and the attitude dynamics principle. Then,the case of a three-line array mapping camera is studied,in which the simulation of the focal plane image motion velocity fields of the forward-view camera,the nadir-view camera and the backward-view camera are carried out,and the optimization schemes for image motion velocity matching and drift angle matching are formulated according the simulation results. Finally,this method is verified with a dynamic imaging experimental system. The results are indicative of that when image motion compensation for nadir-view camera is conducted using the proposed image motion velocity field model,the line pair of target images at Nyquist frequency is clear and distinguishable. Under the constraint that modulation transfer function( MTF) reduces by 5%,when the horizontal frequencies of the forward-view camera and the backward-view camera are adjusted uniformly according to the proposed image motion velocity matching scheme,the time delay integration( TDI) stages reach 6 at most. When the TDI stages are more than 6,the three groups of camera will independently undergo horizontal frequency adjustment. However, when the proposed drift angle matching scheme is adopted for uniform drift angle adjustment,the number of TDI stages will not exceed 81. The experimental results have demonstrated the validity and accuracy of the proposed image motion velocity field model and matching optimization scheme,providing reliable basis for on-orbit image motion compensation of aerospace mapping cameras.

Video object's behavior analyzing based on motion history image and hidden markov model

A novel method was proposed, which extracted video object＇ s track and analyzed video object＇ s be- havior. Firstly, this method tracked the video object based on motion history image, and obtained the co- ordinate-based track sequence and orientation-based track sequence of the video object. Then the pro- posed hidden markov model （HMM） based algorithm was used to analyze the behavior of video object with the track sequence as input. Experimental results on traffic object show that this method can achieve the statistics of a mass of traffic objects＇ behavior efficiently, can acquire the reasonable velocity behavior curve of traffic object, and can recognize traffic object＇ s various behaviors accurately. It provides a base for further research on video object behavior.

Research on single image super-resolution based on very deep super-resolution convolutional neural network

Single image super-resolution(SISR)is a fundamentally challenging problem because a low-resolution(LR)image can correspond to a set of high-resolution(HR)images,while most are not expected.Recently,SISR can be achieved by a deep learning-based method.By constructing a very deep super-resolution convolutional neural network(VDSRCNN),the LR images can be improved to HR images.This study mainly achieves two objectives:image super-resolution(ISR)and deblurring the image from VDSRCNN.Firstly,by analyzing ISR,we modify different training parameters to test the performance of VDSRCNN.Secondly,we add the motion blurred images to the training set to optimize the performance of VDSRCNN.Finally,we use image quality indexes to evaluate the difference between the images from classical methods and VDSRCNN.The results indicate that the VDSRCNN performs better in generating HR images from LR images using the optimized VDSRCNN in a proper method.

Double Deep Q-Network Decoder Based on EEG Brain-Computer Interface

Brain-computer interfaces(BCI)use neural activity as a control signal to enable direct communication between the human brain and external devices.The electrical signals generated by the brain are captured through electroencephalogram(EEG)and translated into neural intentions reflecting the user’s behavior.Correct decoding of the neural intentions then facilitates the control of external devices.Reinforcement learning-based BCIs enhance decoders to complete tasks based only on feedback signals(rewards)from the environment,building a general framework for dynamic mapping from neural intentions to actions that adapt to changing environments.However,using traditional reinforcement learning methods can have challenges such as the curse of dimensionality and poor generalization.Therefore,in this paper,we use deep reinforcement learning to construct decoders for the correct decoding of EEG signals,demonstrate its feasibility through experiments,and demonstrate its stronger generalization on motion imaging(MI)EEG data signals with high dynamic characteristics.

Intravoxel incoherent motion diffusion-weighted imaging for monitoring chemotherapeutic efficacy in gastric cancer

AIM: To assess intravoxel incoherent motion diffusion-weighted imaging (IVIM-DWI) for monitoring early efficacy of chemotherapy in a human gastric cancer mouse model.METHODS: IVIM-DWI was performed with 12 b-values (0-800 s/mm2) in 25 human gastric cancer-bearing nude mice at baseline (day 0), and then they were randomly divided into control and 1-, 3-, 5- and 7-d treatment groups (n = 5 per group). The control group underwent longitudinal MRI scans at days 1, 3, 5 and 7, and the treatment groups underwent subsequent MRI scans after a specified 5-fluorouracil/calcium folinate treatment. Together with tumor volumes (TV), the apparent diffusion coefficient (ADC) and IVIM parameters [true water molecular diffusion coefficient (D), perfusion fraction (f) and pseudo-related diffusion coefficient (D*)] were measured. The differences in those parameters from baseline to each measurement (ΔTV%, ΔADC%, ΔD%, Δf% and ΔD*%) were calculated. After image acquisition, tumor necrosis, microvessel density (MVD) and cellular apoptosis were evaluated by hematoxylin-eosin (HE), CD31 and terminal-deoxynucleotidyl transferase mediated nick end labeling (TUNEL) staining respectively, to confirm the imaging findings. Mann-Whitney test and Spearman’s correlation coefficient analysis were performed.RESULTS: The observed relative volume increase (ΔTV%) in the treatment group were significantly smaller than those in the control group at day 5 (ΔTVtreatment% = 19.63% ± 3.01% and ΔTVcontrol% = 83.60% ± 14.87%, P = 0.008) and day 7 (ΔTVtreatment% = 29.07% ± 10.01% and ΔTVcontrol% = 177.06% ± 63.00%, P = 0.008). The difference in ΔTV% between the treatment and the control groups was not significant at days 1 and 3 after a short duration of treatment. Increases in ADC in the treatment group (ΔADC%treatment, median, 30.10% ± 18.32%, 36.11% ± 21.82%, 45.22% ± 24.36%) were significantly higher compared with the control group (ΔADC%control, median, 4.98% ± 3.39%, 6.26% ± 3.08%, 9.24% ± 6.33%) at days 3, 5 and 7 (P = 0.008, P = 0.016, P = 0.008, respectively). Increases in D in the treatment group (ΔD%treatment, median 17.12% ± 8.20%, 24.16% ± 16.87%, 38.54% ± 19.36%) were higher than those in the control group (ΔD%control, median -0.13% ± 4.23%, 5.89% ± 4.56%, 5.54% ± 4.44%) at days 1, 3, and 5 (P = 0.032, P = 0.008, P = 0.016, respectively). Relative changes in f were significantly lower in the treatment group compared with the control group at days 1, 3, 5 and 7 follow-up (median, -34.13% ± 16.61% vs 1.68% ± 3.40%, P = 0.016; -50.64% ± 6.82% vs 3.01% ± 6.50%, P = 0.008; -49.93% ± 6.05% vs 0.97% ± 4.38%, P = 0.008, and -46.22% ± 7.75% vs 8.14% ± 6.75%, P = 0.008, respectively). D* in the treatment group decreased significantly compared to those in the control group at all time points (median, -32.10% ± 12.22% vs 1.85% ± 5.54%, P = 0.008; -44.14% ± 14.83% vs 2.29% ± 10.38%, P = 0.008; -59.06% ± 19.10% vs 3.86% ± 5.10%, P = 0.008 and -47.20% ± 20.48% vs 7.13% ± 9.88%, P = 0.016, respectively). Furthermore, histopathologic findings showed positive correlations with ADC and D and tumor necrosis (rs = 0.720, P < 0.001; rs = 0.522, P = 0.007, respectively). The cellular apoptosis of the tumor also showed positive correlations with ADC and D (rs = 0.626, P = 0.001; rs = 0.542, P = 0.005, respectively). Perfusion-related parameters (f and D*) were positively correlated to MVD (rs = 0.618, P = 0.001; rs = 0.538, P = 0.006, respectively), and negatively correlated to cellular apoptosis of the tumor (rs = -0.550, P = 0.004; rs = -0.692, P < 0.001, respectively).CONCLUSION: IVIM-DWI is potentially useful for predicting the early efficacy of chemotherapy in a human gastric cancer mouse model.

Image motion and experimental study of a 0.1″space pointing measuring instrument for micro-vibration conditions

There exists an increasing need for Milli-Arc-Seconds(MAS)accuracy pointing measurement for current and future space systems.To meet the 0.1″space pointing measurement accuracy requirements of spacecraft in future,the influence of spacecraft micro-vibration on a 0.1″Space Pointing Measuring Instrument(SPMI)is studied.A Quasi-Zero Stiffness Device(QZSD)with adaptive adjustment and variable stroke was proposed.Then,a series of micro-vibration experiments of the SPMI were carried out.The influence of the micro-vibration generated by Guidance Navigation Control(GNC)attitude control components under different attitudes on the SPMI was analyzed.Point spread function of image motion in micro-vibration was also derived.Further,the changes of image motion under the micro-vibration environment were evaluated by extracting the gray centroid of the images,and the experiment processes and results are deeply discussed.The results show that the firstorder frequency of the QZSD system is 0.114 Hz,and it is induced by a double pendulum system;the image motion of single flywheel spinning reached 0.015 pixels;whilst the image motion reached 0.03 pixels when three flywheels are combined spinning.These latest findings provide a beneficial theoretical and technical support for the development of spacecraft with 0·1″pointing accuracy.

Research on time-frequency cross mutual of motor imagination data based on multichannel EEG signal

At present,multi-channel electroencephalogram(EEG)signal acquisition equipment is used to collect motor imagery EEG data,and there is a problem with selecting multiple acquisition channels.Choosing too many channels will result in a large amount of calculation.Components irrelevant to the task will interfere with the required features,which is not conducive to the real-time processing of EEG data.Using too few channels will result in the loss of useful information and low robustness.A method of selecting data channels for motion imagination is proposed based on the time-frequency cross mutual information(TFCMI).This method determines the required data channels in a targeted manner,uses the common spatial pattern mode for feature extraction,and uses support vector ma-chine(SVM)for feature classification.An experiment is designed to collect motor imagery EEG da-ta with four experimenters and adds brain-computer interface(BCI)Competition IV public motor imagery experimental data to verify the method.The data demonstrates that compared with the meth-od of selecting too many or too few data channels,the time-frequency cross mutual information meth-od using motor imagery can improve the recognition accuracy and reduce the amount of calculation.

Self-service Behavior Recognition Algorithm Based on Improved Motion History Image Network

Aiming at the problem of automatic detection of normal operation behavior in self-service business management,with improved motion history image as input,a recognition method of convolutional neural network is proposed to timely judge the occurrence of anomie behavior.Firstly,the key frame sequence was extracted from the self-service operation video based on the method of uniform energy down-sampling.Secondly,combined with the timing information of key frames to adaptively estimate the decay parameters of the motion history image,adding information contrast to generating a logic matrix can improve the calculation speed of the improved motion history image.Finally,the formed motion history image was input into the established convolutional neural network to obtain the class of self-service behavior and distinguish anomie behavior.In real scenarios of self-service baggage check-in for civil aviation passengers,the typical check-in behavior data set is established and tested in actual self-service baggage check-in system of the airport.The results show that the method proposed can effectively identify typical anomie behaviors and has high practical value.

Motion de-blurring by second-order intensity-correlated imaging

For a Hanbury Brown and Twiss system, the influence of relative motion between the object and the detection plane on the resolution of second-order intensity-correlated imaging is investigated. The analytical results, which are backed up by experiments, demonstrate that the amplitude and mode of the object＇s motion have no effect on the second-order intensity-correlated imaging and that high-resolution imaging can be always achieved by using a phase-retrieval method from the diffraction patterns. The use of motion de-blurring imaging for this approach is also discussed.

Attitude Guidance Algorithms for Agile Satellite Dynamic Imaging

Dynamic imaging modes are increasingly crucial for agile satellites to perform complicated Earth observation tasks.In this study,a direct guidance algorithm is developed to calculate the reference attitude and velocity for dynamic imaging mode from target geolocation information while considering the constraints on both the satellite camera boresight axis and the image motion vector.The two slew angles are determined directly,and no rotation around the boresight or reference vector is required.The proposed approach employs a direct solution instead of the iterative process to obtain the reference attitude,which releases the onboard control system from the intensive computational load.To illustrate the performance of the proposed guidance algorithms,numerical simulation results are presented.

Setup and data analysis for functional magnetic resonance imaging of awake cat visual cortex

Functional magnetic resonance imaging（fMRI）is one of the most commonly used methods in cognitive neuroscience on humans.In recent decades,fMRI has also been used in the awake monkey experiments to localize functional brain areas and to compare the functional differences between human and monkey brains.Several procedures and paradigms have been developed to maintain proper head fixation and to perform motion control training.In this study,we extended the application of fMRI to awake cats without training,receiving a flickering checkerboard visual stimulus projected to a screen in front of them in a block-design paradigm.We found that body movement-induced non-rigid motion introduced artifacts into the functional scans,especially those around the eye and neck.To correct for these artifacts,we developed two methods：one for general experimental design,and the other for studies of whether a checkerboard task could be used as a localizer to optimize the motioncorrection parameters.The results demonstrated that,with proper animal fixation and motion correction procedures,it is possible to perform fMRI experiments with untrained awake cats.

Evaluation Value of Intravoxel Incoherent Motion Diffusion-Weighted Imaging on Early Efficacy of Magnetic Resonance-Guided High-Intensity Focused Ultrasound Ablation for Uterine Adenomyoma

To investigate the evaluation value of intravoxel incoherent motion diffusion-weighted imaging(IVIM-DWI)on the early efficacy of magnetic resonance-guided high-intensity focused ultrasound(MRgFUS)ablation for uterine adenomyoma.The clinical and magnetic resonance imaging(MRI)data of 36 patients with uterine adenomyoma before and after MRgFUS treatment in our hospital from January 2018 to December 2018 were retrospectively analyzed.All the 36 patients underwent MRI examination one day before operation and immediately after operation using GE Discovery MR7503.0T MRI,including conventional sequences(T1WI,T2WI,and T2 fat suppression sequences)plain scan,IVIM-DWI sequences with 9 b values,and contrast enhanced-MRI sequences.The IVIM-DWI quantitative parameters(true diffusion coefficient D,perfusion related diffusion coefficient D*,and perfusion fraction f)of double-exponential model were obtained by using GE ADW 4.7 functool,a postprocessor.SPSS 24.0 software was used to analyze the difference in parameter between the ablation and non-ablation areas of uterine adenomyoma.DWI signal in the ablation area of uterine adenomyoma was increased,and manifested as heterogeneous diffuse high signal,with low central signal and high edge signal.Values of D,D*and f in the ablation area of uterine adenomyoma were significantly lower than those in the non-ablation area,and there was statistical difference between the two(P<0.05).The areas under receiver operating characteristic(ROC)curve of D,D*and f values in the ablation area of uterine adenomyoma were 0.854,0.898 and 0.924,respectively;the optimal thresholds for the diagnosis of ablation area of uterine adenomyoma were 0.81×10−3 mm2/s,4.99×10−3 mm2/s and 0.24,respectively;the diagnostic sensitivity was 80.6%,72.2%and 94.4%,respectively;and the specificity was 91.7%,97.2%and 94.4%,respectively.IVIM-DWI has a certain clinical value in the evaluation on early efficacy of MRgFUS ablation of uterine adenomyosis.

Coordinate difference homogenization matching method for motion correction in 3D range-intensity correlation laser imaging

This Letter proposes a coordinate difference homogenization matching method to solve motion influence in three-dimensional（3D） range-intensity correlation laser imaging. Firstly, features and feature pairs of gate images are obtained by speeded-up robust figures and bi-directional feature matching methods. The original mean value of the feature-pair coordinate differences is calculated. Comparing the coordinate differences with the original mean value, the wrong feature pairs are removed, and then an optimized mean value is updated. The final feature-pair coordinates are re-registered based on the updated mean value. Thus, an accurate transformation is established to rectify motion gate images for 3D reconstruction. In the experiment, a 3D image of a tower at 780 m is successfully captured by our laser gated imaging system on a pan-tilt device.