GAN-DIRNet:A Novel Deformable Image Registration Approach for Multimodal Histological Images

Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.

Nonlinear Registration of Brain Magnetic Resonance Images with Cross Constraints of Intensity and Structure

Many deep learning-based registration methods rely on a single-stream encoder-decoder network for computing deformation fields between 3D volumes.However,these methods often lack constraint information and overlook semantic consistency,limiting their performance.To address these issues,we present a novel approach for medical image registration called theDual-VoxelMorph,featuring a dual-channel cross-constraint network.This innovative network utilizes both intensity and segmentation images,which share identical semantic information and feature representations.Two encoder-decoder structures calculate deformation fields for intensity and segmentation images,as generated by the dual-channel cross-constraint network.This design facilitates bidirectional communication between grayscale and segmentation information,enabling the model to better learn the corresponding grayscale and segmentation details of the same anatomical structures.To ensure semantic and directional consistency,we introduce constraints and apply the cosine similarity function to enhance semantic consistency.Evaluation on four public datasets demonstrates superior performance compared to the baselinemethod,achieving Dice scores of 79.9%,64.5%,69.9%,and 63.5%for OASIS-1,OASIS-3,LPBA40,and ADNI,respectively.

A harmonisation approach to traditional Chinese medicine registration in Asian countries

The traditional Chinese medicine(TCM)industry is critical to not only for public health but also for economic growth.According to the European Union(EU)directives,under the EU framework for(traditional)herbal medicinal products,herbal medicines with long history of use can be registered in EU.However,there is a condition in this directive in which only those that have at least 10e15 years in the EU market are accepted for registration and/or marketing authorization.In author opinion,the condition of 10-15 years of use in EU countries set within the EU regulatory framework is with consideration of the genetic differences which can result in variation in adverse drug responses among different world populations.With this concept in mind,it is reasonable to project the principal of the EU directive to the Asian countries where TCM is originated.Countries like China,Singapore,Japan and South Korea that have well established drug registration framework are in best position in executing the best practice and facilitate harmonization of registration for TCM within the region.Furthermore,the registration process itself allows more safety and efficacy data to be collected systemically before and after product registration/marketing authorization.These are valuable information for future drug development.The therapeutic value of TCM is limitless,it has been left out in the EU regulatory framework,and the opportunity for it to be expanded and carried forward in modern medicines is shadowed by the limited number of TCM that are qualified to be registered under the EU regulatory framework.An early establishment of a harmonized risk-based registration process for TCM in Asian countries is important.This will strengthen the database to substantiate the history of safe use and further preserving and expanding the therapeutic values of TCM within and beyond the Asian region.

Optimization of the Use of Spherical Targets for Point Cloud Registration Using Monte Carlo Simulation

Registrations based on the manual placement of spherical targets are still being employed by many professionals in the industry.However,the placement of those targets usually relies solely on personal experience without scientific evidence supported by numerical analysis.This paper presents a comprehensive investigation,based on Monte Carlo simulation,into determining the optimal number and positions for efficient target placement in typical scenes consisting of a pair of facades.It demonstrates new check-up statistical rules and geometrical constraints that can effectively extract and analyze massive simulations of unregistered point clouds and their corresponding registrations.More than 6×10^(7) sets of the registrations were simulated,whereas more than IOO registrations with real data were used to verify the results of simulation.The results indicated that using five spherical targets is the best choice for the registration of a large typical registration site consisting of two vertical facades and a ground,when there is only a box set of spherical targets available.As a result,the users can avoid placing extra targets to achieve insignificant improvements in registration accuracy.The results also suggest that the higher registration accuracy can be obtained when the ratio between the facade-to-target distance and target-to-scanner distance is approximately 3:2.Therefore,the targets should be placed closer to the scanner rather than in the middle between the facades and the scanner,contradicting to the traditional thought. Besides,the results reveal that the accuracy can be increased by setting the largest projected triangular area of the targets to be large.

Research Progress and Suggestions on China’s Drug Registration Management Based on CiteSpace Knowledge Maps

Objective To analyze the research status and hot spots in the field of drug registration in China,and to provide some suggestions for the follow-up research.Methods CiteSpace was used to conduct literature quantitative analysis on 684 related articles from 2012 to 2022,and the knowledge map was drawn.Based on this,the main characteristics and development trends of the related studies were summarized.Results and Conclusion The number of articles published was closely related to the regulatory policy of drug registration reform.The authors of these articles did not have good continuity.Besides,research hot spots were closely related to the actual work,which was mainly around the improvement of the review and approval policy,encouraging innovative drug research and development,improving the level of new drug development and other directions.The follow-up studies should further strengthen the continuity of research and inter-agency collaboration.In addition,biomedical registration may become a new research focus in the future.

Natural forest ALS-TLS point cloud data registration without control points

Airborne laser scanning(ALS)and terrestrial laser scanning(TLS)has attracted attention due to their forest parameter investigation and research applications.ALS is limited to obtaining fi ne structure information below the forest canopy due to the occlusion of trees in natural forests.In contrast,TLS is unable to gather fi ne structure information about the upper canopy.To address the problem of incomplete acquisition of natural forest point cloud data by ALS and TLS on a single platform,this study proposes data registration without control points.The ALS and TLS original data were cropped according to sample plot size,and the ALS point cloud data was converted into relative coordinates with the center of the cropped data as the origin.The same feature point pairs of the ALS and TLS point cloud data were then selected to register the point cloud data.The initial registered point cloud data was fi nely and optimally registered via the iterative closest point(ICP)algorithm.The results show that the proposed method achieved highprecision registration of ALS and TLS point cloud data from two natural forest plots of Pinus yunnanensis Franch.and Picea asperata Mast.which included diff erent species and environments.An average registration accuracy of 0.06 m and 0.09 m were obtained for P.yunnanensis and P.asperata,respectively.

Disordered Multi-view Registration Method Based on the Soft Trimmed Deep Network

Compared with the pair-wise registration of point clouds,multi-view point cloud registration is much less studied.In this dissertation,a disordered multi-view point cloud registration method based on the soft trimmed deep network is proposed.In this method,firstly,the expression ability of feature extraction module is improved and the registration accuracy is increased by enhancing feature extraction network with the point pair feature.Secondly,neighborhood and angle similarities are used to measure the consistency of candidate points to surrounding neighborhoods.By combining distance consistency and high dimensional feature consistency,our network introduces the confidence estimation module of registration,so the point cloud trimmed problem can be converted to candidate for the degree of confidence estimation problem,achieving the pair-wise registration of partially overlapping point clouds.Thirdly,the results from pair-wise registration are fed into the model fusion to achieve the rough registration of multi-view point clouds.Finally,the hierarchical clustering is used to iteratively optimize the clustering center model by gradually increasing the number of clustering categories and performing clustering and registration alternately.This method achieves rough point cloud registration quickly in the early stage,improves the accuracy of multi-view point cloud registration in the later stage,and makes full use of global information to achieve robust and accurate multi-view registration without initial value.

Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

BACKGROUND It has been confirmed that three-dimensional(3D)imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography(ERCP),which reduces the radiation dose and procedure time with improved safety.However,current 3D biliary imaging does not have good real-time fusion with intraoperative imaging,a process meant to overcome the influence of intraoperative respiratory motion and guide navigation.The present study explored the feasibility of real-time continuous image-guided ERCP.AIM To explore the feasibility of real-time continuous image-guided ERCP.METHODS We selected 23D-printed abdominal biliary tract models with different structures to simulate different patients.The ERCP environment was simulated for the biliary phantom experiment to create a navigation system,which was further tested in patients.In addition,based on the estimation of the patient’s respiratory motion,preoperative 3D biliary imaging from computed tomography of 18 patients with cholelithiasis was registered and fused in real-time with 2D fluoroscopic sequence generated by the C-arm unit during ERCP.RESULTS Continuous image-guided ERCP was applied in the biliary phantom with a registration error of 0.46 mm±0.13 mm and a tracking error of 0.64 mm±0.24mm.After estimating the respiratory motion,3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray image sequence in real-time in 18 patients,with an average fusion rate of 88%.CONCLUSION Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

Free form deformation and symmetry constraint‐based multimodal brain image registration using generative adversarial nets

Multi‐modal brain image registration has been widely applied to functional localisation,neurosurgery and computational anatomy.The existing registration methods based on the dense deformation fields involve too many parameters,which is not conducive to the exploration of correct spatial correspondence between the float and reference images.Meanwhile,the unidirectional registration may involve the deformation folding,which will result in the change of topology during registration.To address these issues,this work has presented an unsupervised image registration method using the free form deformation(FFD)and the symmetry constraint‐based generative adversarial networks(FSGAN).The FSGAN utilises the principle component analysis network‐based structural representations of the reference and float images as the inputs and uses the generator to learn the FFD model parameters,thereby producing two deformation fields.Meanwhile,the FSGAN uses two discriminators to decide whether the bilateral registration have been realised simultaneously.Besides,the symmetry constraint is utilised to construct the loss function,thereby avoiding the deformation folding.Experiments on BrainWeb,high grade gliomas,IXI and LPBA40 show that compared with state‐of‐the‐art methods,the FSGAN provides superior performance in terms of visual comparisons and such quantitative indexes as dice value,target registration error and computational efficiency.

Automatic Extraction of the Sparse Prior Correspondences for Non-Rigid Point Cloud Registration

Non-rigid registration of point clouds is still far from stable,especially for the largely deformed one.Sparse initial correspondences are often adopted to facilitate the process.However,there are few studies on how to build them automatically.Therefore,in this paper,we propose a robust method to compute such priors automatically,where a global and local combined strategy is adopted.These priors in different degrees of deformation are obtained by the locally geometrical-consistent point matches from the globally structural-consistent region correspondences.To further utilize the matches,this paper also proposes a novel registration method based on the Coherent Point Drift framework.This method takes both the spatial proximity and local structural consistency of the priors as supervision of the registration process and thus obtains a robust alignment for clouds with significantly different deformations.Qualitative and quantitative experiments demonstrate the advantages of the proposed method.

Application of Opening and Closing Morphology in Deep Learning-Based Brain Image Registration

In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.

Automated Registration for Infrared Image Based on Wavelet Analysis

To develop a quick, accurate and antinoise automated image registration technique for infrared images, the wavelet analysis technique was used to extract the feature points in two images followed by the compensation for input image with angle difference between them. A hi erarchical feature matching algorithm was adopted to get the final transform parameters between the two images. The simulation results for two infrared images show that the method can effectively, quickly and accurately register images and be antinoise to some extent.

Sensor Registration in Asynchronous Data Fusion

To find an effective method to estimate and remove the registration error in asynchronous multisensor system, Kalman filtering technique and least squares approach have been proposed to estimate and remove sensor bias and sensor frame tilt errors in multisensor systems with asynchronous data. Simulation results is presented to demonstrate the performance of these approaches. The least squares approach can compress measurements to any time. The Kalman filter algorithm can detect registration errors and use the information to converge tracks from independent sensors. This is particularly important if the data from the sensors are to be fused.

Kalman Fliter Approach to Multisensor Registration of Moving Platform

Aim To find an effective method to remove the registration error in multi-sensor systems. Methods A Kalman filtering technique was proposed to estimate and remove sensor bias and sensor fare tilt errors in multisensor systems with a moving platform. Results Simulation results are presented to demonstrate the performance of the approach. Conclusion The Kalman filter algorithm am detect registration errors and use this information to converge tracks from independent sensors. This is particularly important if the data from the sensors are to fused.

Multimodal Medical Image Registration and Fusion for Quality Enhancement

For the last two decades,physicians and clinical experts have used a single imaging modality to identify the normal and abnormal structure of the human body.However,most of the time,medical experts are unable to accurately analyze and examine the information from a single imaging modality due to the limited information.To overcome this problem,a multimodal approach is adopted to increase the qualitative and quantitative medical information which helps the doctors to easily diagnose diseases in their early stages.In the proposed method,a Multi-resolution Rigid Registration(MRR)technique is used for multimodal image registration while Discrete Wavelet Transform(DWT)along with Principal Component Averaging(PCAv)is utilized for image fusion.The proposed MRR method provides more accurate results as compared with Single Rigid Registration(SRR),while the proposed DWT-PCAv fusion process adds-on more constructive information with less computational time.The proposed method is tested on CT and MRI brain imaging modalities of the HARVARD dataset.The fusion results of the proposed method are compared with the existing fusion techniques.The quality assessment metrics such as Mutual Information(MI),Normalize Crosscorrelation(NCC)and Feature Mutual Information(FMI)are computed for statistical comparison of the proposed method.The proposed methodology provides more accurate results,better image quality and valuable information for medical diagnoses.

Projection registration of X-ray image and CT image

A methodology for alignment of an X-ray image and a CT image, based on the Chamfer 3-4 distance transform and simulated annealing optimization algorithm is presented. Firstly, an initial transformation matrix is constructed. For the convenience of computing, geometric models of the X-ray device to reconstruct the calibration matrix are used. Then, by defining the distance between the 3-D protective and the 2-D object image, we optimize this distance matching problem, using the simulated annealing algorithm. This method is also integrated into medical intra-operation, dealing with the data set acquired from 3-D image workstation and active navigation.

IMAGE PANORAMIC MOS AICING WITH GLOBAL AND LOCAL REGISTRATION

Technique s for constructing full view panoramic mosaics from sequences of images are pres ented. The goal of this work is to remove too many limitations for pure panning motion. The best reference block is important for the block-matching method for improving the robustness and performance. It is automatically selected in the h igh-frequency image, which always contains the plenty visible features. In orde r to reduce accumulated registration errors, the global registration using the p hase-correlation matching method with rotation adjustment is applied to the who le sequence of images, which results in an optimal image mosaic with resolving t ranslational or rotational motion. The local registration using the Levenberg-M arquardt iterative non-linear minimization algorithm is applied to compensating for small amounts of motion parallax introduced by translations of the camera a nd other unmodeled distortions, then minimizing the discrepancy after applying t he global registration. The accumulated misregistration errors may cause a visib le gap between the two images. A smoothing filter is introduced for removing the visible artifact.

Novel registration algorithm for 3-D images captured from multiple views of object surface

A novel algorithm of 3-D surface image registration is proposed. It makes use of the array information of 3-D points and takes vector/vertex-like features as the basis of the matching. That array information of 3-D points can be easily obtained when capturing original 3-D images. The iterative least-mean-squared （LMS） algorithm is applied to optimizing adaptively the transformation matrix parameters. These can effectively improve the registration performance and hurry up the matching process. Experimental results show that it can reach a good subjective impression on aligned 3-D images. Although the algorithm focuses primarily on the human head model, it can also be used for other objects with small modifications.

A method based on mutual information and gradient information for medical image registration

Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.

Sanitary Regulation of Pharmaceutical Products in Mexico （Sanitary Registration-Board Arrangement-Patent）-Monoclonal Antibody Case Study

The Federal Commission for Protection against Health Risks in Mexico （COFEPRIS） is responsible for the regulation and health promotion of the production, commercialization, import, export, publicity or involuntary exposure of medicines and technologies for health. For its part, the Mexican Institute of Industrial Property （IMPI） is a decentralized public body with legal personality, own property and legal authority to administer the industrial property system in our country. In the year 2000, on September 19, the official journal published the linking of patents related to allopathic medicines and the sanitary registry. Objectives： To publicize the process of linking the Sanitary-Patent Registry as a case study to delimit the scope of the new pharmaceutical products. Methodology： Request the information to COFEPRIS for the registration of a new pharmaceutical product; Establish flow diagrams for the registration of： innovative products, generic products, biotech products and bio similar products; Integrate the documents in the dossier of the pharmaceutical product for sanitary registration; Elaborate sworn letter of non-invasion of patents and study of patents; Submit the Health Registry; Intergovernmental consultation. Sanitary Registry Results： Search of patents in databases to elaborate state of the art, to write letter under protest to say truth that is not invading any patent, when developing a composition of cetuximab, as generic product since the patent molecule expired in 2016 and 11 patents related to compositions, preparations and finished product preparation processes. Conclusions： COFEPRIS will determine through intergovernmental consultation with the IMPI whether or not a patent is invaded, according to the case it will grant the registration or send the corresponding prevention for cetuximab, in case that invading can also be a refusal of grant.