基于双判别器生成对抗网络的PET和MRI图像融合

PET and MRI Image Fusion Based on Generative Adversarial Network with Dual Discriminators

正电子发射断层扫描(positron emission tomography,PET)是一种功能型成像技术,PET图像能够反映人体丰富的组织信息,但其分辨率较低。磁共振成像(magnetic resonance imaging,MRI)是一种结构型成像技术,MRI图像能够反映人体的解剖结构信息,且分辨率较高。将PET与MRI进行有效融合,同时保留二者所携带的有用信息,能够更好地应用于临床疾病的诊断。为此提出了一种双判别器生成对抗网络(generative adversarial network,GAN)的PET与MRI图像融合方法,具体为:①将PET图像由RGB彩色空间转换为YCbCr彩色空间;②将PET图像的Y分量与MRI图像进行“联结”,然后输入到密集连接与源图像插入的生成器网络中,得到融合结果,即融合的PET图像的Y分量;③将融合的PET图像的Y分量与源图像(PET图像的Y分量和MRI图像)分别输入到两个判别器中,与生成器进行对抗训练;④对融合得到的Y分量,连同PET图像的Cb和Cr分量进行YCbCr到RGB的反变换,从而得到最终的融合图像。实验结果表明,相较于其他几种基于GAN的融合方法,所提出的算法的融合结果在主观判读与客观定量评价中都取得了更优的结果。

Positron emission tomography(PET)is a type of functional imaging,and can reflect the rich tissue information of the human body but lacks high resolution.Magnetic resonance imaging(MRI)is a kind of structural imaging,and can reflect the anatomical structure information of human body with high resolution.In practice,for the purpose of fully utilizing the useful information coming both from the PET and from MRI images,some techniques are adopted to effectively integrate the PET and MRI image together.As a result,the fused images can be fully employed in the clinical diagnosis.To this end,this paper proposes a method for fusing PET and MRI images based on generative adversarial network(GAN)with dual discriminators.Specifically,1)convert the PET image from RGB color space to YCbCr color space;2)concatenate the Y component of the PET image with the MRI image,and then input the concatenated image into generator with dense connection and the fashion of source images’inserting,then correspondingly obtain the fusion result,i.e.,fused Y component;3)input the fusion result and source images(Y component of PET and MRI images)into two discriminators,respectively,and conduct adversarial training with the generator;4)inversely transform the fused Y component together with the Cb and Cr components of PET from YCbCr color space to RGB color space,and then obtain the final fusion image.Experimental results show that compared with several other GAN-based fusion methods,the proposed method gets better results both in subjectively visual interpretation and objectively quantitative evaluation.