Survey of methods and principles in three-dimensional reconstruction from two-dimensional medical images

Three-dimensional(3D)reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units.In the coming years,most patient care will shift toward this new paradigm.However,development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved,most of which are dependent on human expertise.In this review,a survey of pre-processing steps was conducted,and reconstruction techniques for several organs in medical diagnosis were studied.Various methods and principles related to 3D reconstruction were highlighted.The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.

Three-dimensional reconstruction of human spinal cord based on histological serial sections

BACKGROUND： It is not possible to reconstruct the inner structure of the spinal cord, such as gray matter and spinal tracts, from the Visual Human Project database or CT and MRI databases, due to low image resolution and contrast in macrosection images. OBJECTIVE： To explore a semi-automatic computerized three-dimensional （3D） reconstruction of human spinal cord based on histological serial sections, in order to solve issues such as low contrast. DESIGN, TIME AND SETTING： An experimental study combining serial section techniques and 3D reconstruction, performed in the laboratory of Human Anatomy and Histoembryology at the Medical School of Nantong University during January to April 2008. SETTING： Department of Anatomy, Institute of Neurobiology, Jiangsu Province Key Laboratory of Neural Regeneration, Laboratory of Image Engineering. MATERIALS： A human lumbar spinal cord segment from fresh autopsy material of an adult male. METHODS： After 4% paraformaldehyde fixation for three days, serial sections of the lumbar spinal cord were cut on a Leica cryostat and mounted on slides in sequence, with eight sections aligned separately on each slide. All sections were stained with Luxol Fast Blue to reveal myelin sheaths. After gradient dehydration and clearing, the stained slides were coverslipped. Sections were observed and images recorded under a light microscope using a digital camera. Six images were acquired at x25 magnification and automatically stitched into a complete section image. After all serial images were obtained, 96 complete serial images of the human lumbar cord segment were automatically processed with ＂Curves＂, ＂Autocontrast＂, ＂Gray scale 8 bit＂, ＂Invert＂, ＂Image resize to 50%＂ steps using Photoshop 7.0 software. All images were added in order into 3D-DOCTOR 4.0 software as a stack, where serial images were automatically realigned with neighboring images and semi-automatically segmented for white matter and gray matter. Finally, simple surface and volume reconstruction were completed on a personal computer. The reconstructed human lumbar spinal cord segment was interactively observed, cut, and measured. MAIN OUTCOME MEASURES： The reconstructed human lumbar spinal cord segment. RESULTS： Compared with serial images obtained from other image modalities, such as CT, MRI, and macrosections from The Visual Human Project database, the Luxol Fast Blue stained histological serial section images exhibited higher resolution and contrast between gray and white matter. Image processing and 3D reconstruction steps were semi-automatically performed with related software. The 3D reconstructed human lumbar cord segment were observed, cut, and measured on a PC. CONCLUSION： A semi-automatically computerized method, based on histological serial sections, is an effective way to 3D-reconstruct the human spinal cord.

Cosmetic Safety Evaluation Based on In Vitro Three-dimensional Reconstructed Human Epidermis(3D-RHE) Models

Cosmetic safety evaluation employs a series of toxicological tests, on both qualitative and quantitative levels, to assess the potential risks for the daily use of selected cosmetic ingredients and final products. Traditionally, safety evaluation of cosmetics uses animal tests. With the development of in vitro science and the 3R （Reduction, Replacement and Refinement） principle, three-dimensional reconstructed human epidermis （3D-RHE） models have been developed and widely applied in cosmetic safety evaluation. Reconstructed human skin models possess anatomy and metabolism biology similar to real human tissue. This paper reviews the current application of 3D-RHE models in the safety evaluation of skin irritation, eye irritation, phototoxicity and genotoxicity potential of cosmetic ingredients/formulas. The advantages and disadvantages of using skin models are also discussed, and comments and suggestions are given for its future development.

3D Vector Reconstruction of the Typical Cervical Vertebra from Anatomical Sections of Korean Visible Human at the Laboratory of Clinical and Digital Anatomy of Paris Descartes University

Aim: To carry out a 3D vector reconstruction of the typical cervical vertebra from anatomical sections of the “Korean Visible Human” for educational purposes. Material and Methods: The anatomical subject was a 33-year-old Korean man who died of leukemia. He was 164 cm tall and weighed 55 kg. This man donated his body to science. Her body was frozen and cut into several anatomical sections after an MRI and CT scan. These anatomical sections were made using a special saw called a 0.2 mm thick cryomacrotome. Thus 8100 cuts were obtained. Only the sections numbered 940 to 1200 were used for our study. A segmentation by manual contouring of the different parts of the typical cervical vertebra was made using the software Winsurf version 3.5 on a laptop PC running Windows 7 equipped with a Ram of 8 gigas. Results: Our 3D vector model of the typical cervical vertebra is easily manipulated using the Acrobat 3DPDF interface. Each part of the vertebra accessible in a menu can be displayed, hidden or made transparent, and 3D labels are available as well as educational menus for learning anatomy. Conclusion: This original work constitutes a remarkable educational tool for the anatomical study of the typical cervical vertebra and can also be used as a 3D atlas for simulation purposes for training in therapeutic gestures.

Three-Dimensional Prototypes and Evaluation of Their Structure and Form

In order to reveal the appearance of the clothing prototype on the human body,the characteristics of the human body’s structure above the waist section were studied.Based on the experimental data of the fit prototype,three-dimensional prototypes features were comparatively analyzed.And then objectively evaluating the relationship was conducted between the planar structure lines of different prototypes and the human body.The results showed that the prototypes analyzed basically conformed to the size of the human body.However,when they were worn on the human body,there were problems in the structure and forming.The main reason was that the side seam was skewed to different degrees.The results of this study provide reference for many practitioners to choose prototypes.

Human Body Modeling and Posture Simulating Based on 3D Surface Scan Data

This paper presents a new approach for modeling the human body by considering the motion state and the shape of whole body. The body model consists of a skeleton kinematic model and a surface model. The former is used to determine the posture of the body,and the latter is used to generate the body shape according to the given posture. The body surface is reconstructed with multi-segment B-spline surfaces based on the 3D scan data from a real human body.Using only a few joints parameters and the original surface scan data, the various body postures and the shape can be generated easily. The model has a strong potential of being used for ergonomic design,garment design, virtual reality environment, as well as creating human animation, etc.

基于全局一致性网络的参数化人体网格重建

人体网格重建(HMR)在人机交互和虚拟/增强现实等领域有广泛应用。为了进一步提高基于图像的人体网格重建中人体姿势和形状估计的精度,提出了基于混合逆运动学的全局一致性深度卷积神经网络,用于参数化人体网格重建(GloCoNet)。为了增强网络的全局一致性和全局上的长程依赖,该网络在特征提取网络基础上,设计了全局一致性增强器(GCB)模块,它能够增强模型对全局信息的感知能力和表达能力,并且使模型能够自适应地调整不同通道和空间位置的特征图权重。然后引入了多头注意力机制(MHSA)来捕获模型全局上的长程依赖,它可以帮助模型在处理长期依赖时更好地捕捉到关键的关系和模式,并建模全局上下文信息,从而更好的丰富特征子空间的多样性。同时,该网络采用混合逆运动学的方法弥合人体网格估计和3D人体关节点估计之间的差距,最终提升人体3D姿势和形状估计的准确度。实验结果表明,GloCoNet模型在公开的Human 3.6 M数据集上以平均每关节51.3 mm的位置误差(MPJPE)显著优于先前的主流方法。

基于交通事故案例的3岁儿童乘员损伤机制研究

目的通过再现真实交通事故案例探究3岁儿童乘员碰撞损伤机制。方法应用车辆有限元模型和图斯特3岁儿童乘员损伤仿生模型(TUST IBMs 3YO-O),再现CIREN数据库中的交通事故案例,通过综合分析Δv、车辆质量和变形能反求碰撞前车辆的速度,设置碰撞仿真试验再现案例所描述的儿童乘员损伤,分析损伤机制。结果TUST IBMs 3YO-O模型完整地再现出碰撞事故中儿童乘员的损伤。儿童乘员头部运动学响应与生物力学响应表现出差异性,儿童乘员胸腔内部组织器官生物力学响应表明其没有出现损伤,但是胸部3 ms合成加速度达到54 g,超出阈值。结论未来乘员安全评价需要引入生物力学参数。应用高生物逼真度的人体生物力学模型再现交通事故中乘员损伤,不仅更加清晰观察事故中乘员的运动学响应,深度分析碰撞损伤机制,还为数字化评估提供参考依据,也将为儿童乘员保护装置研发和安全法规制定提供参考。

Three-dimensional reconstruction of digitized human liver： based on Chinese Visible Human

Background Comparing with two dimensional （2D） imaging, both in diagnosis and treatment, three dimensional （3D） imaging has many advantages in clinical medicine. 3D reconstruction makes the target easier to identify and reveals the volume and shape of the organ much better than 2D imaging. A 3D digitized visible model of the liver was built to provide anatomical structure for planing of hepatic operation and for realizing accurate simulation of the liver on the computer. Methods Transverse sections of abdomen were chosen from the Chinese Visible Human dataset. And Amira software was selected to segment and reconstruct the structures of the liver. The liver was reconstructed in three-dimensions with both surface and volume rendering reconstruction. Results Accurately segmented images of the main structures of the liver were completed. The reconstructed structures can be displayed singly, in small groups or as a whole and can be continuously rotated in 3D space at different velocities. Conclusions The reconstructed liver is realistic, which demonstrates the natural shape and exact position of liver structures, tt provides an accurate model for the automated segmentation algorithmic study and a digitized anatomical mode of viewing the liver.

轻量化人体和手部网格重建

三维人体网格重建在影视、虚拟现实等下游任务中有广泛应用。然而现有重建方式关注更好的重建精度和纹理表达,也因此更依赖高性能的计算或采集设备,缺乏对低成本、轻量化重建方式的研究。为降低人体重建任务的使用成本和硬件要求,本文提出了一种轻量化的人体和手部网格重建方式,基于参数化模型对身体和手部重建任务进行解耦,针对身体和手部的不同特点分别设计了不同分支网络。身体重建分支和手部重建分支均为编码器-解码器结构。身体重建分支编码器为双阶段编码器,第一阶段通过Litehrnet和Canny算子获得热点图和边缘图,并对图片进行代理表示,第二阶段通过Shufflenet提取全局特征,解码器通过级联低维度多层感知器以概率分布的方式对人体参数进行回归;手部重建分支的编码器以Litehrnet为主干网络获取多分辨率特征分支,通过姿态池化对多分辨率特征分支进行融合得到全局特征,解码器通过深度可分离卷积网络获得手部顶点,并通过MLP对形状进行估计,利用顶点坐标基于逆向拓扑数学求解得到关节旋转参数。与现有方法相比,参数量和计算量显著减少,整体参数量为6.12M,计算量为433M,且具有较好的重建效果,在Human3.6M数据集中平均关节点误差(MPJPE)为86.7 mm,手部重建分支在FreiHand数据集上对齐后平均关节点误差(PA-MPJPE)为10.8 mm。此外该方法完成了在移动设备的推理,在骁龙8Gen3处理器推理速度为79.7 ms(12.5 fps),可以达到实时推理的效果。

基于多监督的三维人体姿势与形状预测

三维人体重建技术指通过图像或视频建立有相应姿势和体型的三维人体模型,其在虚拟现实(VR)、网游、虚拟试衣等方面有着十分广泛的应用前景.其中,参数化的三维人体模型由于参数数量的局限,重建精度较低,缺少细节特征.为了提升参数化三维人体模型的重建精度,增加其脸部与手部细节,提出一种基于多监督的三维人体模型重建方法.该方法结合传统的回归方法和优化方法,利用卷积神经网络回归出参数化人体模型的参数,得到一个较为粗糙的人体模型,将该模型作为初始模板进行拟合和迭代优化,将带有脸部和手部的全身密集关节点信息和轮廓信息作为回归网络的2D监督,同时使用迭代优化后的人体模型作为回归网络的3D监督,最终可由一幅图像获得一个多细节、高精度的参数化三维人体模型.定性分析结果表明,该方法为人体拟合过程提供正确的拟合方向,可有效减少非自然姿势的出现,提高三维人体模型重建的准确度.全身密集关节点监督可为模型增加更多手部与脸部的细节,而轮廓监督可减少重建的人体模型与图像中人体的像素级偏差.定量分析表明,该方法在数据集Human3.6M上的平均逐关节位置误差(MPJPE)为59.9 mm,较经典方法SPIN减少了4.16%,对关节点进行刚性对齐后模型的平均逐关节位置误差(MPJPE-PA)低至38.2 mm,较SPIN减少了7.06%.

Synaptic development of layer V pyramidal neurons in the prenatal human prefrontal neocortex: a Neurolucida-aided Golgi study

The prefrontal neocortex is involved in many high cognitive functions in humans.Deficits in neuronal and neurocircuitry development in this part of the cerebrum have been associated with various neuropsychiatric disorders in adolescents and adults.There are currently little available data regarding prenatal dendrite and spine formation on projecting neurons in the human prefrontal neocortex.Previous studies have demonstrated that Golgi silver staining can identify neurons in the frontal lobe and visual cortex in human embryos.In the present study,five fetal brains,at 19,20,26,35,and 38 gestational weeks,were obtained via the body donation program at Xiangya School of Medicine,Central South University,China.Golgi-stained pyramidal neurons in layer V of Brodmann area 46 in fetuses were quantitatively analyzed using the Neurolucida morphometry system.Results revealed that somal size,total dendritic length,and branching points of these neurons increased from 26 to 38 gestational weeks.There was also a large increase in dendritic spines from 35 to 38 gestational weeks.These findings indicate that,in the human prefrontal neocortex,dendritic growth in layer V pyramidal neurons occurs rapidly during the third trimester of gestation.The use of human fetal brain tissue was approved by the Animal Ethics Committee of Xiangya School of Medicine,Central South University,China(approval No.2011-045)on April 5,2011.

Surface Reconstruction of Scanned Human Body Using Radial Basis Functions and Adaptive Partition of Unity

It is important to reconstruct a continuous surface representation of the point cloud scanned from a human body. In this paper a new implicit surface method is proposed to reconstruct the human body surface from the points based on the combination of radial basis functions （RBFs） and adaptive partition of unity （PoU）. The whole 3D domain of the scanned human body is firstly subdivided into a set of overlapping subdomalns based on the improved octrees. The smooth local surfaces are then computed in the subdomalns based on RBFs. And finally the global human body surface is reconstructed by blending the local surfaces with the adaptive PoU functions. This method is robust for the surface reconstruction of the scanned human body even with large or non-uniform point cloud which has a sharp density variation.

基于参数模型和法线推理的单视图三维人体隐式重建

为了克服单视图三维人体重建中出现的伪影及肢体残缺现象,提出了一种基于参数模型和法线推理的三维人体隐式重建算法(Parametric⁃Model and Normal Inference,PMNI),该方法能够从单一RGB图像重建出包含服装的三维人体。网络的输入仅为一张包含人物全身的RGB图像,首先基于图卷积神经网络预测对应的SMPL参数模型,接着基于条件GAN(Generative Adversarial Networks)网络生成人物的后视图像,并分别从前后视图中提取法线特征,最后将它们作为深度隐式函数的额外参数辅助训练。实验结果表明,相较于传统方法,该方法有效提升了重建结果的整体质量和表面细节。得益于参数体和法线作为先验,该方法也可以很好地处理一些复杂人体姿态。

基于单张图像的人体准确姿势3D重建研究

为了解决单张图像人体重建时存在部分姿势不准确,模型精确度不足的问题,提出了基于单张图像进行人体准确姿势3D重建的方法。首先,提取图像特征,并将特征与SMPL基础模板信息相融合,通过图卷积网络将2D图像映射为3D人体表面;其次,将参数化SMPL模型所表达的3D人体投影到2D平面,通过投影后的3D信息与输入图像的2D信息相匹配,实现体型与姿势的估计;然后,使用生成对抗网络判断模型数据是否来自未配对数据的真实人体网格数据;最后,使用多种损失函数共同训练,生成最终生成器模型。使用Human 3.6M测试时与SMPLify, NBF,Rogez等算法进行了对比,姿势估计准确度有所提升。

影子辅助的三维人体重建

光线投射至地面的人体影子可以为人体重建任务提供约束信息,有助于缓解单目图像中的深度歧义问题和自遮挡问题.为了重建三维人体模型,提出一种基于影子输入和深度神经网络的重建方法.首先搭建2个独立的卷积神经网络模块用于提取图像特征和影子特征,并估计对应的人体模型参数;然后建立一个可导投影算子(differentiable projection operator,DPO)对齐2个网络的输出;最后将DPO构建的损失函数应用于微调神经网络和直接优化输出.在Human3.6M合成影子数据集上的实验结果表明,与基线方法相比,微调神经网络的关节误差指标下降了1.4~4.9 mm;在自采集的一个含真实影子的3人2视角小规模数据集上的定性实验结果表明,直接优化的人体姿态更加接近原始图像.

二维碰撞感知的单张图像参数化人体与服装重构

已有的基于图像的人体与服装重构大都不考虑身体与衣物之间的交互,导致独立重构结果互相穿刺.本文提出一个二维碰撞感知的从单张图像重构人体与服装的优化方法,利用SMPL和TailorNet参数化模型来分别表示人体和衣服的三维形状,进而建立约束能量求解人体形状参数,运动参数和服装细节控制参数.我们的方法在初始化阶段对输入图像进行了语义分割以及二维关节点的估计,并采用human mesh recovery初步估计人体的形状与运动参数作为能量优化的初始值.我们的能量优化由两部分构成:其一是形状与姿态约束,利用图像中人体的关节位置和着装人体区域对三维参数化模型的投影的关节位置和投影区域进行约束,保证重建模型与图像在形状与姿态上的一致性;其二是人体与衣服的碰撞约束,引入重构人体与服装模型的二维投影区域间的误差对人体与衣服进行碰撞约束,以避免相互穿刺,考虑到基于投影的约束对视点敏感,我们在三维空间中进行视点采样,从而建立多视角的二维投影约束.考虑到能量中包含了TailorNet,不容易计算梯度,我们利用爬山法交替地对人体形状,姿态和服装尺寸参数进行优化求取最优解.最后,通过一系列实验对本文方法和最近的一些相关工作进行了定性和定量分析,结果表明本文方法能有效缓解人体与服装穿刺,重构精度也更高。

基于移动设备的三维人体重建

针对三维重建方法计算量大,耗费内存高的问题,提出了一种基于知识蒸馏的人体三维模型重建方法应用于电子商务服装定制,利用Resnet-18构建人体正面、侧面掩膜数据集,分别针对男性、女性、中性三种人体特征提取信息,然后通过利用知识蒸馏的方法进行模型压缩,使其能够部署于移动设备。实验表明,论文方法能够在移动设备上以较快的速度、消耗消耗较少的储存空间,实现三维人体模型的重建。

一种基于模型的人体建模系统

提出了一种基于Poser人体模型库模型的人体曲面建模和模型局部尺寸调节的方法 ,并以此为基础开发出了一套基于模型的人体建模系统。实验证明 ,该系统在得到令人满意的重建视觉效果的前提下 。

基于深度扫描仪的高辨识度三维人体模型重建方法

3D照相打印馆人像的打印质量取决于3D扫描获得的三维人体模型的辨识度。然而,由于现有3D人体扫描仪价格昂贵、操作复杂等原因,使得3D人像打印成本高、耗时长和打印精度较低。针对这些缺点提出一种基于深度扫描仪重建高辨识度三维人体模型方法。利用多组深度扫描仪分工协作、优势互补,分别获取高辨识度的人体面部五官点云数据,上半身与全身表面轮廓点云数据。然后,通过引入特征点和改进的最近点迭代法将采集到的三组点云数据进行对齐、替换、拼接,将拼接后的无拓扑关系的点云数据进行曲面重构即可获得高辨识度的三维人体模型。该方法的扫描时间较短,以较低的成本构建了具有高辨识度的三维人像模型。