Image Fusion Real-time System Based on FPGA and Multi-DSP

In order to solve complex algorithm that is difficult to achieve real-time processing of Multiband image fusion within large amount of data, a real-time image fusion system based on FPGA and multi-DSP is designed. Five-band image acquisition, image registration, image fusion and display output can be done within the system which uses FPGA as the main processor and the other three DSP as an algorithm processor. Making full use of Flexible and high-speed characteristics of FPGA, while an image fusion algorithm based on multi-wavelet transform is optimized and applied to the system. The final experimental results show that the frame rate of 15 Hz, with a resolution of 1392 × 1040 of the five-band image can be used by the system to complete processing within 41ms.

Production of High-Resolution Remote Sensing Images for Navigation Information Infrastructures

This paper introduces the image fusion approach of multi-resolutionanalysis-based intensity modulation (MRAIM) to produce the high-resolution multi-spectral imagesfrom high-resolution panchromatic image and low-resolution multi-spectral images for navigationinformation infrastructure. The mathematical model of image fusion is derived according to theprinciple of remote sensing image formation. It shows that the pixel values of a high-resolutionmulti-spectral images are determined by the pixel values of the approximation of a high-resolutionpanchromatic image at the resolution level of low-resolution multi-spectral images, and in the pixelvalae computation the M-band wavelet theory and the a trous algorithm are then used. In order toevaluate the MRAIM approach, an experiment has been carried out on the basis of the IKONOS 1 mpanchromatic image and 4 m multi-spectral images. The result demonstrates that MRAIM image fusionapproach gives promising fusion results and it can be used to produce the high-resolution remotesensing images required for navigation information infrastructures.

Application of Dual-Energy X-Ray Image Detection of Dangerous Goods Based on YOLOv7

X-ray security equipment is currently a more commonly used dangerous goods detection tool, due to the increasing security work tasks, the use of target detection technology to assist security personnel to carry out work has become an inevitable trend. With the development of deep learning, object detection technology is becoming more and more mature, and object detection framework based on convolutional neural networks has been widely used in industrial, medical and military fields. In order to improve the efficiency of security staff, reduce the risk of dangerous goods missed detection. Based on the data collected in X-ray security equipment, this paper uses a method of inserting dangerous goods into an empty package to balance all kinds of dangerous goods data and expand the data set. The high-low energy images are combined using the high-low energy feature fusion method. Finally, the dangerous goods target detection technology based on the YOLOv7 model is used for model training. After the introduction of the above method, the detection accuracy is improved by 6% compared with the direct use of the original data set for detection, and the speed is 93FPS, which can meet the requirements of the online security system, greatly improve the work efficiency of security personnel, and eliminate the security risks caused by missed detection.

Research on Windowing Image Fusion

The speed and quality of the image fusion always restrain each other.The real-time image fusion is one of the problems which needs to be studied and solved urgently.The windowing processing technology for the image fusion proposed in this paper can solve this problem in a certain extent.The windowing rules were put forward and the applicable scope for the windowing fusion and the calculation method for the maximum windowing area were determined.And,the results of the windowing fusion were analyzed,verified and compared to confirm the feasibility of this technology.

Real-time virtual sonography visualization and its clinical application in biliopancreatic disease

AIM:To evaluate the usefulness of real-time virtual sonography(RVS)in biliary and pancreatic diseases.METHODS:This study included 15 patients with biliary and pancreatic diseases.RVS can be used to observe an ultrasound image in real time by merging the ultrasound image with a multiplanar reconstruction computed tomography(CT)image,using pre-scanned CT volume data.The ultrasound used was EUB-8500with a convex probe EUP-C514.The RVS images were evaluated based on 3 levels,namely,excellent,good and poor,by the displacement in position.RESULTS:By combining the objectivity of CT with free scanning using RVS,it was possible to easily interpret the relationship between lesions and the surrounding organs as well as the position of vascular structures.The resulting evaluation levels of the RVS images were12 excellent(pancreatic cancer,bile duct cancer,cholecystolithiasis and cholangiocellular carcinoma)and 3 good(pancreatic cancer and gallbladder cancer).Compared with conventional B-mode ultrasonography and CT,RVS images achieved a rate of 80%superior visualization and 20%better visualization.CONCLUSION:RVS has potential usefulness in objective visualization and diagnosis in the field of biliary and pancreatic diseases.

RRVPE:A Robust and Real-Time Visual-Inertial-GNSS Pose Estimator for Aerial Robot Navigation

Self-localization and orientation estimation are the essential capabilities for mobile robot navigation.In this article,a robust and real-time visual-inertial-GNSS(Global Navigation Satellite System)tightly coupled pose estimation(RRVPE)method for aerial robot navigation is presented.The aerial robot carries a front-facing stereo camera for self-localization and an RGB-D camera to generate 3D voxel map.Ulteriorly,a GNSS receiver is used to continuously provide pseudorange,Doppler frequency shift and universal time coordinated(UTC)pulse signals to the pose estimator.The proposed system leverages the Kanade Lucas algorithm to track Shi-Tomasi features in each video frame,and the local factor graph solution process is bounded in a circumscribed container,which can immensely abandon the computational complexity in nonlinear optimization procedure.The proposed robot pose estimator can achieve camera-rate(30 Hz)performance on the aerial robot companion computer.We thoroughly experimented the RRVPE system in both simulated and practical circumstances,and the results demonstrate dramatic advantages over the state-of-the-art robot pose estimators.

基于轨迹引导的移动机器人导航策略优化算法

针对在杂乱、障碍物密集的复杂环境下移动机器人使用深度强化学习进行自主导航所面临的探索困难,进而导致学习效率低下的问题,提出了一种基于轨迹引导的导航策略优化(TGNPO)算法。首先,使用模仿学习的方法为移动机器人训练一个能够同时提供专家示范行为与导航轨迹预测功能的专家策略,旨在全面指导深度强化学习训练;其次,将专家策略预测的导航轨迹与当前时刻移动机器人所感知的实时图像进行融合,并结合坐标注意力机制提取对移动机器人未来导航起引导作用的特征区域,提高导航模型的学习性能;最后,使用专家策略预测的导航轨迹对移动机器人的策略轨迹进行约束,降低导航过程中的无效探索和错误决策。通过在仿真和物理平台上部署所提算法,实验结果表明,相较于现有的先进方法,所提算法在导航的学习效率和轨迹平滑方面取得了显著的优势。这充分证明了该算法能够高效、安全地执行机器人导航任务。

超声图像融合及导航技术在临床的应用与进展

超声图像融合及导航技术是结合超声图像与其他影像图像的信息优势,用超声检查显示实时的病灶融合影像,从而在超声实时引导下对病灶进行治疗。该技术现已在临床上广泛应用,使病变的检出率与治疗率得到进一步提高,同时能更准确地评估疗效。本文对超声图像融合的相关技术及其在临床中的应用进展进行综述。

基于视觉补充的水稻插秧机多传感器组合定位研究

为改善基于GNSS/INS组合定位的水稻插秧机在遇到遮挡、电磁干扰、传感器失效等情况时的导航效果,在原有GNSS/INS组合定位的基础上,提出一种视觉导航系统(vision navigation system,VNS)补充的水稻插秧机多传感器组合定位方法。首先设计改进的Otsu法和改进的Hough变换算法用于视觉定位信息提取,并构建插秧机和相机坐标系关系方程以求解位姿值;然后采用具有容错功能的联邦卡尔曼滤波算法将VNS输出的定位信息和GNSS、INS输出的定位信息进行融合;最后分别在水泥地和水田进行试验。结果显示,空旷水泥地场景下,GNSS/INS/VNS组合定位和GNSS/INS组合定位精度相近,而在遮挡水泥地场景下,GNSS/INS/VNS组合定位解算出的位置误差和航向误差的平均值分别为1.77 cm和0.99°,相较于GNSS/INS组合定位方法分别提高46.8%和61.5%;水田试验中,经过视觉补充后导航系统的横向偏差和航向偏差平均值分别降低45.7%和67.9%,横向偏差平均值为1.97cm,航向偏差平均值为0.49°。试验结果表明,基于视觉补充的多传感器组合定位方法能有效降低导航系统的定位误差和跟踪偏差,满足插秧机自动驾驶作业的要求。

激光导航与决策融合的小样本无人机航拍图像分类

近年来,激光导航技术与小样本无人机航拍图像分类,为土地利用调查、城市规划和环境监测等领域提供了精确的空间定位与大量价值信息,显著提升了分类技术的水平。本文提出了一种激光导航与决策融合技术的小样本无人机航拍图像分类方法,旨在提高分类性能与空间定位精度。通过激光导航系统提供的高精度地理位置信息,优化了航拍图像的特征提取过程,采用自监督学习构建辅助任务,通过旋转和翻转技术增强特征提取器的泛化能力。此外,结合两种自监督范式训练得到的特征提取器,通过逻辑回归分类器完成分类任务,设计了一种新型的决策融合模块,以自动调整各决策权重,提高了分类准确性。通过NWPU-RESISC45和UC Merced数据集上进行试验,结果验证了本文方法的有效性和先进性,展现了激光导航技术在提高小样本无人机航拍图像分类中的潜力。

3D超声融合成像导航技术联合实时超声造影在原发性小肝癌微波消融术中的应用研究

目的探讨3D超声融合成像导航系统(3DUIFNS)联合实时超声造影(CUES)在原发性小肝癌微波消融术中的应用价值,进一步明确预后独立影响因素,分析不同指导技术对预后的影响。方法前瞻性选取2020年2月至2021年2月哈尔滨市第二医院收治的原发性小肝癌患者104例作为研究对象,均择期行微波消融术。按照随机抽签法将患者分成CUES组、3DUIFNS+CEUS组,每组各52例。比较两组一次完全消融率、补充消融率及并发症情况。术后随访2年,记录2年无瘤生存与累积生存情况,经Kaplan-Meier法与Log-Rank检验进行生存分析,并采用Cox回归模型分析预后独立影响因素。结果3DUIFNS+CEUS组一次完全消融率为88.46%,高于CUES组(71.15%),补充消融率为11.54%,低于CUES组(28.85%),差异有统计学意义(P<0.05)。两组腹腔积液、胸腔积液、腹腔出血、血气胸、肝功能损害、肝区疼痛发生率比较,差异均无统计学意义(P>0.05)。3DUIFNS+CEUS组平均无瘤生存期22个月,无瘤生存率92.00%;CUES组平均无瘤生存期20个月,无瘤生存率76.47%。3DUIFNS+CEUS组无瘤生存期长于CUES组,差异有统计学意义(P<0.05)。3DUIFNS+CEUS组平均总生存期23个月,总生存率94.00%;CUES组平均总生存期22个月,总生存率80.39%。3DUIFNS+CEUS组平均总生存期长于CUES组,差异有统计学意义(P<0.05)。Cox回归分析提示,分化程度、肿瘤转移、大血管受累、肿瘤包膜、消融效果、手术指导方式是原发性小肝癌患者无瘤生存期、总生存期的独立影响因素(P<0.05)。结论3DUIFNS+CEUS能提高原发性小肝癌患者微波消融术的一次完全消融率,且会影响远期预后,可提升无瘤生存率、总生存率,此外,患者远期预后还受肿瘤分化程度、肿瘤转移、大血管受累、肿瘤包膜、消融效果的影响,临床需引起重视。

基于GNSS/INS视觉融合智能车导航控制策略研究

为了提高智能导航车辆在复杂环境下的控制精度,提出一种基于GNSS/INS视觉融合处理算法,设计以英飞凌TC364单片机为控制核心单元的智能导航小车。通过对采集到的GNSS、IMU以及灰度图像数据进行预先分析,并对数据进行卡尔曼滤波融合,灰度图像处理边界提取,拟合出符合要求的车辆运行轨迹后,采用纯跟踪算法对车辆进行实时跟踪控制,验证算法可行后移植到智能车。通过实验数据以及实际运行结果分析对比,证明该方案满足车辆的自动巡航要求。

Navigated liver surgery:State of the art and future perspectives

Background:In recent years,the development of digital imaging technology has had a significant influence in liver surgery.The ability to obtain a 3-dimensional(3D)visualization of the liver anatomy has provided surgery with virtual reality of simulation 3D computer models,3D printing models and more recently holograms and augmented reality(when virtual reality knowledge is superimposed onto reality).In addition,the utilization of real-time fluorescent imaging techniques based on indocyanine green(ICG)uptake allows clinicians to precisely delineate the liver anatomy and/or tumors within the parenchyma,applying the knowledge obtained preoperatively through digital imaging.The combination of both has transformed the abstract thinking until now based on 2D imaging into a 3D preoperative conception(virtual reality),enhanced with real-time visualization of the fluorescent liver structures,effectively facilitating intraoperative navigated liver surgery(augmented reality).Data sources:A literature search was performed from inception until January 2021 in MEDLINE(Pub Med),Embase,Cochrane library and database for systematic reviews(CDSR),Google Scholar,and National Institute for Health and Clinical Excellence(NICE)databases.Results:Fifty-one pertinent articles were retrieved and included.The different types of digital imaging technologies and the real-time navigated liver surgery were estimated and compared.Conclusions:ICG fluorescent imaging techniques can contribute essentially to the real-time definition of liver segments;as a result,precise hepatic resection can be guided by the presence of fluorescence.Furthermore,3D models can help essentially to further advancing of precision in hepatic surgery by permitting estimation of liver volume and functional liver remnant,delineation of resection lines along the liver segments and evaluation of tumor margins.In liver transplantation and especially in living donor liver transplantation(LDLT),3D printed models of the donor’s liver and models of the recipient’s hilar anatomy can contribute further to improving the results.In particular,pediatric LDLT abdominal cavity models can help to manage the largest challenge of this procedure,namely large-for-size syndrome.

A Fractal Dimension Based Framework for Night Vision Fusion

In this paper, a novel fusion framework is proposed for night-vision applications such as pedestrian recognition,vehicle navigation and surveillance. The underlying concept is to combine low-light visible and infrared imagery into a single output to enhance visual perception. The proposed framework is computationally simple since it is only realized in the spatial domain. The core idea is to obtain an initial fused image by averaging all the source images. The initial fused image is then enhanced by selecting the most salient features guided from the root mean square error(RMSE) and fractal dimension of the visual and infrared images to obtain the final fused image.Extensive experiments on different scene imaginary demonstrate that it is consistently superior to the conventional image fusion methods in terms of visual and quantitative evaluations.

A DR-WFOI fusion system for the real-time molecular imaging in vivo

Digital radiography （DR） and whole-body fluorescent optical imaging （WFOI） have been widely applied in the field of molecular imaging, with the advantages in tissues and functional imaging. The integration of them contributes to the development and discovery of medicine. We introduce an equipment, performance of which is better than that of another molecular imaging system manufactured by Kodak Corp. It can take real-time small animal imaging in vivo, with lower cost and shorter development cycle on the LabVIEW platform. At last, a paradigm experiment on a nude mouse with green fluorescent protein （GFP） transgenic tumor is given to present a real-time DR-WFOI fusion simultaneous image.

基于倾向评分匹配法评估影像融合导航技术引导经皮穿刺活检在肺深部微小癌变检测中的应用价值

目的基于倾向评分匹配法研究影像融合导航技术引导经皮穿刺活检在肺深部微小癌变检测中应用的价值。方法选取2018年4月至2021年4月河北省胸科医院收治的影像融合导航技术引导经皮穿刺肺活检术患者105例为观察组,选择同期本院常规CT引导下经皮穿刺肺活检术患者223例为对照组。分析两种方法对肺深部微小癌变的诊断准确性及患者并发症发生情况的差异。结果采用1∶1最邻近匹配法,倾向评分匹配法匹配后,观察组90例,对照组90例。与对照组比较,观察组操作时间、探针次数减少(P<0.05),针刺深度增加(P<0.05),气胸、少量出血并发症发生率、总并发症发生率降低(P<0.05)。与对照组比较,观察组诊断腺癌、鳞癌、小细胞癌的准确性均提高(P<0.05)。结论影像融合导航技术引导经皮穿刺活检对肺深部微小癌变的诊断具有较高的准确性,且具有操作时间短、辐射剂量小、并发症发生率低等特点,值得临床推广应用。

实时影像融合介入导航系统在小肝癌微波消融中应用评价

目的评价实时影像融合介入导航系统在小肝癌微波消融中的应用。方法选择2019年12月至2021年12月在邢台市人民医院接受实时影像融合介入导航系统辅助微波消融治疗的64例小肝癌患者作为研究组,同时段未接受实时影像融合介入导航系统辅助的52例小肝癌患者作为对照组。研究组在微波消融前分别采用常规超声、实时影像融合介入导航系统、增强CT/MRI扫描病灶,对照组仅行常规超声检查,计算检出率。微波消融手术在实时影像融合介入导航系统导引下进行。术后1个月行常规超声、实时影像融合介入导航系统、增强CT/MRI检查,计算各项影像学技术的特异度、灵敏度与准确度。术后1年评估两组疗效。结果实时影像融合介入导航系统对病灶最长径的测量值大于常规超声(t=3.887,P<0.01),与增强CT/MRI的测量值差异无统计学意义(t=0.119,P=0.905);常规超声、实时影像融合介入导航系统检出率分别为95.83%、100%。实时影像融合导入系统与增强CT/MRI对小肝癌疗效的评估呈良好一致性(Kappa=0.529,P<0.01)。术后1年,研究组客观缓解率(ORR)、疾病控制率(DCR)均高于对照组(P<0.05)。结论实时影像融合介入导航系统在小肝癌微波消融治疗中具有重要作用,可精准评估病灶位置、数量及大小,用于术后随访的准确度、灵敏度及特异度均较高。

船舶航行态势融合算法及平台开发

为使船舶航行操控人员或自动驾驶系统能够获得更为全面和精准的航行环境信息,提出一种融合雷达、船舶自动识别系统(automatic identification system,AIS)、可见光多源信息的目标融合算法,并开发可应用于实船的软件平台。首先,设计多源目标融合算法,根据同一性原理将雷达捕获目标与AIS解析目标融合,根据水平面坐标与垂直面坐标变换原理融合可见光目标;其次,设计船舶多源目标融合数据样本集制作原理,利用实船采集数据构建包含3种来源的数据样本,并开发多源目标识别算法;再次,设计船舶航行多源信息融合平台,开发可应用于实船的可视化人机交互界面;最后,利用实船开展算法验证和平台试验,验证融合算法的准确性和软件平台的有效性。多目标融合算法可实现3种来源目标的准确融合,融合软件平台可提供较为精准的可视化信息。

MRI融合导航经颈静脉肝内门体分流术的临床优势和经验

目的 探讨MRI融合导航技术在经颈静脉肝内门体分流术(TIPS)中的应用价值及其在食管胃底静脉栓塞的技术优势。方法 选取2020年12月至2021年3月在珠海市人民医院治疗的6例需行TIPS手术患者。术前行肝脏容积加速采集成像(LAVA)序列腹部MRI增强扫描,将MRI增强图像与术中DSA图像匹配并2次校准后的融合图像叠加至术中透视影像,作为由肝静脉至门静脉穿刺的路径指引,同时在MRI融合导航下不使用对比剂完成胃底静脉插管,根据静脉曲张情况行栓塞治疗。结果 6例导引手术全部成功,技术成功率100%。2例采用气管插管麻醉,4例局部麻醉。术中肝静脉和门静脉穿刺为(2.5±1.7)次,穿刺时间(开始穿刺至导丝通过)为(211.7±180.6)s,手术时长为(117.5±53.5)min,X射线累积剂量为(1 010.3±833.8)mGy,对比剂使用量为(122.5±25.9)mL,均未出现严重并发症。结论 MRI融合导航技术辅助TIPS手术安全可行,融合导航下可快速完成食管胃底静脉插管,技术优势明显,值得临床推广。

基于小波变换的图像融合研究

在导航手术过程中,图像引导起着关键作用,直接影响导航精度。CT图像能清晰显示颅骨和粘贴的标记点,却对脑组织的显示有限。MRI图像对脑内部软组织结构分辨高,但对头颅表面信息显示模糊。因此单一模态的图像对于导航手术引导有限。本文提出基于小波变化的图像融合方法,对CT和MRI图像进行了价值信息的有效融合,弥补了单幅图像信息不足的缺点。实验结果表明,该图像融合方法具有较强的鲁棒性,能够满足临床应用。