Automatic three-dimensional reconstruction based on four-view stereo vision using checkerboard pattern

An automatic three-dimensional(3D) reconstruction method based on four-view stereo vision using checkerboard pattern is presented. Mismatches easily exist in traditional binocular stereo matching due to the repeatable or similar features of binocular images. In order to reduce the probability of mismatching and improve the measure precision, a four-camera measurement system which can add extra matching constraints and offer multiple measurements is applied in this work. Moreover, a series of different checkerboard patterns are projected onto the object to obtain dense feature points and remove mismatched points. Finally, the 3D model is generated by performing Delaunay triangulation and texture mapping on the point cloud obtained by four-view matching. This method was tested on the 3D reconstruction of a terracotta soldier sculpture and the Buddhas in the Mogao Grottoes. Their point clouds without mismatched points were obtained and less processing time was consumed in most cases relative to binocular matching. These good reconstructed models show the effectiveness of the method.

Determination of Surface Roughness in Wire and Arc Additive Manufacturing Based on Laser Vision Sensing

Wire and arc additive manufacturing(WAAM) shows a great promise for fabricating fully dense metal parts by means of melting materials in layers using a welding heat source. However, due to a large layer height produced in WAAM, an unsatisfactory surface roughness of parts processed by this technology has been a key issue. A methodology based on laser vision sensing is proposed to quantitatively calculate the surface roughness of parts deposited by WAAM.Calibrations for a camera and a laser plane of the optical system are presented. The reconstruction precision of the laser vision system is verified by a standard workpiece. Additionally, this determination approach is utilized to calculate the surface roughness of a multi-layer single-pass thin-walled part. The results indicate that the optical measurement approach based on the laser vision sensing is a simple and effective way to characterize the surface roughness of parts deposited by WAAM. The maximum absolute error is less than 0.15 mm. The proposed research provides the foundation for surface roughness optimization with different process parameters.

Vision sensing of P-GMAW weld pool using narrow band filters with different center wavelengths

The selection of narrow band filter＇s center wavelength has an important influence on the quality of the weld pool image captured in passive vision mettmd for controlling arc welding process. In this paper, 29 narrow band filters with different center wavelengths ranging from ultraviolet to near infrared wavelength were used to capture images of weld pool during both the peak current period and the base current period of P-GMA W （ pulsed gas metal arc welding）. The experimental results showed that the filters with near infrared center wavelength could weaken the arc most during the peak current period, while during the base current period the entire weld pool could be seen with the filters with wavelength above 600 nm. The camera spectral response and the radio of arc to weld pool emission were considered to analyze the phenomenon of experimental results.

Evaluation of Three-dimensional Urban Expansion: A Case Study of Yangzhou City, Jiangsu Province, China

With rapid urban development in China in the last two decades, the three-dimensional(3D) characteristic has been the main feature of urban morphology. However, the vast majority of researches of urban growth have focused on the planar area(two-dimensional(2D)) expansion. Few studies have been conducted from a 3D perspective. In this paper, the 3D urban expansion of the Yangzhou City, Jiangsu Province, China from 2003 to 2012 was evaluated based on Geographical Information System(GIS) tools and high-resolution remote sensing images. Four indices, namely weighted average height of buildings, volume of buildings, 3D expansion intensity and 3D fractal dimension are used to quantify the 3D urban expansion. The weighted average height of buildings and the volume of buildings are used to illustrate the temporal change of the 3D urban morphology, while the other two indices are used to calculate the expansion intensity and the fractal dimension of the 3D urban morphology. The results show that the spatial distribution of the high-rise buildings in Yangzhou has significantly spread and the utilization of the 3D space of Yangzhou has become more efficient and intensive. The methods proposed in this paper laid a foundation for a wide range of study of 3D urban morphology changes.

Optical encryption of multiple three-dimensional objects based on multiple interferences and single-pixel digital holography

We present an optical encryption method of multiple three-dimensional objects based on multiple interferences and single-pixel digital holography. By modifying the Mach-Zehnder interferometer, the interference of the multiple objects beams and the one reference beam is used to simultaneously encrypt multiple objects into a ciphertext. During decryption, each three-dimensional object can be decrypted independently without having to decrypt other objects. Since the single- pixel digital holography based on compressive sensing theory is introduced, the encrypted data of this method is effectively reduced. In addition, recording fewer encrypted data can greatly reduce the bandwidth of network transmission. Moreover, the compressive sensing essentially serves as a secret key that makes an intruder attack invalid, which means that the system is more secure than the conventional encryption method. Simulation results demonstrate the feasibility of the proposed method and show that the system has good security performance.

Clinical Value of Stereoscopic Three-dimensional Echocardiography in Assessment of Atrial Septal Defects: Feasibility and Efficiency

Stereoscopic three-dimensional echocardiography（S-3DE） is a novel displaying technol-ogy based on real-time 3-dimensional echocardiography （RT-3DE）. Our study was to evaluate the feasibility and efficiency of S-3DE in the diagnosis of atrial septal defect （ASD） and its use in the guidance for transcatheter ASD occlusion. Twelve patients with secundum ASD underwent RT-3DE examination and 9 of the 12 were subjected to transcatheter closure of ASD. Stereoscopic vision was generated with a high-performance volume renderer with red-green stereoscopic glasses. S-3DE was compared with standard RT-3D display for the assessment of the shape, size, and the surrounding tis-sues of ASD and for the guidance of ASD occlusion. The appearance rate of coronary sinus and the mean formation time of the IVC, SVC were compared. Our results showed that S-3DE could measure the diameter of ASD accurately and there was no significant difference in the measurements between S-3DE and standard 3D display （2.89±0.73 cm vs 2.85±0.72 cm, P〉0.05; r=0.96, P〈0.05）. The appearance of coronary sinus for S-3DE was higher as compared with the standard 3D display （93.3% vs 100%）. The mean time of the IVC, SVC for S-3DE monitor was slightly shorter than that of the standard 3D display （11.0±3.8 s vs 10.3±3.6 s, P〉0.05）. The mean completion time of interven-tional procedure was shortened with S-3DE display as compared with standard 3D display （17.3±3.1 min vs 23.0±3.9 min, P〈0.05）. Stereoscopic three-dimensional echocardiography could improve the visualization of three-dimensional echocardiography, facilitate the identification of the adjacent structures, decrease the time required for interventional manipulation. It may be a feasible, safe, and efficient tool for guiding transcatheter septal occlusion or the surgical interventions.

Single exposure passive three-dimensional information reconstruction based on an ordinary imaging system

Existing three-dimensional(3D) imaging technologies have issues such as requiring active illumination, multiple exposures, or coding modulation. We propose a passive single 3D imaging method based on an ordinary imaging system.Using the point spread function of the imaging system to realize the non-coding measurement on the target, the full-focus images and depth information of the 3D target can be extracted from a single two-dimensional(2D) image through the compressed sensing algorithm. Simulation and experiments show that this approach can complete passive 3D imaging based on an ordinary imaging system without any coding operations. This method can achieve millimeter-level vertical resolution under single exposure conditions and has the potential for real-time dynamic 3D imaging. It improves the efficiency of 3D information detection, reduces the complexity of the imaging system, and may be of considerable value to the field of computer vision and other related applications.

Sparse three-dimensional imaging for forward-looking array SAR using spatial continuity

For forward-looking array synthetic aperture radar(FASAR),the scattering intensity of ground scatterers fluctuates greatly since there are kinds of vegetations and topography on the surface of the ground,and thus the signal-to-noise ratio(SNR)of its echo signals corresponding to different vegetations and topography also varies obviously.Owing to the reason known to all,the performance of the sparse reconstruction of compressed sensing(CS)becomes worse in the case of lower SNR,and the quality of the sparse three-dimensional imaging for FASAR would be affected significantly in the practical application.In this paper,the spatial continuity of the ground scatterers is introduced to the sparse recovery algorithm of CS in the threedimensional imaging for FASAR,in which the weighted least square method of the cubic interpolation is used to filter out the bad and isolated scatterer.The simulation results show that the proposed method can realize the sparse three-dimensional imaging of FASAR more effectively in the case of low SNR.

Transformer-Based Cloud Detection Method for High-Resolution Remote Sensing Imagery

Cloud detection from satellite and drone imagery is crucial for applications such as weather forecasting and environmentalmonitoring.Addressing the limitations of conventional convolutional neural networks,we propose an innovative transformer-based method.This method leverages transformers,which are adept at processing data sequences,to enhance cloud detection accuracy.Additionally,we introduce a Cyclic Refinement Architecture that improves the resolution and quality of feature extraction,thereby aiding in the retention of critical details often lost during cloud detection.Our extensive experimental validation shows that our approach significantly outperforms established models,excelling in high-resolution feature extraction and precise cloud segmentation.By integrating Positional Visual Transformers(PVT)with this architecture,our method advances high-resolution feature delineation and segmentation accuracy.Ultimately,our research offers a novel perspective for surmounting traditional challenges in cloud detection and contributes to the advancement of precise and dependable image analysis across various domains.

基于Vision Transformer-LSTM(ViTL)的多时序遥感影像农作物分类方法

针对当前遥感农作物分类研究中深度学习模型对光谱时间和空间信息特征采样不足,农作物提取仍然存在边界模糊、漏提、误提的问题,提出了一种名为视觉Transformer-长短期记忆递归神经网络(Vision Transformer-long short term memory,ViTL)的深度学习模型,ViTL模型集成了双路Vision-Transformer特征提取、时空特征融合和长短期记忆递归神经网络(LSTM)时序分类等3个关键模块,双路Vision-Transformer特征提取模块用于捕获图像的时空特征相关性,一路提取空间分类特征,一路提取时间变化特征;时空特征融合模块用于将多时特征信息进行交叉融合;LSTM时序分类模块捕捉多时序的依赖关系并进行输出分类。综合利用基于多时序卫星影像的遥感技术理论和方法,对黑龙江省齐齐哈尔市讷河市作物信息进行提取,研究结果表明,ViTL模型表现出色,其总体准确率(Overall Accuracy,OA)、平均交并比(Mean Intersection over Union,MIoU)和F1分数分别达到0.8676、0.6987和0.8175,与其他广泛使用的深度学习方法相比,包括三维卷积神经网络(3-D CNN)、二维卷积神经网络(2-D CNN)和长短期记忆递归神经网络(LSTM),ViTL模型的F1分数提高了9%~12%,显示出显著的优越性。ViTL模型克服了面对多时序遥感影像的农作物分类任务中的时间和空间信息特征采样不足问题,为准确、高效地农作物分类提供了新思路。

Specular reflection based sensing surface deformation of gas tungsten arc weld pool

A sensing system is developed to measure the weld pool boundary and pool suoface deformation in gas tungsten arc welding. LaserStrobe technique is used to eliminate the strong arc light inteoference, and specular reflection from the pool suoface is sensed to describe the relation between the deformed stripes and pool surface depression. Clear images of both the pool boundary and the deformed stripes edges are obtained during gas tungsten arc welding process, which lays foundation for realtime monitoring the pool suoface depression and weld penetration.

Benchmarking dynamic properties of structures using non-contact sensing

Non-contact sensing can be a rapid and convenient alternative for determining structure response compared to conventional instrumentation.Computer vision has been broadly implemented to enable accurate non-contact dynamic response measurements for structures.This study has analyzed the effect of non-contact sensors,including type,frame rate,and data collection platform,on the performance of a novel motion detection technique.Video recordings of a cantilever column were collected using a high-speed camera mounted on a tripod and an unmanned aerial system(UAS)equipped with visual and thermal sensors.The test specimen was subjected to an initial deformation and released.Specimen acceleration data were collected using an accelerometer installed on the cantilever end.The displacement from each non-contact sensor and the acceleration from the contact sensor were analyzed to measure the specimen′s natural frequency and damping ratio.The specimen′s first fundamental frequency and damping ratio results were validated by analyzing acceleration data from the top of the specimen and a finite element model.

Airborne sparse flight array SAR 3D imaging based on compressed sensing in frequency domain

In airborne array synthetic aperture radar(SAR), the three-dimensional(3D) imaging performance and cross-track resolution depends on the length of the equivalent array. In this paper, Barker sequence criterion is used for sparse flight sampling of airborne array SAR, in order to obtain high cross-track resolution in as few times of flights as possible. Under each flight, the imaging algorithm of back projection(BP) and the data extraction method based on modified uniformly redundant arrays(MURAs) are utilized to obtain complex 3D image pairs. To solve the side-lobe noise in images, the interferometry between each image pair is implemented, and compressed sensing(CS) reconstruction is adopted in the frequency domain. Furthermore, to restore the geometrical relationship between each flight, the phase information corresponding to negative MURA is compensated on each single-pass image reconstructed by CS. Finally,by coherent accumulation of each complex image, the high resolution in cross-track direction is obtained. Simulations and experiments in X-band verify the availability.

Simultaneous observation of keyhole and weld pool in plasma arc welding with a single cost-effective sensor

The dynamic behaviors of the keyhole and weld pool are coupled together in plasma arc welding, and the geometric variations of both the keyhole and the weld pool determine the weld quality. It is of great significance to simultaneously sense and monitor the keyhole and the weld pool behaviors by using a single low-cost vision sensor in plasma arc welding process. In this study, the keyhole and weld pool were observed and measured under different levels of welding current by using the near infrared sensing technology and the charge coupled device （CCD） sensing system. The shapes and relative position of weld pool and keyhole under different conditions were compared and analyzed. The observation results lay solid foundation for controlling weld quality and understanding the underlying process mechanisms.

Measuring tree stem diameters and straightness with depth-image computer vision

Current techniques of forest inventory rely on manual measurements and are slow and labor intensive.Recent developments in computer vision and depth sensing can produce accurate measurement data at significantly reduced time and labor costs.We developed the ForSense system to measure the diameters of trees at various points along the stem as well as stem straightness.Time use,mean absolute error(MAE),and root mean squared error(RMSE)metrics were used to compare the system against manual methods,and to compare the system against itself(reproducibility).Depth-derived diameter measurements of the stems at the heights of 0.3,1.4,and 2.7 m achieved RMSE of 1.7,1.5,and 2.7 cm,respectively.The ForSense system produced straightness measurement data that was highly correlated with straightness ratings by trained foresters.The ForSense system was also consistent,achieving sub-centimeter diameter difference with subsequent measures and less than 4%difference in straightness value between runs.This method of forest inventory,which is based on depth-image computer vision,is time efficient compared to manual methods and less computationally and technologically intensive compared to Structure-from-Motion(SFM)photogrammetry and ground-based LiDAR or terrestrial laser scanning(TLS).

Intelligent Risk-Identification Algorithm with Vision and 3D LiDAR Patterns at Damaged Buildings

Existingfirefighting robots are focused on simple storage orfire sup-pression outside buildings rather than detection or recognition.Utilizing a large number of robots using expensive equipment is challenging.This study aims to increase the efficiency of search and rescue operations and the safety offirefigh-ters by detecting and identifying the disaster site by recognizing collapsed areas,obstacles,and rescuers on-site.A fusion algorithm combining a camera and three-dimension light detection and ranging(3D LiDAR)is proposed to detect and loca-lize the interiors of disaster sites.The algorithm detects obstacles by analyzingfloor segmentation and edge patterns using a mask regional convolutional neural network(mask R-CNN)features model based on the visual data collected from a parallelly connected camera and 3D LiDAR.People as objects are detected using you only look once version 4(YOLOv4)in the image data to localize persons requiring rescue.The point cloud data based on 3D LiDAR cluster the objects using the density-based spatial clustering of applications with noise(DBSCAN)clustering algorithm and estimate the distance to the actual object using the center point of the clustering result.The proposed artificial intelligence(AI)algorithm was verified based on individual sensors using a sensor-mounted robot in an actual building to detectfloor surfaces,atypical obstacles,and persons requiring rescue.Accordingly,the fused AI algorithm was comparatively verified.

Reduced Imaging Time and Improved Image Quality of 3D Isotropic T2-Weighted Magnetic Resonance Imaging with Compressed Sensing for the Female Pelvis

This study is to compare three-dimensional(3D)isotropic T2-weighted magnetic resonance imaging(MRI)with compressed sensing-sampling perfection with application optimized contrast(CS-SPACE)and the conventional image(3D-SPACE)sequence in terms of image quality,estimated signal-to-noise ratio(SNR),relative contrast-to-noise ratio(CNR),and the lesions’conspicuous of the female pelvis.Thirty-six females(age:51,28-73)with cervical carcinoma(n=20),rectal carcinoma(n=7),or uterine fibroid(n=9)were included.Patients underwent magnetic resonance(MR)imaging at a 3T scanner with the sequences of 3D-SPACE,CS-SPACE,and twodimensional(2D)T2-weighted turbo-spin echo(TSE).Quantitative analyses of estimated SNR and relative CNR between tumors and other tissues,image quality,and tissue conspicuity were performed.Two radiologists assessed the difference in diagnostic findings for carcinoma.Quantitative values and qualitative scores were analyzed,respectively.The estimated SNR and the relative CNR of tumor-to-muscle obturator internus,tumor-to-myometrium,and myometrium-to-muscle obturator internus was comparable between 3D-SPACE and CS-SPACE.The overall image quality and the conspicuity of the lesion scores of the CS-SPACE were higher than that of the 3D-SPACE(P<0.01).The CS-SPACE sequence offers shorter scan time,fewer artifacts,and comparable SNR and CNR to conventional 3D-SPACE,and has the potential to improve the performance of T2-weighted images.

运动与不运动老年人姿势平衡中的感觉交互作用研究

目的:对比运动与不运动老年人前庭、视觉和本体感觉在维持静态平衡中的作用,为预防老年人跌倒提供参考。方法:使用SunlightTetrax平衡诊断训练仪对运动老年人(203例)和不运动老年人(168例)进行不同站立条件稳定性系数(ST)测试。研究结果:①女性NC、PO、PC姿势ST低于男性(P<0.05)。②本体感觉功能受影响时,不运动老年人主要通过调节视觉,运动老年人通过调节前庭觉维持静态平衡。③NC、PO姿势下老年男性骑自行车运动组及PO姿势下老年女性广场舞组的ST低于散步组(P<0.05)。研究结论:规律运动通过改善前庭觉、本体感觉达到维持平衡能力的作用。广场舞、骑自行车可改善视觉或本体感觉受干扰情况下老年人的稳定性。

结合多尺度多注意力的遥感图像超分辨率重构

视觉Transformer在改进图像超分辨率性能方面展现了良好的潜能.然而,遥感图像中不同目标表现的尺度多样性限制了其超分辨率的图像质量.为此,研究了一种结合多尺度多注意力的Transformer遥感图像超分辨率网络,旨在增强其特征学习能力,从而有效提升遥感图像的超分辨率性能.具体来说,输入特征首先通过多级下采样,得到多个尺度的特征;然后,逐级将低维特征通过一种交替密集注意力与稀疏注意力的Transformer网络进行变换,并将输出结果升维后与高维特征融合.密集注意力与稀疏注意力的结合可同时兼顾对局部相关性和全局相关性的有效提取,而多通路多尺度变换能够增强对图像小目标的建模能力.基于两个开源的遥感数据集的大量实验结果,验证了该方法的有效性.

融合FEB的YOLOX遥感图像目标检测算法

针对遥感图像的复杂背景干扰大、目标尺度变化大以及小目标检测困难等导致检测精度降低的问题,提出一种增强YOLOX主干网络输出特征提取能力的检测算法。通过加入连续膨胀残差卷积和注意力机制,设计一种全新的提取主干网络输出特征增强块(feature enhance block,FEB),让连续膨胀残差卷积串联4个具有不同膨胀率的膨胀残差卷积,扩大算法的感受野,丰富上下文信息,同时减轻背景对检测的影响,有效加强算法对目标尺度变化大及小目标的检测能力,使用SA注意力机制抑制背景对算法检测的干扰,提高算法的检测精度。在RSOD数据集上的实验表明,FEB相较于其他同类型模块具有更好的特征提取能力。