Warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography

High speed photography technique is potentially the most effective way to measure the motion parameter of warhead fragment benefiting from its advantages of high accuracy,high resolution and high efficiency.However,it faces challenge in dense objects tracking and 3D trajectories reconstruction due to the characteristics of small size and dense distribution of fragment swarm.To address these challenges,this work presents a warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography.Firstly,background difference algorithm is utilized to extract the center and area of each fragment in the image sequence.Subsequently,a multi-object tracking(MOT)algorithm using Kalman filtering and Hungarian optimal assignment is developed to realize real-time and robust trajectories tracking of fragment swarm.To reconstruct 3D motion trajectories,a global stereo trajectories matching strategy is presented,which takes advantages of epipolar constraint and continuity constraint to correctly retrieve stereo correspondence followed by 3D trajectories refinement using polynomial fitting.Finally,the simulation and experimental results demonstrate that the proposed method can accurately track the motion trajectories and reconstruct the spatio-temporal distribution of 1.0×10^(3)fragments in a field of view(FOV)of 3.2 m×2.5 m,and the accuracy of the velocity estimation can achieve 98.6%.

Sparse Reconstructive Evidential Clustering for Multi-View Data

Although many multi-view clustering(MVC) algorithms with acceptable performances have been presented, to the best of our knowledge, nearly all of them need to be fed with the correct number of clusters. In addition, these existing algorithms create only the hard and fuzzy partitions for multi-view objects,which are often located in highly-overlapping areas of multi-view feature space. The adoption of hard and fuzzy partition ignores the ambiguity and uncertainty in the assignment of objects, likely leading to performance degradation. To address these issues, we propose a novel sparse reconstructive multi-view evidential clustering algorithm(SRMVEC). Based on a sparse reconstructive procedure, SRMVEC learns a shared affinity matrix across views, and maps multi-view objects to a 2-dimensional humanreadable chart by calculating 2 newly defined mathematical metrics for each object. From this chart, users can detect the number of clusters and select several objects existing in the dataset as cluster centers. Then, SRMVEC derives a credal partition under the framework of evidence theory, improving the fault tolerance of clustering. Ablation studies show the benefits of adopting the sparse reconstructive procedure and evidence theory. Besides,SRMVEC delivers effectiveness on benchmark datasets by outperforming some state-of-the-art methods.

Improvement of Binocular Reconstruction Algorithm for Measuring 3D Pavement Texture Using a Single Laser Line Scanning Constraint

The dense and accurate measurement of 3D texture is helpful in evaluating the pavement function.To form dense mandatory constraints and improve matching accuracy,the traditional binocular reconstruction technology was improved threefold.First,a single moving laser line was introduced to carry out global scanning constraints on the target,which would well overcome the difficulty of installing and recognizing excessive laser lines.Second,four kinds of improved algorithms,namely,disparity replacement,superposition synthesis,subregion segmentation,and subregion segmentation centroid enhancement,were established based on different constraint mechanism.Last,the improved binocular reconstruction test device was developed to realize the dual functions of 3D texture measurement and precision self-evaluation.Results show that compared with traditional algorithms,the introduction of a single laser line scanning constraint is helpful in improving the measurement’s accuracy.Among various improved algorithms,the improvement effect of the subregion segmentation centroid enhancement method is the best.It has a good effect on both overall measurement and single pointmeasurement,which can be considered to be used in pavement function evaluation.

Automatic three-dimensional reconstruction based on four-view stereo vision using checkerboard pattern

An automatic three-dimensional(3D) reconstruction method based on four-view stereo vision using checkerboard pattern is presented. Mismatches easily exist in traditional binocular stereo matching due to the repeatable or similar features of binocular images. In order to reduce the probability of mismatching and improve the measure precision, a four-camera measurement system which can add extra matching constraints and offer multiple measurements is applied in this work. Moreover, a series of different checkerboard patterns are projected onto the object to obtain dense feature points and remove mismatched points. Finally, the 3D model is generated by performing Delaunay triangulation and texture mapping on the point cloud obtained by four-view matching. This method was tested on the 3D reconstruction of a terracotta soldier sculpture and the Buddhas in the Mogao Grottoes. Their point clouds without mismatched points were obtained and less processing time was consumed in most cases relative to binocular matching. These good reconstructed models show the effectiveness of the method.

3D Measurement and Stereo Reconstruction for Aeroengine Interior Damage

The borescopy inspection problem of aeroengine interior important partdamages such as firebox's burn and corruption, vane' s crack, bump, abrade and concave pit, is aimedat. A new system is developed to carry out 3D measurement and stereo reconstruction of engineinterior damage, in which the borescope of Japanese OLYMPUS Corporation is used as hardware. In thesystem, functions are implemented, such as image collection, camera calibration, imagepreprocessing, stereo matching, 3D measurement and stereo reconstruction. It can provide moredetailed inspection and more accurate estimation of engine interior damages. Finally, an example isused to verify the effectivity of the new method.

Fast Estimation of Loader’s Shovel Load Volume by 3D Reconstruction of Material Piles

Fast and accurate measurement of the volume of earthmoving materials is of great signifcance for the real-time evaluation of loader operation efciency and the realization of autonomous operation. Existing methods for volume measurement, such as total station-based methods, cannot measure the volume in real time, while the bucket-based method also has the disadvantage of poor universality. In this study, a fast estimation method for a loader’s shovel load volume by 3D reconstruction of material piles is proposed. First, a dense stereo matching method (QORB–MAPM) was proposed by integrating the improved quadtree ORB algorithm (QORB) and the maximum a posteriori probability model (MAPM), which achieves fast matching of feature points and dense 3D reconstruction of material piles. Second, the 3D point cloud model of the material piles before and after shoveling was registered and segmented to obtain the 3D point cloud model of the shoveling area, and the Alpha-shape algorithm of Delaunay triangulation was used to estimate the volume of the 3D point cloud model. Finally, a shovel loading volume measurement experiment was conducted under loose-soil working conditions. The results show that the shovel loading volume estimation method (QORB–MAPM VE) proposed in this study has higher estimation accuracy and less calculation time in volume estimation and bucket fll factor estimation, and it has signifcant theoretical research and engineering application value.

New Method of Reconstruction from Nonparallel Stereo and Application to Surgical Navigator

A new method to reconstruct 3D scene points from nonparallel stereo is proposed. From a pair of conjugate images in an arbitrarily configured stereo system that has been calibrated, coordinates of 3D scene points can be computed directly using the method, bypassing the process of rectifying images or iterative solution involved in existing methods. Experiment results from both simulated data and real images validate the method. Practical application to surgical navigator shows that the method has advantages to improve efficiency and accuracy of 3D reconstruction from nonparallel stereo system in comparison with the conventional method that employs algorithm for standard parallel axes stereo geometry.

Research on Multi-View Image Reconstruction Technology Based on Auto-Encoding Learning

Traditional three-dimensional(3D)image reconstruction method,which highly dependent on the environment and has poor reconstruction effect,is easy to lead to mismatch and poor real-time performance.The accuracy of feature extraction from multiple images affects the reliability and real-time performance of 3D reconstruction technology.To solve the problem,a multi-view image 3D reconstruction algorithm based on self-encoding convolutional neural network is proposed in this paper.The algorithm first extracts the feature information of multiple two-dimensional(2D)images based on scale and rotation invariance parameters of Scale-invariant feature transform(SIFT)operator.Secondly,self-encoding learning neural network is introduced into the feature refinement process to take full advantage of its feature extraction ability.Then,Fish-Net is used to replace the U-Net structure inside the self-encoding network to improve gradient propagation between U-Net structures,and Generative Adversarial Networks(GAN)loss function is used to replace mean square error(MSE)to better express image features,discarding useless features to obtain effective image features.Finally,an incremental structure from motion(SFM)algorithm is performed to calculate rotation matrix and translation vector of the camera,and the feature points are triangulated to obtain a sparse spatial point cloud,and meshlab software is used to display the results.Simulation experiments show that compared with the traditional method,the image feature extraction method proposed in this paper can significantly improve the rendering effect of 3D point cloud,with an accuracy rate of 92.5%and a reconstruction complete rate of 83.6%.

3D Surface Reconstruction of Coarse Aggregate Particles from Occlusion-Free Multi-View Images

Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.

Estimation of the Scale of Artificial Reef Sets on the Basis of Underwater 3D Reconstruction

The creation of three-dimensional models from an unorganized set of points is an active research area in computer graphics.One of the purposes of this study is to explore the 3D reconstruction of a cube-type artificial reef(CTAR)set by linear structured light and binocular stereo vision technology in an underwater environment.The experimental setup is composed of two ca-meras in a stereo vision configuration.The alpha shapes method can be used to construct a surface that most closely reflects the arti-ficial reef set described by the points.A parameter study is conducted to assess the scales of the set(i.e.,usable volume,surface area,projected area,height,and base diameter)on the basis of 3D reconstruction.Experimental results show that the quality of 3D recon-struction in an underwater environment is acceptable for estimating the scale size of the CTAR set.According to the measurement of the scale sizing of the CTAR set,the relationships between the parameters of the CTAR set and the number of CTAR modules were determined.Moreover,the usable volume of the CTAR set can be estimated depending on the basis of the number of CTAR modules.

基于自适应聚合循环递归的稠密点云重建网络

为了解决弱纹理重建难、资源消耗大和重建时间长等问题,提出了一种基于自适应聚合循环递归卷积的多阶段稠密点云重建网络,即A2R2-MVSNet(adaptive aggregation recurrent recursive multi view stereo net)。该方法首先引入一种基于多尺度循环递归残差的特征提取模块,聚合上下文语义信息,以解决弱纹理或无纹理区域特征提取难的问题。在代价体正则化部分,提出一种残差正则化模块,该模块在略微增加内存消耗的前提下,提高了3D CNN提取和聚合上下文语意的能力。实验结果表明,提出的方法在DTU数据集上的综合指标排名靠前,在重建细节上有着更好的体现,且在BlendedMVS数据集上生成了不错的深度图和点云结果,此外网络还在自采集的大规模高分辨率数据集上进行了泛化测试。归功于由粗到细的多阶段思想和我们提出的模块,网络在生成高准确性和完整性深度图的同时,还能进行高分辨率重建以适用于实际问题。

注意力机制与神经渲染的多视图三维重建算法

针对多视图立体网络在弱纹理或非朗伯曲面等挑战性区域重建效果差的问题,首先提出一个基于3个并行扩展卷积和注意力机制的多尺度特征提取模块,在增加感受野的同时捕获特征之间的依赖关系以获取全局上下文信息,从而提升多视图立体网络在挑战性区域特征的表征能力以进行鲁棒的特征匹配。其次在代价体正则化3D CNN部分引入注意力机制,使网络注意于代价体中的重要区域以进行平滑处理。另外建立一个神经渲染网络,该网络利用渲染参考损失精确地解析辐射场景表达的几何外观信息,并引入深度一致性损失保持多视图立体网络与神经渲染网络之间的几何一致性,有效地缓解有噪声代价体对多视图立体网络的不利影响。该算法在室内DTU数据集中测试,点云重建的完整性和整体性指标分别为0.289和0.326,与基准方法CasMVSNet相比,分别提升24.9%和8.2%,即使在挑战性区域也得到高质量的重建效果;在室外Tanks and Temples中级数据集中,点云重建的平均F-score为60.31,与方法UCS-Net相比提升9.9%,体现出较强的泛化能力。

基于变分立体匹配算法的GMAW熔池形貌三维重建

为实现完整熔池表面形貌三维传感,构建了双棱镜单摄像机立体视觉传感系统.针对熔池图像纹理缺乏造成的立体匹配困难的问题,引入了全局优化的变分立体匹配算法,通过建立包含灰度差异数据项和空间连续性约束项的能量函数的可行性泛函,经过迭代求解获得具有丰富细节的熔池表面稠密视差图.对自制非标准凹面形状进行立体匹配和三维重建,结果表明,宽度误差小于3.16%,深度误差小于4.82%.基于该算法实现了熔化极气体保护焊(gas metal arc welding,GMAW)的堆焊及V形坡口对焊条件下,不同熔透状态熔池稠密视差图计算和表面形貌的三维重建.

信号博弈的三维重建形变识别算法

现有双目立体视觉算法常常需要双目相机位置固定,在现实应用中,这类算法难以重构空间三维几何关系.为此,本文提出了一种不受位置限制的多视角图像三维重建与形变检测算法.该算法首先采用sift算法获取成对图像的特征点,以获取形变前后比对点对图像的特征信息;其次,利用信号博弈方法确定图像拍摄时相机的空间位置与视角,以准确获取图像的空间位置坐标;再次,依据上述信息完成物体的三维点云重建;进而,利用三维数据信息比对实现物体形变识别.最后,本文利用真实物体的实验,验证了三维重建形变识别算法的有效性.

语义增强的多视立体视觉方法

针对在基于深度学习技术的特征提取网络中,深层次的卷积神经网络提取的特征缺乏低级语义信息的问题,该文提出了语义增强的多视立体视觉方法。首先,提出了一种ConvLSTM(Convolutional Long Short-Term Memory)语义聚合网络,通过使用ConvLSTM网络结构,对多个卷积层提取的特征图进行预测,得到融合每层语义信息的特征图,有助于在空间上层层抽取图像的高级特征时,利用长短期记忆神经网络结构的记忆功能来增强高层特征图中的低级语义信息,提高了弱纹理区域的重建效果,提高了3D重建的鲁棒性和完整性;其次,提出了一种可见性网络,在灰度图的基础上,通过突出特征图上可见区域的特征,加深了可见区域在特征图中的影响,有助于提高三维重建效果;最后,提取图像的纹理信息,并进入ConvLSTM语义聚合网络提取深层次特征,提高了弱纹理区域的重建效果。与主流的多视立体视觉重建方法相比,重建效果较好。

高山峡谷多视卫星立体像对真三维重建

为解决大面积艰险条件下高山峡谷真三维重建难题,提高卫星影像空间及光谱分辨率,以中国西藏某高山峡谷流域Pleiades卫星立体像对为切入点,探讨了RPC轨道参数校正、SIFT算子优化密集匹配等卫星影像三维重建关键技术,构建无控定位条件下全色、多光谱及融合状态下真三维重建模型。结果表明:3种方案下多视卫星立体像对匹配点误差均小于1个像素,全色方案匹配点精度达到0.67个像素,丰富了1∶5千、1∶1万地形图测绘、地质灾害早期识别及监测、高海拔地区冰川演变分析手段及方法。研究成果可为卫星影像真三维重建提供借鉴。

基于双目立体匹配的三维重建系统研究

在计算机视觉领域,基于双目立体匹配的三维重建系统是一个研究热点。首先进行双目相机标定,获取标定参数。而后采用立体校正方法,使成像平面对准共面行,借助SGBM(semi-globalblock matching)立体匹配算法可以计算出视差。得到视差图后,计算像素深度,获取像素点的三维坐标。恢复三维坐标后使用Open3D恢复并显示点云图。

Wheel center detection based on stereo vision

As the location of the wheel center is the key to accurately measuring the wheelbase, the wheelbase difference and the wheel static radius, a high-precision wheel center detection method based on stereo vision is proposed. First, according to the prior information, the contour of the wheel hub is extracted and fitted as an ellipse curve, and the ellipse fitting equation can be obtained. Then, a new un-tangent constraint is adopted to improve the ellipse matching precision. Finally, the 3D coordinates of the wheel center can be reconstructed by the spatial circle projection algorithm with low time complexity and high measurement accuracy. Simulation experiments verify that compared with the ellipse center reconstruction algorithm and the planar constraint optimization algorithm, the proposed method can acquire the 3D coordinates of the spatial circle more exactly. Furthermore, the measurements of the wheelbase, the wheelbase difference and the wheel static radius for three types of vehicles demonstrate the effectiveness of the proposed method for wheel center detection.

基于双目立体视觉的实验室波浪场实时重建

准确测量波浪场演化对于海洋结构物的水动力研究十分重要。本文基于双目立体视觉建立了一套用于测量室内波浪的图像识别方法,以轻质泡沫作为示踪粒子标记波面以获得具有丰富纹理的波面图像,根据极线约束矫正图像,使用阈值分割避免光照不均和水底反射的干扰,使用GPU加速的立体视觉匹配算法,实现波浪图像的实时立体匹配,重建出波面的三维点云与网格模型。通过对图像序列的重建,提取出的波浪时历与浪高仪的测量结果基本一致,波高以及周期值的吻合性良好。结果表明,该方法可以准确测量实验室内的三维波浪场,捕捉瞬时波浪演化,并可提供实时的监控。

基于双目视觉和单线激光雷达的三维场景重建系统和算法

双目视觉和激光雷达是实现三维场景重建的两种有效方法,但它们都具有自身的局限性。将视觉传感器和激光传感器的数据相融合,可克服其各自的缺陷,具有重要意义。本文针对单线旋转激光雷达设备的独特性,提出了一种基于双目图像和激光雷达数据的模块化双目视觉-激光雷达SLAM(Stereo-LiDAR SLAM)系统和算法。在该方法中,双目视觉信息被用于消除激光雷达点云的畸变,并支撑双目视觉定位(VO)模块,为整个系统提供初始的位姿估计。然后,独立于VO模块的激光雷达SLAM(L-SLAM)模块对位姿参数进行优化,从而得到高精度的三维场景重建结果。试验表明,本文设计的系统和算法可以有效提高大规模低成本三维场景重建的精度和环境适应能力。