基于Stereo Camera-YOLOv5自然环境下百香果检测与定位模型

针对百香果采摘机器人在自然环境中作业时受复杂光线及遮挡影响,难以快速精确地检测及定位成熟百香果的问题,提出一种基于Stereo Camera-YOLOv5自然环境下成熟百香果检测及定位模型。针对自然环境下光线以及遮挡的影响,通过MSRCP算法、随机遮挡、数据增扩等图像处理算法对原始数据集进行优化。将优化的数据集输入到YOLOv5网络中训练出最优模型,在检测代码中嵌入双目立体视觉算法。该模型对自然环境下百香果进行检测及成熟度判断,将判断为成熟的百香果进行图像处理,并提取到中心点二维坐标。通过立体匹配及视差计算得到中心点的三维坐标。田间试验结果表明,该模型的目标检测准确率为97.8%,总体准确率为90.2%,平均运行时间为4.85 s。该系统鲁棒性强、实时性好,能够更好地实现自然环境下成熟百香果的检测及定位,为百香果采摘机器人后续工作奠定基础。

A Skeletal Camera Network for Close-range Images with a Data Driven Approach in Analyzing Stereo Configuration

Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.

Comparison between Plane and Stereo Block Adjustment for ZY-3 Satellite Images

Block adjustment for satellite images cannot be solved with weak convergence geometric conditions,therefore a plane block adjustment method to improve the targeting precision of images is proposed utilizing DEM as height constraint plane block adjustment method.First,a rational function model with affine transformation is selected as the mathematical model of the satellite image plane block adjustment.Second,to update the ground coordinates of tie points(TPs),the plane coordinates of TPs are only solved in the adjustment process.Elevation values are obtained by using DEM interpolation.Finally,the plane coordinates of all TPs and orientation parameters of all satellite images are solved through plane block adjustment with a few ground control points ZY-3 nadir images for two regions are tested for plane block adjustment while ZY-3 forward-nadir-back images of the same two regions are tested for stereo block adjustment.A comparison indicates that almost the same accuracy can be obtained with plane block adjustment support using a 1∶50 000 DEM and stereo block adjustment for ZY-3 images.For ZY-3 nadir images,almost no loss of plane block adjustment accuracy occurred when global DEM with 1 km grid and SRTM with 90 m grid replaced the 1∶50 000 DEM as elevation control,.Test results demonstrate the effectiveness and feasibility of the plane block adjustment method.

Stereo particle image velocimetry measurement of 3D soil deformation around laterally loaded pile in sand

A developed stereo particle image velocimetry(stereo-PIV) system was proposed to measure three-dimensional(3D) soil deformation around a laterally loaded pile in sand.The stereo-PIV technique extended 2D measurement to 3D based on a binocular vision model,where two cameras with a well geometrical setting were utilized to image the same object simultaneously.This system utilized two open software packages and some simple programs in MATLAB,which can easily be adjusted to meet user needs at a low cost.The failure planes form an angle with the horizontal line,which are measured at 27°-29°,approximately three-fourths of the frictional angle of soil.The edge of the strain wedge formed in front of the pile is an arc,which is slightly different from the straight line reported in the literature.The active and passive influence zones are about twice and six times of the diameter of the pile,respectively.The test demonstrates the good performance and feasibility of this stereo-PIV system for more advanced geotechnical testing.

Disparity-Based Generation of Line-of-Sight DSM for Image-Elevation Co-Registration to Support Building Detection in Off-Nadir VHR Satellite Images

The integration of optical images and elevation data is of great importance for 3D-assisted mapping applications. Very high resolution (VHR) satellite images provide ideal geo-data for mapping building information. Since buildings are inherently elevated objects, these images need to be co-registered with their elevation data for reliable building detection results. However, accurate co-registration is extremely difficult for off-nadir VHR images acquired over dense urban areas. Therefore, this research proposes a Disparity-Based Elevation Co-Registration (DECR) method for generating a Line-of-Sight Digital Surface Model (LoS-DSM) to efficiently achieve image-elevation data co-registration with pixel-level accuracy. Relative to the traditional photogrammetric approach, the RMSE value of the derived elevations is found to be less than 2 pixels. The applicability of the DECR method is demonstrated through elevation-based building detection (EBD) in a challenging dense urban area. The quality of the detection result is found to be more than 90%. Additionally, the detected objects were geo-referenced successfully to their correct ground locations to allow direct integration with other maps. In comparison to the original LoS-DSM development algorithm, the DECR algorithm is more efficient by reducing the calculation steps, preserving the co-registration accuracy, and minimizing the need for elevation normalization in dense urban areas.

Measurement of 3-D motion parameters of jacket launch based on stereo image sequences

To make sure that the process of jacket launch occurs in a seml-controlled manner, this paper deals with measurement of kinematic parameters of jacket launch using stereo vision and motion analysis. The system captured stereo image sequences by two separate CCD cameras, and then rebuilt 3D coordinates of the feature points to analyze the jacket launch motion. The possibility of combining stereo vision and motion analysis for measurement was examined. Resuhs by experiments using scale model of jacket confirm the theoretical data.

Optimization of the forearm angle for arm wrestling using multi-camera stereo digital image correlation: A preliminary study

This study analyzes the function of different muscles during arm wrestling and proposes a method to analyze the optimal forearm angle for professional arm wrestlers.We built a professional arm-wrestling platform to measure the shape and deformation of the skin at the biceps brachii of a volunteer in vivo during arm wrestling.We observed the banding phenomenon of arm skin strain during muscle contraction and developed a model to evaluate the moment provided by the biceps brachii.According to this model,the strain field of the area of interest on the skin was measured,and the forearm angles most favorable and unfavorable to the work of the biceps brachii were analyzed.This study demonstrates the considerable potential of applying DIC and its extension method to the in vivo measurement of human skin and facilitates the use of the in vivo measurement of skin deformation in various sports in the future.

SAR STEREO IMAGING ALGORITHM BASED ON CAPON SPECTRUM ESTIMATION

This letter presents a novel approach for the Synthetic Aperture Radar (SAR) stereo imaging based on the Capon spectrum estimation technique. In order to deal with nonuniform sampling space and lead to super resolution in the elevation direction, Capon approach is used to focus the SAR data on a certain height. Results obtained on simulated data demonstrate the feasibility of the Capon based algorithm. Compared with the classical Fast Fourier Transform (FFT), the Capon based algo-rithm shows better resolution quality.

An Adaptive and Image-guided Fusion for Stereo Satellite Image Derived Digital Surface Models

The accuracy of Digital Surface Models(DSMs)generated using stereo matching methods varies due to the varying acquisition conditions and configuration parameters of stereo images.It has been a good practice to fuse these DSMs generated from various stereo pairs to achieve enhanced,in which multiple DSMs are combined through computational approaches into a single,more accurate,and complete DSM.However,accurately characterizing detailed objects and their boundaries still present a challenge since most boundary-ware fusion methods still struggle to achieve sharpened depth discontinuities due to the averaging effects of different DSMs.Therefore,we propose a simple and efficient adaptive image-guided DSM fusion method that applies k-means clustering on small patches of the orthophoto to guide the pixel-level fusion adapted to the most consistent and relevant elevation points.The experiment results show that our proposed method has outperformed comparing methods in accuracy and the ability to preserve sharpened depth edges.

Wavelet-Based Mixed-Resolution Coding Approach Incorporating with SPT for the Stereo Image

With the advances of display technology, three-dimensional(3-D) imaging systems are becoming increasingly popular. One way of stimulating 3-D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pairs, data compression algorithms should be employed to represent stereo pairs efficiently. The proposed techniques generally use block-based disparity compensation. In order to get the higher compression ratio, this paper employs the wavelet-based mixed-resolution coding technique to incorporate with SPT-based disparity-compensation to compress the stereo image data. The mixed-resolution coding is a perceptually justified technique that is achieved by presenting one eye with a low-resolution image and the other with a high-resolution image. Psychophysical experiments show that the stereo image pairs with one high-resolution image and one low-resolution image provide almost the same stereo depth to that of a stereo image with two high-resolution images. By combining the mixed-resolution coding and SPT-based disparity-compensation techniques, one reference (left) high-resolution image can be compressed by a hierarchical wavelet transform followed by vector quantization and Huffman encoder. After two level wavelet decompositions, for the low-resolution right image and low-resolution left image, subspace projection technique using the fixed block size disparity compensation estimation is used. At the decoder, the low-resolution right subimage is estimated using the disparity from the low-resolution left subimage. A full-size reconstruction is obtained by upsampling a factor of 4 and reconstructing with the synthesis low pass filter. Finally, experimental results are presented, which show that our scheme achieves a PSNR gain (about 0.92dB) as compared to the current block-based disparity compensation coding techniques.

Three-dimensional motion parameters estimation from stereo image sequences based on infrared reflective markers

In this paper,an innovative 3D motion parameters estimation method from stereo image sequences based on infrared(IR) reflective markers is presented.It was assumed that two high speed CCD cameras had been calibrated previously.The method consists of the following steps:1) the coordinate of several markers and depth map for each stereo pair was determined from the sequences of stereo images by relations of markers' coordinate the correspondence between markers was established,2) the 3D motion parameters of the target was computed based upon the matched markers' coordinate,and 3) translated 3D motion parameters estimation into the problem of least square according to the movement model of the object to be measured.Without using line,curve or corner correspondence,this method can calculate the depth of these markers feature easily and quickly in contrast to traditional approaches.The two CCD cameras work on 200 f/s,and each processing cost time is about 3 ms.It was found that,by using several markers and a large number of stereo images,this method can improve the computational speed,robustness and numerical accuracy of the motion parameters in comparison with traditional methods.The virtual simulation experiment was conducted using synthesized stereo image sequences based on 6-DOF motion platform and the experimental results proved the validity of our approach and showed that the translation and rotation precision is up to 0.1 mm and 0.1°.

Optical imaging technology in colonoscopy:Is there a role for photometric stereo?

Colonoscopy screening for the detection and removal of colonic adenomas is central to efforts to reduce the morbidity and mortality of colorectal cancer.However,up to a third of adenomas may be missed at colonoscopy,and the majority of post-colonoscopy colorectal cancers are thought to arise from these.Adenomas have three-dimensional surface topographic features that differentiate them from adjacent normal mucosa.However,these topographic features are not enhanced by white light colonoscopy,and the endoscopist must infer these from two-dimensional cues.This may contribute to the number of missed lesions.A variety of optical imaging technologies have been developed commercially to enhance surface topography.However,existing techniques enhance surface topography indirectly,and in two dimensions,and the evidence does not wholly support their use in routine clinical practice.In this narrative review,co-authored by gastroenterologists and engineers,we summarise the evidence for the impact of established optical imaging technologies on adenoma detection rate,and review the development of photometric stereo(PS)for colonoscopy.PS is a machine vision technique able to capture a dense array of surface normals to render three-dimensional reconstructions of surface topography.This imaging technique has several potential clinical applications in colonoscopy,including adenoma detection,polyp classification,and facilitating polypectomy,an inherently three-dimensional task.However,the development of PS for colonoscopy is at an early stage.We consider the progress that has been made with PS to date and identify the obstacles that need to be overcome prior to clinical application.

Shape recovery using high dynamic range images

An effective method for object shape recovery using HDRIs （high dynamic range images） is proposed. The radiance values of each point on the reference sphere and target object are firstly calculated, thus the set of candidate normals of each target point are found by comparing its radiance to that of each reference sphere point. In single-image shape recovery, a smoothness operation is applied to the target normals to obtain a stable and reasonable result; while in photometric stereo, radiance vectors of reference and target objects formed due to illuminations under different fight source directions are directly compared to get the most suitable target normals. Finally, the height values can be recovered from the resulting normal field. Because diffuse and specular reflection are handled in an unified framework with radiance, our approach eliminates the limitation presented in most recovery strategies, i.e., only Lambertian model can be used. The experiment results from the real and synthesized images show the performance of our approach.

No-Reference Stereo Image Quality Assessment Based on Transfer Learning

In order to apply the deep learning to the stereo image quality evaluation,two problems need to be solved:The first one is that we have a bit of training samples,another is how to input the dimensional image’s left view or right view.In this paper,we transfer the 2D image quality evaluation model to the stereo image quality evaluation,and this method solves the first problem;use the method of principal component analysis is used to fuse the left and right views into an input image in order to solve the second problem.At the same time,the input image is preprocessed by phase congruency transformation,which further improves the performance of the algorithm.The structure of the deep convolution neural network consists of four convolution layers and three maximum pooling layers and two fully connected layers.The experimental results on LIVE3D image database show that the prediction quality score of the model is in good agreement with the subjective evaluation value.

可见光光色对南极磷虾集群行为的影响

南极磷虾为南极海洋生态系统中的关键物种,但有关南极磷虾反馈光色的机制研究极为缺乏。为了加深对该物种生态学的理解,本研究通过海上现场水箱暂养的方式探究了可见光光色对南极磷虾集群行为的影响。利用光学立体相机同步获取磷虾集群影像,借助摄影测量技术完成南极磷虾个体关键特征点几何信息的高精度自动量测,进而得到可见光中短波蓝光至长波红光间不同光色下集群内个体间距及其游泳倾角。结果显示,磷虾群的绝对个体间距为(14.51±9.71)cm,相对个体间距为(2.84±1.51)BL,不同光照下的游泳角度为(46.08±35.50)°,并且主要集中在60°左右。研究表明,光色会影响南极磷虾集群内个体的间距,短波蓝光和长波红光间具有显著差异,相较于长波的红光,蓝光的趋向性更佳,但游泳倾角在不同的光色间无显著差异。实验结果将有助于理解野外南极磷虾集群的大小规模以及提升声学评估中目标强度(TS)模型的精度。

电力变压器关键尺寸视觉检测方法及其缩比模型验证

电力变压器的关键尺寸测量是其装配制造、运输安装过程中重要一环,现有测量方法操作繁琐且效率较低。为此,本文提出了一种适用于110 kV油浸式电力变压器关键尺寸视觉检测方法,该方法利用YOLO目标检测算法、Grabcut图像分割算法实现关键组部件的智能检测与分割,然后基于双目立体视觉原理实现套管相间距离及变压器最大截面外形尺寸等关键尺寸的测量。本文搭建了110 kV油浸式电力变压器外观缩比模型,试验分析了拍摄距离和角度等因素对电力变压器关键尺寸视觉检测的影响。结果表明,本文基于缩比模型试验实现了在不同拍摄距离与角度下变压器关键尺寸的视觉检测,验证了该方法的有效性。本文方法可以为110 kV油浸式电力变压器现场尺寸测量提供参考。

基于转像激光立体视觉的油管接箍内螺纹参数检测

油管接箍是油田钻井用重要连接工具,其内螺纹质量对于油气安全开采至关重要。针对现有油管接箍内螺纹接触式检测方法效率低、磨损大、光谱共焦等非接触式方法测量可达性差的问题,提出基于转像激光立体视觉的油管接箍内螺纹参数检测系统与方法,利用激光和反射镜实现内螺纹牙型轮廓的强化和转角窥视,借助已标定的相机模型参数,通过立体视觉和旋转进给重构纵截面中双侧牙型的三维点云,经进一步处理,从三维点云中计算出内螺纹螺距、齿高和锥度三项几何参数。依据美国石油协会(API)标准规定的参数容限范围实现油管接箍内螺纹质量的判定,研制出视觉系统,并与传统量具在相同工况下测量27/8与31/2英寸两种规格的油管内螺纹几何参数,结果表明,提出的视觉系统可实现内螺纹多项参数的一体化高精、高效测量,满足工业现场需求。

基于双目视觉的柔性塑胶套壁厚检测系统研究

为了完成锥形波纹管形状的柔性塑胶防尘套壁厚的检测,建立一个基于双目立体视觉模型的测量系统。该系统中的两个工业数字摄像机通过PLC给予的外触发对工件同时进行图像采集,对采集的多组图像进行相应的预处理后,采用特征点提取方法实现图像拼接,运用特征匹配算法提取出相应待测点三维坐标信息,完成柔性塑胶防尘套壁厚检测。该测量系统,使用气动柔性夹爪夹取工件,减少柔性工件的夹取变形影响,使用高精度移动滑台带动工件运动到拍照位置,通过控制滑台特定的移动距离和对图形特征提取结合方法对多次拍照后的工件图像进行特征点筛选和图像拼接,完成工件壁厚的高精度检测。实验数据表明,该测量系统绝对检测精度达±0.05mm以内,简单快捷,缩短了工件生产工作中的设备调整时间。

基于轻量化PASMnet神经网络的立体匹配方法

针对传统方法用于立体匹配、三维地形重建效果差,而深度学习的方法模型大、速度慢的问题,提出了一种轻量化PASMnet神经网络的立体匹配方法。首先,对PASMnet进行结构化剪枝,精化参数配置,减少模型参数数量,提升模型运行速度。然后对归一化后的图像进行极线校正,得到成像平面平行的立体影像,进行立体匹配任务。在KITTI2015数据集上进行验证,实验结果表明改进的模型参数量减少为原模型的1/6,运行时间为0.56 s。剪枝量化后的模型运行速度更快,推理时间缩短,并且可以用于月面影像的立体匹配,拓展了模型应用。

高山峡谷多视卫星立体像对真三维重建

为解决大面积艰险条件下高山峡谷真三维重建难题,提高卫星影像空间及光谱分辨率,以中国西藏某高山峡谷流域Pleiades卫星立体像对为切入点,探讨了RPC轨道参数校正、SIFT算子优化密集匹配等卫星影像三维重建关键技术,构建无控定位条件下全色、多光谱及融合状态下真三维重建模型。结果表明:3种方案下多视卫星立体像对匹配点误差均小于1个像素,全色方案匹配点精度达到0.67个像素,丰富了1∶5千、1∶1万地形图测绘、地质灾害早期识别及监测、高海拔地区冰川演变分析手段及方法。研究成果可为卫星影像真三维重建提供借鉴。