Imaging simulation and analysis of attitude jitter effect on topographic mapping for lunar orbiter stereo optical cameras

The geometric accuracy of topographic mapping with high-resolution remote sensing images is inevita-bly affected by the orbiter attitude jitter.Therefore,it is necessary to conduct preliminary research on the stereo mapping camera equipped on lunar orbiter before launching.In this work,an imaging simulation method consid-ering the attitude jitter is presented.The impact analysis of different attitude jitter on terrain undulation is conduct-ed by simulating jitter at three attitude angles,respectively.The proposed simulation method is based on the rigor-ous sensor model,using the lunar digital elevation model(DEM)and orthoimage as reference data.The orbit and attitude of the lunar stereo mapping camera are simulated while considering the attitude jitter.Two-dimensional simulated stereo images are generated according to the position and attitude of the orbiter in a given orbit.Experi-mental analyses were conducted by the DEM with the simulated stereo image.The simulation imaging results demonstrate that the proposed method can ensure imaging efficiency without losing the accuracy of topographic mapping.The effect of attitude jitter on the stereo mapping accuracy of the simulated images was analyzed through a DEM comparison.

A Skeletal Camera Network for Close-range Images with a Data Driven Approach in Analyzing Stereo Configuration

Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.

Comparison between Plane and Stereo Block Adjustment for ZY-3 Satellite Images

Block adjustment for satellite images cannot be solved with weak convergence geometric conditions,therefore a plane block adjustment method to improve the targeting precision of images is proposed utilizing DEM as height constraint plane block adjustment method.First,a rational function model with affine transformation is selected as the mathematical model of the satellite image plane block adjustment.Second,to update the ground coordinates of tie points(TPs),the plane coordinates of TPs are only solved in the adjustment process.Elevation values are obtained by using DEM interpolation.Finally,the plane coordinates of all TPs and orientation parameters of all satellite images are solved through plane block adjustment with a few ground control points ZY-3 nadir images for two regions are tested for plane block adjustment while ZY-3 forward-nadir-back images of the same two regions are tested for stereo block adjustment.A comparison indicates that almost the same accuracy can be obtained with plane block adjustment support using a 1∶50 000 DEM and stereo block adjustment for ZY-3 images.For ZY-3 nadir images,almost no loss of plane block adjustment accuracy occurred when global DEM with 1 km grid and SRTM with 90 m grid replaced the 1∶50 000 DEM as elevation control,.Test results demonstrate the effectiveness and feasibility of the plane block adjustment method.

Stereo particle image velocimetry measurement of 3D soil deformation around laterally loaded pile in sand

A developed stereo particle image velocimetry(stereo-PIV) system was proposed to measure three-dimensional(3D) soil deformation around a laterally loaded pile in sand.The stereo-PIV technique extended 2D measurement to 3D based on a binocular vision model,where two cameras with a well geometrical setting were utilized to image the same object simultaneously.This system utilized two open software packages and some simple programs in MATLAB,which can easily be adjusted to meet user needs at a low cost.The failure planes form an angle with the horizontal line,which are measured at 27°-29°,approximately three-fourths of the frictional angle of soil.The edge of the strain wedge formed in front of the pile is an arc,which is slightly different from the straight line reported in the literature.The active and passive influence zones are about twice and six times of the diameter of the pile,respectively.The test demonstrates the good performance and feasibility of this stereo-PIV system for more advanced geotechnical testing.

Optimization of the forearm angle for arm wrestling using multi-camera stereo digital image correlation: A preliminary study

This study analyzes the function of different muscles during arm wrestling and proposes a method to analyze the optimal forearm angle for professional arm wrestlers.We built a professional arm-wrestling platform to measure the shape and deformation of the skin at the biceps brachii of a volunteer in vivo during arm wrestling.We observed the banding phenomenon of arm skin strain during muscle contraction and developed a model to evaluate the moment provided by the biceps brachii.According to this model,the strain field of the area of interest on the skin was measured,and the forearm angles most favorable and unfavorable to the work of the biceps brachii were analyzed.This study demonstrates the considerable potential of applying DIC and its extension method to the in vivo measurement of human skin and facilitates the use of the in vivo measurement of skin deformation in various sports in the future.

Disparity-Based Generation of Line-of-Sight DSM for Image-Elevation Co-Registration to Support Building Detection in Off-Nadir VHR Satellite Images

The integration of optical images and elevation data is of great importance for 3D-assisted mapping applications. Very high resolution (VHR) satellite images provide ideal geo-data for mapping building information. Since buildings are inherently elevated objects, these images need to be co-registered with their elevation data for reliable building detection results. However, accurate co-registration is extremely difficult for off-nadir VHR images acquired over dense urban areas. Therefore, this research proposes a Disparity-Based Elevation Co-Registration (DECR) method for generating a Line-of-Sight Digital Surface Model (LoS-DSM) to efficiently achieve image-elevation data co-registration with pixel-level accuracy. Relative to the traditional photogrammetric approach, the RMSE value of the derived elevations is found to be less than 2 pixels. The applicability of the DECR method is demonstrated through elevation-based building detection (EBD) in a challenging dense urban area. The quality of the detection result is found to be more than 90%. Additionally, the detected objects were geo-referenced successfully to their correct ground locations to allow direct integration with other maps. In comparison to the original LoS-DSM development algorithm, the DECR algorithm is more efficient by reducing the calculation steps, preserving the co-registration accuracy, and minimizing the need for elevation normalization in dense urban areas.

Measurement of 3-D motion parameters of jacket launch based on stereo image sequences

To make sure that the process of jacket launch occurs in a seml-controlled manner, this paper deals with measurement of kinematic parameters of jacket launch using stereo vision and motion analysis. The system captured stereo image sequences by two separate CCD cameras, and then rebuilt 3D coordinates of the feature points to analyze the jacket launch motion. The possibility of combining stereo vision and motion analysis for measurement was examined. Resuhs by experiments using scale model of jacket confirm the theoretical data.

SAR STEREO IMAGING ALGORITHM BASED ON CAPON SPECTRUM ESTIMATION

This letter presents a novel approach for the Synthetic Aperture Radar (SAR) stereo imaging based on the Capon spectrum estimation technique. In order to deal with nonuniform sampling space and lead to super resolution in the elevation direction, Capon approach is used to focus the SAR data on a certain height. Results obtained on simulated data demonstrate the feasibility of the Capon based algorithm. Compared with the classical Fast Fourier Transform (FFT), the Capon based algo-rithm shows better resolution quality.

An Adaptive and Image-guided Fusion for Stereo Satellite Image Derived Digital Surface Models

The accuracy of Digital Surface Models(DSMs)generated using stereo matching methods varies due to the varying acquisition conditions and configuration parameters of stereo images.It has been a good practice to fuse these DSMs generated from various stereo pairs to achieve enhanced,in which multiple DSMs are combined through computational approaches into a single,more accurate,and complete DSM.However,accurately characterizing detailed objects and their boundaries still present a challenge since most boundary-ware fusion methods still struggle to achieve sharpened depth discontinuities due to the averaging effects of different DSMs.Therefore,we propose a simple and efficient adaptive image-guided DSM fusion method that applies k-means clustering on small patches of the orthophoto to guide the pixel-level fusion adapted to the most consistent and relevant elevation points.The experiment results show that our proposed method has outperformed comparing methods in accuracy and the ability to preserve sharpened depth edges.

Wavelet-Based Mixed-Resolution Coding Approach Incorporating with SPT for the Stereo Image

With the advances of display technology, three-dimensional(3-D) imaging systems are becoming increasingly popular. One way of stimulating 3-D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pairs, data compression algorithms should be employed to represent stereo pairs efficiently. The proposed techniques generally use block-based disparity compensation. In order to get the higher compression ratio, this paper employs the wavelet-based mixed-resolution coding technique to incorporate with SPT-based disparity-compensation to compress the stereo image data. The mixed-resolution coding is a perceptually justified technique that is achieved by presenting one eye with a low-resolution image and the other with a high-resolution image. Psychophysical experiments show that the stereo image pairs with one high-resolution image and one low-resolution image provide almost the same stereo depth to that of a stereo image with two high-resolution images. By combining the mixed-resolution coding and SPT-based disparity-compensation techniques, one reference (left) high-resolution image can be compressed by a hierarchical wavelet transform followed by vector quantization and Huffman encoder. After two level wavelet decompositions, for the low-resolution right image and low-resolution left image, subspace projection technique using the fixed block size disparity compensation estimation is used. At the decoder, the low-resolution right subimage is estimated using the disparity from the low-resolution left subimage. A full-size reconstruction is obtained by upsampling a factor of 4 and reconstructing with the synthesis low pass filter. Finally, experimental results are presented, which show that our scheme achieves a PSNR gain (about 0.92dB) as compared to the current block-based disparity compensation coding techniques.

Three-dimensional motion parameters estimation from stereo image sequences based on infrared reflective markers

In this paper,an innovative 3D motion parameters estimation method from stereo image sequences based on infrared(IR) reflective markers is presented.It was assumed that two high speed CCD cameras had been calibrated previously.The method consists of the following steps:1) the coordinate of several markers and depth map for each stereo pair was determined from the sequences of stereo images by relations of markers' coordinate the correspondence between markers was established,2) the 3D motion parameters of the target was computed based upon the matched markers' coordinate,and 3) translated 3D motion parameters estimation into the problem of least square according to the movement model of the object to be measured.Without using line,curve or corner correspondence,this method can calculate the depth of these markers feature easily and quickly in contrast to traditional approaches.The two CCD cameras work on 200 f/s,and each processing cost time is about 3 ms.It was found that,by using several markers and a large number of stereo images,this method can improve the computational speed,robustness and numerical accuracy of the motion parameters in comparison with traditional methods.The virtual simulation experiment was conducted using synthesized stereo image sequences based on 6-DOF motion platform and the experimental results proved the validity of our approach and showed that the translation and rotation precision is up to 0.1 mm and 0.1°.

Optical imaging technology in colonoscopy:Is there a role for photometric stereo?

Colonoscopy screening for the detection and removal of colonic adenomas is central to efforts to reduce the morbidity and mortality of colorectal cancer.However,up to a third of adenomas may be missed at colonoscopy,and the majority of post-colonoscopy colorectal cancers are thought to arise from these.Adenomas have three-dimensional surface topographic features that differentiate them from adjacent normal mucosa.However,these topographic features are not enhanced by white light colonoscopy,and the endoscopist must infer these from two-dimensional cues.This may contribute to the number of missed lesions.A variety of optical imaging technologies have been developed commercially to enhance surface topography.However,existing techniques enhance surface topography indirectly,and in two dimensions,and the evidence does not wholly support their use in routine clinical practice.In this narrative review,co-authored by gastroenterologists and engineers,we summarise the evidence for the impact of established optical imaging technologies on adenoma detection rate,and review the development of photometric stereo(PS)for colonoscopy.PS is a machine vision technique able to capture a dense array of surface normals to render three-dimensional reconstructions of surface topography.This imaging technique has several potential clinical applications in colonoscopy,including adenoma detection,polyp classification,and facilitating polypectomy,an inherently three-dimensional task.However,the development of PS for colonoscopy is at an early stage.We consider the progress that has been made with PS to date and identify the obstacles that need to be overcome prior to clinical application.

基于Stereo Camera-YOLOv5自然环境下百香果检测与定位模型

针对百香果采摘机器人在自然环境中作业时受复杂光线及遮挡影响,难以快速精确地检测及定位成熟百香果的问题,提出一种基于Stereo Camera-YOLOv5自然环境下成熟百香果检测及定位模型。针对自然环境下光线以及遮挡的影响,通过MSRCP算法、随机遮挡、数据增扩等图像处理算法对原始数据集进行优化。将优化的数据集输入到YOLOv5网络中训练出最优模型,在检测代码中嵌入双目立体视觉算法。该模型对自然环境下百香果进行检测及成熟度判断,将判断为成熟的百香果进行图像处理,并提取到中心点二维坐标。通过立体匹配及视差计算得到中心点的三维坐标。田间试验结果表明,该模型的目标检测准确率为97.8%,总体准确率为90.2%,平均运行时间为4.85 s。该系统鲁棒性强、实时性好,能够更好地实现自然环境下成熟百香果的检测及定位,为百香果采摘机器人后续工作奠定基础。

Shape recovery using high dynamic range images

An effective method for object shape recovery using HDRIs （high dynamic range images） is proposed. The radiance values of each point on the reference sphere and target object are firstly calculated, thus the set of candidate normals of each target point are found by comparing its radiance to that of each reference sphere point. In single-image shape recovery, a smoothness operation is applied to the target normals to obtain a stable and reasonable result; while in photometric stereo, radiance vectors of reference and target objects formed due to illuminations under different fight source directions are directly compared to get the most suitable target normals. Finally, the height values can be recovered from the resulting normal field. Because diffuse and specular reflection are handled in an unified framework with radiance, our approach eliminates the limitation presented in most recovery strategies, i.e., only Lambertian model can be used. The experiment results from the real and synthesized images show the performance of our approach.

No-Reference Stereo Image Quality Assessment Based on Transfer Learning

In order to apply the deep learning to the stereo image quality evaluation,two problems need to be solved:The first one is that we have a bit of training samples,another is how to input the dimensional image’s left view or right view.In this paper,we transfer the 2D image quality evaluation model to the stereo image quality evaluation,and this method solves the first problem;use the method of principal component analysis is used to fuse the left and right views into an input image in order to solve the second problem.At the same time,the input image is preprocessed by phase congruency transformation,which further improves the performance of the algorithm.The structure of the deep convolution neural network consists of four convolution layers and three maximum pooling layers and two fully connected layers.The experimental results on LIVE3D image database show that the prediction quality score of the model is in good agreement with the subjective evaluation value.

一种基于Integral Imaging和多基线立体匹配算法的深度测量方法

Integral Imaging(II)是一种能够记录和显示全真三维场景的图像技术.该技术采用微透镜阵列记录空间场景,空间任意一点的深度信息只需通过一次成像即可直接获得.本文研究结合多基线立体匹配算法采用II直接获取物体空间信息的方法.其结果既可为下一代基于II的三维电视图像的数据处理提供应用基础,同时也可望应用于开发新型的深度测量工具.

Real-Time Characterization of Crystal Shape and Size Distribution Based on Moving Window and 3D Imaging in a Stirred Tank

Crystal shape distribution, i.e. the multidimensional size distribution of crystals, is of great importance to their down-stream processing such as in filtration as well as to the end-use properties including the dissolution rate and bioavailability for crystalline pharmaceuticals. Engineering crystal shape and shape distribution requires knowledge about the growth behavior of different crystal facets under varied operational conditions e.g. supersaturations. Measurement of the facet growth rates and growth kinetics of static crystals in a crystallizer without stirring has been reported previously. Here attention is given to study on real-time characterization of the 3D facet growth behavior of crystals in a stirred tank where crystals are constantly moving and rotating. The measurement technique is stereo imaging and the crystal shape reconstruction is based on a stereo imaging camera model. By reference to a case study on potash alum crystallization, it is demonstrated that the crystal size and shape distributions (CSSD) of moving and rotating potash alum crystals in the solution can be reconstructed. The moving window approach was used to correlate 3D face growth kinetics with supersaturation (in the range 0.04 - 0.12) given by an ATR FTIR probe. It revealed that {100} is the fastest growing face, leading to a rapid reduction of its area, while the {111} face has the slowest growth rate, reflected in its area continuously getting larger.

Microlens Light Field Imaging Method Based on Bionic Vision and 3-3 Dimensional Information Transforming

his paper adopts the 3-3-2 information processing method for the capture of moving objects as its premise, and proposes a basic principle of three-dimensional (3D) imaging using biological compound eye. Traditional bionic vision is limited by the available hardware. Therefore, in this paper, the new-generation technology of microlens-array light-field camera is proposed as a potential method for the extraction of depth information from a single image. A significant characteristic of light-field imaging is that it records intensity and directional information from the lights entering the camera. Herein, a refocusing method using light-field image is proposed. By calculating the focusing cost at different depths from the object, the imaging plane of the object is determined, and a depth map is constructed based on the position of the object’s imaging plane. Compared with traditional light-field depth estimation, the depth map calculated by this method can significantly improve resolution and does not depend on the number of light-field microlenses. In addition, considering that software algorithms rely on hardware structure, this study develops an imaging hardware that is only 7 cm long based on the second-generation microlens camera’s structure, further validating its important refocusing characteristics. It thereby provides a technical foundation for 3D imaging with a single camera.

Development of a Stereo Vision-based Pick and Place System for Robotic Manipulators

Conventional robotic manipulators consist of touch and vision sensors in order to pick and place differently shaped objects.Due to the technology development and degrading sensors over a long period,the stereo vision technique has become a promising alternative.In this study,a low-cost stereo vision-based system,and a gripper to be placed at the end of the robot arm(Fanuc M10 iA/12)are developed for position and orientation estimation of robotic manipulators to pick and place different shaped objects.The stereo vision system developed in this research is used to estimate the position(X,Y,Z),orientation(P_(y))of the Center of Volume of four standard objects(cube,cuboid,cylinder,and sphere)whereas the robot arm with the gripper is used to mechanically pick and place the objects.The stereo vision system is placed on the movable robot arm,and it consists of two cameras to capture two 2D views of a stationary object to derive 3D depth information in 3D space.Moreover,a graphical user interface is developed to train a linear regression model,live predict the coordinates of the objects,and check the accuracy of the predicted data.The graphical user interface can also send predicted coordinates and angles to the gripper and the robot arm.The project is facilitated with python programming language modules and image processing techniques.Identification of the stationary object and estimation of its coordinates is done using image processing techniques.The final product can be identified as a device that converts conventional robot arms without an image processing vision system into a highly precise and accurate robot arm with an image processing vision system.Experimental studies are performed to test the efficiency and effectiveness of used techniques and the gripper prototype.Necessary actions are taken to minimize the errors in position and orientation estimation.In addition,as a future implementation,an embedded system will be developed with a user-friendly software interface to install the vision system into the Fanuc M10 iA/12 robot arm and will upgrade the system to a device that can be implemented with any kind of customized robot arms available in the industry.

Fast and Robust Stereo Vision Algorithm for Obstacle Detection

Binocular computer vision is based on bionics, after the calibration through the camera head by double-exposure image synchronization, access to the calculation of two-dimensional image pixels of the three-dimensional depth information. In this paper, a fast and robust stereo vision algorithm is described to perform in-vehicle obstacles detection and characterization. The stereo algorithm which provides a suitable representation of the geometric content of the road scene is described, and an in-vehicle embedded system is presented. We present the way in which the algorithm is used, and then report experiments on real situations which show that our solution is accurate, reliable and efficient. In particular, both processes are fast, generic, robust to noise and bad conditions, and work even with partial occlusion.