Single-pixel imaging(SPI)can transform 2D or 3D image data into 1D light signals,which offers promising prospects for image compression and transmission.However,during data communication these light signals in public ...Single-pixel imaging(SPI)can transform 2D or 3D image data into 1D light signals,which offers promising prospects for image compression and transmission.However,during data communication these light signals in public channels will easily draw the attention of eavesdroppers.Here,we introduce an efficient encryption method for SPI data transmission that uses the 3D Arnold transformation to directly disrupt 1D single-pixel light signals and utilizes the elliptic curve encryption algorithm for key transmission.This encryption scheme immediately employs Hadamard patterns to illuminate the scene and then utilizes the 3D Arnold transformation to permutate the 1D light signal of single-pixel detection.Then the transformation parameters serve as the secret key,while the security of key exchange is guaranteed by an elliptic curve-based key exchange mechanism.Compared with existing encryption schemes,both computer simulations and optical experiments have been conducted to demonstrate that the proposed technique not only enhances the security of encryption but also eliminates the need for complicated pattern scrambling rules.Additionally,this approach solves the problem of secure key transmission,thus ensuring the security of information and the quality of the decrypted images.展开更多
3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimat...3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimation of monocular RGB images and videos.An overall perspective ofmethods integrated with deep learning is introduced.Novel image-based and video-based inputs are proposed as the analysis framework.From this viewpoint,common problems are discussed.The diversity of human postures usually leads to problems such as occlusion and ambiguity,and the lack of training datasets often results in poor generalization ability of the model.Regression methods are crucial for solving such problems.Considering image-based input,the multi-view method is commonly used to solve occlusion problems.Here,the multi-view method is analyzed comprehensively.By referring to video-based input,the human prior knowledge of restricted motion is used to predict human postures.In addition,structural constraints are widely used as prior knowledge.Furthermore,weakly supervised learningmethods are studied and discussed for these two types of inputs to improve the model generalization ability.The problem of insufficient training datasets must also be considered,especially because 3D datasets are usually biased and limited.Finally,emerging and popular datasets and evaluation indicators are discussed.The characteristics of the datasets and the relationships of the indicators are explained and highlighted.Thus,this article can be useful and instructive for researchers who are lacking in experience and find this field confusing.In addition,by providing an overview of 3D human pose estimation,this article sorts and refines recent studies on 3D human pose estimation.It describes kernel problems and common useful methods,and discusses the scope for further research.展开更多
The acquisition of digital regional-scale information and ecological environmental data has high requirements for structural texture,spatial res-olution,and multiple parameter categories,which is challenging to achiev...The acquisition of digital regional-scale information and ecological environmental data has high requirements for structural texture,spatial res-olution,and multiple parameter categories,which is challenging to achieve using satellite remote sensing.Considering the convenient,facilitative,and flexible characteristics of UAV(unmanned air vehicle)remote sensing tech-nology,this study selects a campus as a typical research area and uses the Pegasus D2000 equipped with a D-MSPC2000 multi-spectral camera and a CAM3000 aerial camera to acquire oblique images and multi-spectral data.Using professional software,including Context Capture,ENVI,and ArcGIS,a 3D(three-dimensional)campus model,a digital orthophoto map,and multi-spectral remote sensing map drawing are realized,and the geometric accuracy of typical feature selection is evaluated.Based on a quantitative remote sensing model,the campus ecological environment assessment is performed from the perspectives of vegetation and water body.The results presented in this study could be of great significance to the scientific management and sustainable development of regional natural resources.展开更多
This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, w...This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using "active shape models", the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints (edge points' distance and color variance). We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments' results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.展开更多
Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting w...Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera.展开更多
Instead of traditionally using a 3D physical model with many control points on it, a calibration plate with printed chess grid and movable along its normal direction is implemented to provide large area 3D control poi...Instead of traditionally using a 3D physical model with many control points on it, a calibration plate with printed chess grid and movable along its normal direction is implemented to provide large area 3D control points with variable Z values. Experiments show that the approach presented is effective for reconstructing 3D color objects in computer vision system.展开更多
This work explores an alternative 3D geometry measurement method for non-cooperative spacecraft guiding navigation and proximity operations.From one snapshot of an unfocused light-field camera, the 3D point cloud of a...This work explores an alternative 3D geometry measurement method for non-cooperative spacecraft guiding navigation and proximity operations.From one snapshot of an unfocused light-field camera, the 3D point cloud of a non-cooperative spacecraft can be calculated from sub-aperture images with the epipolar plane image(EPI) based light-field rendering algorithm.A Chang'e-3 model(7.2 cm×5.6 cm×7.0 cm) is tested to validate the proposed technique.Three measurement distances(1.0 m, 1.2 m, 1.5 m) are considered to simulate different approaching stages.Measuring errors are quantified by comparing the light-field camera data with a high precision commercial laser scanner.The mean error distance for the three cases are 0.837 mm, 0.743 mm, and 0.973 mm respectively, indicating that the method can well reconstruct 3D geometry of a non-cooperative spacecraft with a densely distributed 3D point cloud and is thus promising in space-related missions.展开更多
Crystal shape distribution, i.e. the multidimensional size distribution of crystals, is of great importance to their down-stream processing such as in filtration as well as to the end-use properties including the diss...Crystal shape distribution, i.e. the multidimensional size distribution of crystals, is of great importance to their down-stream processing such as in filtration as well as to the end-use properties including the dissolution rate and bioavailability for crystalline pharmaceuticals. Engineering crystal shape and shape distribution requires knowledge about the growth behavior of different crystal facets under varied operational conditions e.g. supersaturations. Measurement of the facet growth rates and growth kinetics of static crystals in a crystallizer without stirring has been reported previously. Here attention is given to study on real-time characterization of the 3D facet growth behavior of crystals in a stirred tank where crystals are constantly moving and rotating. The measurement technique is stereo imaging and the crystal shape reconstruction is based on a stereo imaging camera model. By reference to a case study on potash alum crystallization, it is demonstrated that the crystal size and shape distributions (CSSD) of moving and rotating potash alum crystals in the solution can be reconstructed. The moving window approach was used to correlate 3D face growth kinetics with supersaturation (in the range 0.04 - 0.12) given by an ATR FTIR probe. It revealed that {100} is the fastest growing face, leading to a rapid reduction of its area, while the {111} face has the slowest growth rate, reflected in its area continuously getting larger.展开更多
The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interaction...The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interactions.With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks(DNNs),numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation.Nonetheless,the existence of complicated hand articulation,depth and scale ambiguities,occlusions,and finger similarity remain challenging.In this study,we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras.Related RGB-D cameras,hand datasets,and a performance analysis are also discussed to provide a holistic view of recent achievements.We also discuss the research potential of this rapidly growing field.展开更多
Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,t...Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.展开更多
The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing.T...The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing.To address this demand,a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera’s trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene.A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process.Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction.To ensure safe and effective movement,two spatial constraints,Domain of View Admissible Position(DVAP)and Domain of View Admissible Orientation(DVAO),are implemented to account for robot reachability and collision avoidance.By using diffeomorphism mapping,the orientation path is transformed into 3D,consistent with the position path.Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO.The reconstruction cell is capable of automatic data acquisition and fine scene modeling,using the generated robot C-space trajectory.Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve highprecision 3D reconstruction while optimizing robot motion quality.展开更多
An effective approach, mapping the texture for building model based on the digital photogrammetric theory, is proposed. The easily-acquired image sequences from digital video camera on helicopter are used as texture r...An effective approach, mapping the texture for building model based on the digital photogrammetric theory, is proposed. The easily-acquired image sequences from digital video camera on helicopter are used as texture resource, and the correspondence between the space edge in building geometry model and its line feature in image sequences is determined semi-automatically. The experimental results in production of three-dimensional data for car navigation show us an attractive future both in efficiency and effect.展开更多
基金Project supported by the National Natural Science Foundation of China(Grant No.62075241).
文摘Single-pixel imaging(SPI)can transform 2D or 3D image data into 1D light signals,which offers promising prospects for image compression and transmission.However,during data communication these light signals in public channels will easily draw the attention of eavesdroppers.Here,we introduce an efficient encryption method for SPI data transmission that uses the 3D Arnold transformation to directly disrupt 1D single-pixel light signals and utilizes the elliptic curve encryption algorithm for key transmission.This encryption scheme immediately employs Hadamard patterns to illuminate the scene and then utilizes the 3D Arnold transformation to permutate the 1D light signal of single-pixel detection.Then the transformation parameters serve as the secret key,while the security of key exchange is guaranteed by an elliptic curve-based key exchange mechanism.Compared with existing encryption schemes,both computer simulations and optical experiments have been conducted to demonstrate that the proposed technique not only enhances the security of encryption but also eliminates the need for complicated pattern scrambling rules.Additionally,this approach solves the problem of secure key transmission,thus ensuring the security of information and the quality of the decrypted images.
基金supported by the Program of Entrepreneurship and Innovation Ph.D.in Jiangsu Province(JSSCBS20211175)the School Ph.D.Talent Funding(Z301B2055)the Natural Science Foundation of the Jiangsu Higher Education Institutions of China(21KJB520002).
文摘3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimation of monocular RGB images and videos.An overall perspective ofmethods integrated with deep learning is introduced.Novel image-based and video-based inputs are proposed as the analysis framework.From this viewpoint,common problems are discussed.The diversity of human postures usually leads to problems such as occlusion and ambiguity,and the lack of training datasets often results in poor generalization ability of the model.Regression methods are crucial for solving such problems.Considering image-based input,the multi-view method is commonly used to solve occlusion problems.Here,the multi-view method is analyzed comprehensively.By referring to video-based input,the human prior knowledge of restricted motion is used to predict human postures.In addition,structural constraints are widely used as prior knowledge.Furthermore,weakly supervised learningmethods are studied and discussed for these two types of inputs to improve the model generalization ability.The problem of insufficient training datasets must also be considered,especially because 3D datasets are usually biased and limited.Finally,emerging and popular datasets and evaluation indicators are discussed.The characteristics of the datasets and the relationships of the indicators are explained and highlighted.Thus,this article can be useful and instructive for researchers who are lacking in experience and find this field confusing.In addition,by providing an overview of 3D human pose estimation,this article sorts and refines recent studies on 3D human pose estimation.It describes kernel problems and common useful methods,and discusses the scope for further research.
基金supported by the National Natural Science Foundation of China (Grant No.42171311)the Open Fund of State Key Laboratory of Remote Sensing Science (Grant No.OFSLRSS202218)+1 种基金the Key Research and Development Program of the Hainan Province,China (Grant No.ZDYF2021SHFZ105)the Training Program of Excellent Master Thesis of Zhejiang Ocean University.
文摘The acquisition of digital regional-scale information and ecological environmental data has high requirements for structural texture,spatial res-olution,and multiple parameter categories,which is challenging to achieve using satellite remote sensing.Considering the convenient,facilitative,and flexible characteristics of UAV(unmanned air vehicle)remote sensing tech-nology,this study selects a campus as a typical research area and uses the Pegasus D2000 equipped with a D-MSPC2000 multi-spectral camera and a CAM3000 aerial camera to acquire oblique images and multi-spectral data.Using professional software,including Context Capture,ENVI,and ArcGIS,a 3D(three-dimensional)campus model,a digital orthophoto map,and multi-spectral remote sensing map drawing are realized,and the geometric accuracy of typical feature selection is evaluated.Based on a quantitative remote sensing model,the campus ecological environment assessment is performed from the perspectives of vegetation and water body.The results presented in this study could be of great significance to the scientific management and sustainable development of regional natural resources.
基金This work was supported by Grant-in-Aid for Scientific Research (C) (No.17500119)
文摘This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using "active shape models", the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints (edge points' distance and color variance). We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments' results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.
基金National Natural Science Foundation of China(61732016).
文摘Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera.
基金Supported by the Natural Science Foundation of China (69775022)the State High-Technology Development program of China(863 306ZT04 06 3)
文摘Instead of traditionally using a 3D physical model with many control points on it, a calibration plate with printed chess grid and movable along its normal direction is implemented to provide large area 3D control points with variable Z values. Experiments show that the approach presented is effective for reconstructing 3D color objects in computer vision system.
文摘This work explores an alternative 3D geometry measurement method for non-cooperative spacecraft guiding navigation and proximity operations.From one snapshot of an unfocused light-field camera, the 3D point cloud of a non-cooperative spacecraft can be calculated from sub-aperture images with the epipolar plane image(EPI) based light-field rendering algorithm.A Chang'e-3 model(7.2 cm×5.6 cm×7.0 cm) is tested to validate the proposed technique.Three measurement distances(1.0 m, 1.2 m, 1.5 m) are considered to simulate different approaching stages.Measuring errors are quantified by comparing the light-field camera data with a high precision commercial laser scanner.The mean error distance for the three cases are 0.837 mm, 0.743 mm, and 0.973 mm respectively, indicating that the method can well reconstruct 3D geometry of a non-cooperative spacecraft with a densely distributed 3D point cloud and is thus promising in space-related missions.
文摘Crystal shape distribution, i.e. the multidimensional size distribution of crystals, is of great importance to their down-stream processing such as in filtration as well as to the end-use properties including the dissolution rate and bioavailability for crystalline pharmaceuticals. Engineering crystal shape and shape distribution requires knowledge about the growth behavior of different crystal facets under varied operational conditions e.g. supersaturations. Measurement of the facet growth rates and growth kinetics of static crystals in a crystallizer without stirring has been reported previously. Here attention is given to study on real-time characterization of the 3D facet growth behavior of crystals in a stirred tank where crystals are constantly moving and rotating. The measurement technique is stereo imaging and the crystal shape reconstruction is based on a stereo imaging camera model. By reference to a case study on potash alum crystallization, it is demonstrated that the crystal size and shape distributions (CSSD) of moving and rotating potash alum crystals in the solution can be reconstructed. The moving window approach was used to correlate 3D face growth kinetics with supersaturation (in the range 0.04 - 0.12) given by an ATR FTIR probe. It revealed that {100} is the fastest growing face, leading to a rapid reduction of its area, while the {111} face has the slowest growth rate, reflected in its area continuously getting larger.
基金the National Key R&D Program of China(2018YFB1004600)the National Natural Science Foundation of China(61502187,61876211)the National Science Foundation Grant CNS(1951952).
文摘The field of vision-based human hand three-dimensional(3D)shape and pose estimation has attracted significant attention recently owing to its key role in various applications,such as natural human computer interactions.With the availability of large-scale annotated hand datasets and the rapid developments of deep neural networks(DNNs),numerous DNN-based data-driven methods have been proposed for accurate and rapid hand shape and pose estimation.Nonetheless,the existence of complicated hand articulation,depth and scale ambiguities,occlusions,and finger similarity remain challenging.In this study,we present a comprehensive survey of state-of-the-art 3D hand shape and pose estimation approaches using RGB-D cameras.Related RGB-D cameras,hand datasets,and a performance analysis are also discussed to provide a holistic view of recent achievements.We also discuss the research potential of this rapidly growing field.
基金National Natural Science Foundation of China(No.41701534)Open Fund of State Key Laboratory of Coal Resources and Safe Mining(No.SKLCRSM19KFA01)+1 种基金Ecological and Smart Mine Joint Foundation of Hebei Province(No.E2020402086)State Key Laboratory ofGeohazard Prevention and Geoenvironment Protection(No.SKLGP2019K015)
文摘Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.
基金supported by the National Key Research and Development Program of China(2019YFB1707505)the National Natural Science Foundation of China(Grant No.52005436)。
文摘The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing.To address this demand,a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera’s trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene.A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process.Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction.To ensure safe and effective movement,two spatial constraints,Domain of View Admissible Position(DVAP)and Domain of View Admissible Orientation(DVAO),are implemented to account for robot reachability and collision avoidance.By using diffeomorphism mapping,the orientation path is transformed into 3D,consistent with the position path.Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO.The reconstruction cell is capable of automatic data acquisition and fine scene modeling,using the generated robot C-space trajectory.Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve highprecision 3D reconstruction while optimizing robot motion quality.
文摘An effective approach, mapping the texture for building model based on the digital photogrammetric theory, is proposed. The easily-acquired image sequences from digital video camera on helicopter are used as texture resource, and the correspondence between the space edge in building geometry model and its line feature in image sequences is determined semi-automatically. The experimental results in production of three-dimensional data for car navigation show us an attractive future both in efficiency and effect.