Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision...Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision-based algorithms. In this paper we propose a novel approach to address the problem of rolling shutter distortion in aerial imaging. A mathematical model is established by the coordinate transformation method. It can directly calculate the pixel distortion when an aerial camera is imaging at arbitrary gesture angles.Then all pixel distortions form a distortion map over the whole CMOS array and the map is exploited in the image rectification process incorporating reverse projection. The error analysis indicates that within the margin of measuring errors,the final calculation error of our model is less than 1/2 pixel. The experimental results show that our approach yields good rectification performance in a series of images with different distortions. We demonstrate that our method outperforms other vision-based algorithms in terms of the computational complexity,which makes it more suitable for aerial real-time imaging.展开更多
The star identification algorithm usually identifies stars by angular distance matching.However,under high dynamic conditions,the rolling shutter effect distorts the angular distances between the measured and true sta...The star identification algorithm usually identifies stars by angular distance matching.However,under high dynamic conditions,the rolling shutter effect distorts the angular distances between the measured and true star positions,leading to plethoric false matches and requiring complex and time-consuming verification for star identification.Low identification rate hinders the application of low-noise and cost-effective rolling shutter image sensors.In this work,we first study a rolling shutter distortion model of angular distances between stars,and then propose a novel three-stage star identification algorithm to identify distorted star images captured by the rolling shutter star sensor.The first stage uses a modified grid algorithm with adaptive error tolerance and an expanded pattern database to efficiently eliminate spurious matches.The second stage performs angular velocity estimation based on Hough transform to verify the matches that follow the same distortion pattern.The third stage applies a rolling shutter error correction method for further verification.Both the simulation and night sky image test demonstrate the effectiveness and efficiency of our algorithm under high dynamic conditions.The accuracy of angular velocity estimation method by Hough transform is evaluated and the root mean square error is below 0.5(°)/s.Our algorithm achieves a 95.7% identification rate at an angular velocity of 10(°)/s,which is much higher than traditional algorithms.展开更多
Although optical camera communication(OCC) is gaining increasing attention in research, developing a practical OCC system to increase data rate and transmission distance is still an issue. The rolling shutter can incr...Although optical camera communication(OCC) is gaining increasing attention in research, developing a practical OCC system to increase data rate and transmission distance is still an issue. The rolling shutter can increase data rate, but it also limits the transmission distance at the same time. When the transmission distance is long, the thresholding method will become difficult and the block sequence number(BSN) is easy to be wrong due to bit errors, which will make it impossible to reorder the frame sequences. In this paper, in order to increase the transmission distance, we propose the efficient thresholding method which transforms partial mean gray values of per column nonlinearly to increase the contrast between light and dark bands. In addition, this method takes the frame image of the camera as the basic unit to adjust the threshold for the environment in which each frame of the image is located. Experimental results show that this method is better than the polynomial regression. Moreover, we perform the error correction of BSN by using its continuity to specify the legal BSNs and the illegal BSNs.展开更多
Most modern consumer-grade cameras are often equipped with a rolling shutter mechanism,which is becoming increasingly important in computer vision,robotics and autonomous driving applications.However,its temporal-dyna...Most modern consumer-grade cameras are often equipped with a rolling shutter mechanism,which is becoming increasingly important in computer vision,robotics and autonomous driving applications.However,its temporal-dynamic imaging nature leads to the rolling shutter effect that manifests as geometric distortion.Over the years,researchers have made significant progress in developing tractable rolling shutter models,optimization methods,and learning approaches,aiming to remove geometry distortion and improve visual quality.In this survey,we review the recent advances in rolling shutter cameras from two aspects of motion modeling and deep learning.To the best of our knowledge,this is the first comprehensive survey of rolling shutter cameras.In the part of rolling shutter motion modeling and optimization,the principles of various rolling shutter motion models are elaborated and their typical applications are summarized.Then,the applications of deep learning in rolling shutter based image processing are presented.Finally,we conclude this survey with discussions on future research directions.展开更多
为了对高速公路车辆速度进行限制,建立了基于卷帘式CMOS(Complementary Metal Oxide Semiconductor)图像传感器的相机测速系统,并对该系统采用的标定技术、测速算法等进行了研究。卷帘式快门数字相机测量速度是利用卷帘式快门曝光,具有...为了对高速公路车辆速度进行限制,建立了基于卷帘式CMOS(Complementary Metal Oxide Semiconductor)图像传感器的相机测速系统,并对该系统采用的标定技术、测速算法等进行了研究。卷帘式快门数字相机测量速度是利用卷帘式快门曝光,具有一定的延时,对运动物体拍摄会产生一定的畸变,通过对图像的畸变部分进行图像处理和计算,即可得到运动物体的速度参数。根据相机成像的几何原理,分析了相机的标定原理,根据相机测速时的特点,提出了简单、有效的标定方法。进一步研究了相机标定和测速的实验方法和步骤,进行了标定及测速实验。经实验验证:利用卷帘式快门CMOS数字相机方法对运动目标测速值较精确,误差在2.5%以内。基本满足高速公路相机测速系统精度要求。展开更多
基金Sponsored by the National Natural Science Foundation of China(Grant No.60902067)the Foundation for Science & Technology Research Project of Jilin Province(Grant No.11ZDGG001)
文摘Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision-based algorithms. In this paper we propose a novel approach to address the problem of rolling shutter distortion in aerial imaging. A mathematical model is established by the coordinate transformation method. It can directly calculate the pixel distortion when an aerial camera is imaging at arbitrary gesture angles.Then all pixel distortions form a distortion map over the whole CMOS array and the map is exploited in the image rectification process incorporating reverse projection. The error analysis indicates that within the margin of measuring errors,the final calculation error of our model is less than 1/2 pixel. The experimental results show that our approach yields good rectification performance in a series of images with different distortions. We demonstrate that our method outperforms other vision-based algorithms in terms of the computational complexity,which makes it more suitable for aerial real-time imaging.
基金supported by the National Key Research and Development Program of China(No.2019YFA0706002).
文摘The star identification algorithm usually identifies stars by angular distance matching.However,under high dynamic conditions,the rolling shutter effect distorts the angular distances between the measured and true star positions,leading to plethoric false matches and requiring complex and time-consuming verification for star identification.Low identification rate hinders the application of low-noise and cost-effective rolling shutter image sensors.In this work,we first study a rolling shutter distortion model of angular distances between stars,and then propose a novel three-stage star identification algorithm to identify distorted star images captured by the rolling shutter star sensor.The first stage uses a modified grid algorithm with adaptive error tolerance and an expanded pattern database to efficiently eliminate spurious matches.The second stage performs angular velocity estimation based on Hough transform to verify the matches that follow the same distortion pattern.The third stage applies a rolling shutter error correction method for further verification.Both the simulation and night sky image test demonstrate the effectiveness and efficiency of our algorithm under high dynamic conditions.The accuracy of angular velocity estimation method by Hough transform is evaluated and the root mean square error is below 0.5(°)/s.Our algorithm achieves a 95.7% identification rate at an angular velocity of 10(°)/s,which is much higher than traditional algorithms.
基金supported by the Department of Science and Technology of Jilin Province(No.20180101040JC)
文摘Although optical camera communication(OCC) is gaining increasing attention in research, developing a practical OCC system to increase data rate and transmission distance is still an issue. The rolling shutter can increase data rate, but it also limits the transmission distance at the same time. When the transmission distance is long, the thresholding method will become difficult and the block sequence number(BSN) is easy to be wrong due to bit errors, which will make it impossible to reorder the frame sequences. In this paper, in order to increase the transmission distance, we propose the efficient thresholding method which transforms partial mean gray values of per column nonlinearly to increase the contrast between light and dark bands. In addition, this method takes the frame image of the camera as the basic unit to adjust the threshold for the environment in which each frame of the image is located. Experimental results show that this method is better than the polynomial regression. Moreover, we perform the error correction of BSN by using its continuity to specify the legal BSNs and the illegal BSNs.
基金This work was supported in part by National Natural Science Foundation of China(Nos.62271410,61901387 and 62001394)the Fundamental Research Funds for the Central Universities,China,and the Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University,China(No.CX2022046).
文摘Most modern consumer-grade cameras are often equipped with a rolling shutter mechanism,which is becoming increasingly important in computer vision,robotics and autonomous driving applications.However,its temporal-dynamic imaging nature leads to the rolling shutter effect that manifests as geometric distortion.Over the years,researchers have made significant progress in developing tractable rolling shutter models,optimization methods,and learning approaches,aiming to remove geometry distortion and improve visual quality.In this survey,we review the recent advances in rolling shutter cameras from two aspects of motion modeling and deep learning.To the best of our knowledge,this is the first comprehensive survey of rolling shutter cameras.In the part of rolling shutter motion modeling and optimization,the principles of various rolling shutter motion models are elaborated and their typical applications are summarized.Then,the applications of deep learning in rolling shutter based image processing are presented.Finally,we conclude this survey with discussions on future research directions.
文摘为了对高速公路车辆速度进行限制,建立了基于卷帘式CMOS(Complementary Metal Oxide Semiconductor)图像传感器的相机测速系统,并对该系统采用的标定技术、测速算法等进行了研究。卷帘式快门数字相机测量速度是利用卷帘式快门曝光,具有一定的延时,对运动物体拍摄会产生一定的畸变,通过对图像的畸变部分进行图像处理和计算,即可得到运动物体的速度参数。根据相机成像的几何原理,分析了相机的标定原理,根据相机测速时的特点,提出了简单、有效的标定方法。进一步研究了相机标定和测速的实验方法和步骤,进行了标定及测速实验。经实验验证:利用卷帘式快门CMOS数字相机方法对运动目标测速值较精确,误差在2.5%以内。基本满足高速公路相机测速系统精度要求。
