Sensor planning method for visual tracking in 3D camera networks

Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.

A Skeletal Camera Network for Close-range Images with a Data Driven Approach in Analyzing Stereo Configuration

Structure-from-Motion(SfM)techniques have been widely used for 3D geometry reconstruction from multi-view images.Nevertheless,the efficiency and quality of the reconstructed geometry depends on multiple factors,i.e.,the base-height ratio,intersection angle,overlap,and ground control points,etc.,which are rarely quantified in real-world applications.To answer this question,in this paper,we take a data-driven approach by analyzing hundreds of terrestrial stereo image configurations through a typical SfM algorithm.Two main meta-parameters with respect to base-height ratio and intersection angle are analyzed.Following the results,we propose a Skeletal Camera Network(SCN)and embed it into the SfM to lead to a novel SfM scheme called SCN-SfM,which limits tie-point matching to the remaining connected image pairs in SCN.The proposed method was applied in three terrestrial datasets.Experimental results have demonstrated the effectiveness of the proposed SCN-SfM to achieve 3D geometry with higher accuracy and fast time efficiency compared to the typical SfM method,whereas the completeness of the geometry is comparable.

Web-Based Object Tracking Using Collaborated Camera Network

The paper presents a web based vision system using a networked IP camera for tracking objects of interest. Three critical issues are addressed in this paper. First is the detection of moving objects in the foreground;second is the control of pan-tilt-zoom (PTZ) IP cameras based on object location;and third is the collaboration of multiple cameras over the network to track objects of interests independently. The developed system utilized a network of PTZ cameras along with a number of software tools for this implementation. The system was able to track a single and multiple objects successfully. The difficulties in the detection of moving objects are also analyzed while multiple cameras are collaborating over a network utilizing PTZ cameras.

Automatic Service Discovery of IP Cameras over Wide Area Networks with NAT Traversal

A novel framework for remote service discovery and access of IP cameras with Network address Translation (NAT) traversal is presented in this paper. The proposed protocol, termed STDP (Service Trader Discovery Protocol), is a hybrid combination of Zeroconf and SIP (Session Initial Protocol). The Zeroconf is adopted for the discovery and/or publication of local services;whereas, the SIP is used for the delivery of local services to the remote nodes. In addition, both the SIP-ALG (Application Layer Gateway) and UPnP (Universal Plug and Play)-IGD (Internet Gateway Device) protocols are used for NAT traversal. The proposed framework is well-suited for high mobility applications where the fast deployment and low administration efforts of IP cameras are desired.

Modeling Camera Image Formation Using a Feedforward Neural Network

One fundamental problem in computer vision and image processing is modeling the image formation of a camera, i.e., mapping a point in three-dimensional space to its projected position on the camera’s image plane. If the relationship between the space and the image plane is assumed to be linear, the relationship can be expressed in terms of a transfor-mation matrix and the matrix is often identified by regression. In this paper, we show that the space-to-image relation-ship in a camera can be modeled by a simple neural network. Unlike most other cases employing neural networks, the structure of the network is optimized so as for each link between neurons to have a physical meaning. This makes it possible to effectively initialize link weights and quickly train the network.

基于Smart Camera的目标识别和加密传输

提出用一种以DSP为硬件核心的智能数字网络摄像机Smart Camera取代传统摄像机。它能够在对监控图像进行目标识别及其信息提取的同时,将特征信息经过RSA加密后通过网络传输,并可根据后方PC机的远程控制,给出不同的报警需求,实现了摄像机的智能化、数字化、网络化以及多用途。

Energy Efficient Content Based Image Retrieval in Sensor Networks

The presence of increased memory and computational power in imaging sensor networks attracts researchers to exploit image processing algorithms on distributed memory and computational power. In this paper, a typical perimeter is investigated with a number of sensors placed to form an image sensor network for the purpose of content based distributed image search. Image search algorithm is used to enable distributed content based image search within each sensor node. The energy model is presented to calculate energy efficiency for various cases of image search and transmission. The simulations are carried out based on consideration of continuous monitoring or event driven activity on the perimeter. The simulation setups consider distributed image processing on sensor nodes and results show that energy saving is significant if search algorithms are embedded in image sensor nodes and image processing is distributed across sensor nodes. The tradeoff between sensor life time, distributed image search and network deployed cost is also investigated.

An Automatic System of Vehicle Number-Plate Recognition Based on Neural Networks

This paper presents an automatic system of vehicle number-plate recognition based on neural networks. In this system, location of number-plate and recognition of characters in number-plate can be automatically completed. Pixel colors of Number-plate area are classified using neural network, then color features are extracted by analyzing scanning lines of the cross-section of number-plate. It takes full use of number-plate color features to locate number-plate. Characters in number-plate can be effectively recognized using the neural networks. Experimental results show that the correct rate of number-plate location is close to 100%, and the time of number-plate location is less than 1 second. Moreover, recognition rate of characters is improved due to the known number-plate type. It is also observed that this system is not sensitive to variations of weather, illumination and vehicle speed. In addition, and also the size of number-plate need not to be known in prior. This system is of crucial significance to apply and spread the automatic system of vehicle number-plate recognition.

In-Motes EYE: A Real Time Application for Automobiles in Wireless Sensor Networks

Wireless sensor networks have been identified as one of the key technologies for the 21st century. In order to overcome their limitations such as fault tolerance and conservation of energy, we propose a middleware solution, In-Motes. In-Motes stands as a fault tolerant platform for deploying and monitoring applications in real time offers a number of possibilities for the end user giving him in parallel the freedom to experiment with various parameters, in an effort the deployed applications to run in an energy efficient manner inside the network. The proposed scheme is evaluated through the In-Motes EYE application, aiming to test its merits under real time conditions. In-Motes EYE application which is an agent based real time In-Motes application developed for sensing acceleration variations in an environment. The application was tested in a prototype area, road alike, for a period of four months.

基于深度相机和神经网络的下肢关节力矩估计

目的 通过深度相机和神经网络估计人在直线行走时髋、膝和踝关节的屈伸力矩。方法 利用光学运动捕捉系统、测力板和Azure Kinect深度相机采集20个人的步态信息,受试者被要求以其偏好的步行速度直线行走,同时踏在测力板上。并利用Visual 3D仿真得到关节力矩作为参考值,分别训练人工神经网络(artificial neural network, ANN)模型与长短期记忆(long short-term memory, LSTM)模型进行关节力矩估计。结果 ANN模型估计髋、膝和踝关节的关节力矩的相对均方根误差(relative root mean square error, rRMSE)分别为15.87%~17.32%、18.36%~25.34%和14.11%~16.82%,相关系数分别为0.81~0.85、0.69~0.74和0.76~0.82。LSTM模型具有更好的估计效果,rRMSE分别为8.53%~12.18%、14.32%~18.78%和6.51%~11.83%,相关系数分别达到了0.89~0.95、0.85~0.91和0.90~0.97。结论 本文证实了利用深度相机和神经网络无接触估计人体下肢关节力矩方案的可行性,其中LSTM模型具有更佳的表现。关节力矩估计结果与现有研究相比具有更好的精度,潜在应用场景包含远程医疗、个性化康复方案制定以及矫形器辅助设计等。

基于相位靶和神经网络的单目相机标定

针对传统基于BP神经网络的标定方法标定精度低,深度方向误差大等问题,提出了一种基于相位靶和径向基函数(RBF)神经网络的单目相机标定方法。采用三步相移法的特征提取方法,并使用多频法计算绝对相位,将相位靶特征点承载的绝对相位转换到三维空间,建立特征点图像坐标与世界坐标之间的对应关系,最后使用RBF神经网络完成二维图像坐标到三维空间坐标的直接映射。实验结果表明,对比传统使用BP神经网络进行标定与棋盘格,圆形标定靶的标定结果,该方法的平均标定误差为0.0980mm,同时在焦距1.4mm,视场220°鱼眼镜头下,仍能保持较高的精度,证明了所提方法的可行性和有效性。

基于视频的天气现象识别及其应用研究

针对传统天气现象观测因采用专用设备导致布设和维护成本高、获取难的问题,提出一种利用广泛布设的视频实现对天气现象观测的方法。通过将因特网获取的16327张天气现象图片放入深度神经网络中训练,建立一个天气现象分类的预训练模型。在此基础上加入上海徐家汇和洋山港气象站2021年视频图像,对模型进行微调,开发基于站点视频的天气现象识别模型。利用2022年1—10月份的视频图像数据对模型进行检验,模型识别结果的F1评分分别为0.74和0.67,而人工识别结果分别为0.67和0.61,表明所建立的模型性能接近或优于人眼识别的效果。通过例举两个应用案例,证明该项技术具有较好的应用前景。

基于直方图的手机玉米冠层数字图像氮素诊断方法

【目的】便捷准确地诊断作物氮素营养状况是实现作物精准施肥和氮肥资源合理利用的关键。近年来应用数码相机等工具进行作物营养诊断的研究受到广泛关注。本研究采用智能手机相机获取玉米冠层图像,建立完善的基于手机相机的氮素营养诊断技术,并比较传统的图像均值方法和直方图方法对氮素营养诊断的可靠性,以探明夏玉米氮素营养诊断的最佳适用模型。【方法】基于田间氮肥用量试验,采用智能手机相机获取夏玉米拔节期冠层图像,提取夏玉米冠层图像的G/R、G/B、NRI[R/(R+G+B)]、NGI[G/(R+G+B)]、NBI[B/(R+G+B)]和(G-R)/(R+G+B)6种颜色指数均值及直方图敏感区间,分别建立冠层图像色彩参数均值模型与直方图模型,分析其与玉米叶片含氮量和产量的关系。利用决定系数(R^(2))、均方根误差(RMSE)、平均绝对百分比误差(MAPE)对比不同指数模型模拟估算玉米叶片含氮量和产量的稳定性和准确性,建立基于手机相机获取夏玉米冠层图像的氮素营养诊断模型。【结果】施氮量显著影响玉米叶片含氮量、产量及冠层图像色调和植被覆盖度。直方图波峰b随叶片含氮量的增加而发生变化,相较于冠层图像色彩参数指数均值方法,指数直方图法适用于不同品种的氮素诊断。色彩参数(G-R)/(R+G+B)直方图可以更好地反映作物覆盖率及整体颜色信息,指数直方图与玉米叶片含氮量和产量也呈现较好的相关性。基于神经网络模型验证数据集精度评价指标,指数直方图模型中玉米叶片含氮量和产量的MAPE值和RMSE值均低于指数均值模型,R^(2)达到0.753,大于指数均值模型。指数直方图模型验证结果MAPE值达到5.80%,RMSE值为0.07,估算精度高,泛化性强。结果表明,冠层图像色彩参数指数直方图在估算叶片含氮量和产量时具有更高精度和更强鲁棒性,能够有效利用玉米叶片覆盖度、颜色等特点,具有较好的稳定性。【结论】利用智能手机相机获取玉米冠层数字图像,结合冠层图像色彩参数直方图方法建立的神经网络模型具有较好的应用效果,提高了估测精度,作为一种新方法在玉米氮素营养快速无损诊断和精准施肥中具有较好的应用潜力。

基于事件相机的目标检测算法研究

事件相机是模仿生物视网膜的成像方式,具有高动态、低延迟、高时间分辨率以及低功耗的特性。其突破传统相机难以捕捉在高动态范围情况下的物体并进行目标识别的困境,事件相机的特性对于研究基于事件相机的目标检测问题具有实验意义。简要叙述事件相机的现状、发展过程、优势与挑战,介绍了各种类型事件相机的工作原理和一些基于事件相机的目标检测算法,阐述了基于事件相机的目标检测算法面对的挑战和未来趋势,并进行了总结。

绝缘梯式配网搭接引流线机器人的作业控制方法研究

配电网人工搭接引流线工作繁重、效率低、危险系数高,当前配电网作业的机器人依赖斗臂车,成本高且不适用于农配网复杂的作业地形环境,因此,研制了一种绝缘梯式配网搭接引流线机器人。采用YOLOv8目标检测算法结合双目相机获取机器人作业过程中相关部件在作业坐标系下的三维坐标信息,并采用Fuzzy-PID控制方法精准控制相应作业机构到达指定位置;设计了一种基于多元线性回归控制方法的剥皮器,可以实现切削不损铝芯的精准控制;设计了一种基于滑模观测器的电动扳手,其具备轻量化、无转矩传感器的特点,能够实现作业过程中对转矩的估计与控制。通过仿真及在10 kV配网现场进行的三相搭接引流线试验结果可知,所提方法可以提高机器人的自动化程度和控制精度,保障机器人高效、高质量、安全稳定地完成三相搭接引流线试验,具有很好的实用价值。

火山SO_(2)排放速率反演

SO_(2)紫外相机因在时间分辨率、空间分辨率、探测灵敏度以及探测精度等诸多方面均具有显著优势而成功应用于火山活动监测及其动力学研究。为解决紫外相机反演SO_(2)排放速率容易受烟羽湍流及图像低对比度影响等问题,提出了融入神经网络的光流算法。首先,基于大气紫外辐射传输特性,阐述了SO_(2)紫外相机的工作机理及SO_(2)浓度图像的反演方法;其次,将神经网络融入光流算法,实现了火山烟羽图像中SO_(2)排放速率的精确反演;最后,与传统光流法进行对比,论证了神经网络光流算法的科学性及优越性与精确性。实验结果表明:在图像低对比度及烟羽湍流效应的双重影响下,神经网络光流法可以把边缘反演的误差从94%降低至5%,显著提高了SO_(2)排放速率反演的精确性。

基于坐标注意力脉冲神经网络的注视估计方法

针对传统相机在拍摄人眼运动时易产生动态模糊、时间分辨率低等问题,采用事件相机近眼拍摄构建Spiking-Eye数据集,并提出一种坐标注意力的脉冲神经网络模型(CA-SpikingRepVGG)。模型读取编码后的事件数据,经过带坐标注意力的主干网络进行特征提取,最后馈入检测头进行检测。实验结果显示:CA-SpikingRepVGG的平均检测精确率R_(P)达到了70.8%,与SpikingVGG-16比较,该模型的R_(P)提高了15.9%,召回率R_(r)提高了14.2%;仅需SpikingDensenet模型1/3的训练时间,比其R_(P)提高1.8%、R_(r)提高0.9%。结果表明:该模型在针对眼球运动这一场景下对人眼的检测追踪能力更强,可以很好地完成注视估计任务。

一种挂轨巡检车在城市调蓄池中的应用与研究

近几年,城市调蓄池的建设发展迅速,为了解决地下调蓄池环境检测难题,需要设计一种可在调蓄池内自动巡检的设备,检测污水和周围空气状态。文章设计了一种挂轨行驶的巡检车,其方法在调蓄池顶部预铺无源巡检轨道,调蓄池内搭建无线局域网络,通过工业相机和传感器检测技术自动检测、识别并记录池水和空气的状态数据及对应的坐标位置,并在上位机操作平台上实时显示。该文设计的巡检车实现了实时采集甲烷、乙炔、氨气、氧气浓度、温度、湿度、液位高度、污水状态判断、位置坐标、以及驱动参数,实现自动巡检功能,巡检速度0~1.5m/s,视频流采集帧率25fps,图像分辨率最大可达1920×1080dpi,方便快捷检测城市调蓄池内环境,警示用户是否存在安全隐患,保障城市调蓄池安全稳定运行。

基于事件信息与深度学习的高动态范围三维重建

采用光学三维成像技术测量金属零件、黑色物体以及半透明物体等高动态范围(High dynamic range,HDR)表面的三维轮廓是一个极具挑战性的问题。目前,传统方法对存在较低反射以及半透明区域的场景进行重建还有一定的局限性,半透明物体的内部反射噪声很难消除。现有基于深度学习的方法通常使用相对较强的激光强度,这可能会损坏样品,同时会出现采集图像过曝现象,需要对激光强度进行繁琐的调整。针对这些问题,本文提出基于事件信息和深度学习算法的高动态场景三维测量方法。事件相机通过异步记录单个像素的亮度变化,无需等待全局曝光时间,具有高动态响应范围,能够充分采集到HDR场景的激光条纹反射信息。引入深度卷积神经网络(Deep convolutional neural network,DCNN)来消除半透明物体的内部噪声以及金属物体高反光的过曝影响,同时增强弱激光条纹图像质量。实验结果表明,本文方法能够应用低功率线激光扫描成功实现HDR场景的高质量三维重建。