基于脉冲神经网络微调方法的遥感图像目标检测

Research on Spiking Neural Network fine-tuning method for object detection in remote sensing images

遥感影像目标检测问题是视觉图像识别任务的重要研究内容之一,但是在船舶遥感图像中,船舶目标小且分布稀疏,使用传统的人工神经网络(ANN)进行目标检测往往会浪费大量的计算资源。脉冲神经网络(SNN)的事件驱动与低功耗特性可以极大地节省能量消耗同时解放更多的计算资源。然而SNN神经元由于其复杂动态与不可微调的脉冲操作,难以正常进行训练。作为替代,将训练好的ANN转换为SNN可以有效规避这一问题。对于转换后的深层SNN,需要大量时间步长(time steps)来维持其性能。这一过程需要大量的计算资源并对产生较大的延迟,与低功耗的研究初衷相违背。本文研究了转换后SNN需要大量time steps维持模型性能的原因,并提出了新的转换方法,基于微调的逐层转换方法;考虑硬件部署的合理性,提出了泊松群编码,相比泊松编码,泊松群编码输出的脉冲序列噪声更小,对模型性能的影响更小。实验表明,微调转换方法在SAR舰船检测数据集(SSDD、AIR-SARShip)上取得与转换前模型(97.9%、79.6%)相近的性能(96.9%、70.3%),在PASCAL VOC数据集上也获得了较好的检测性能(49.2%),而且对于泊松群编码,time steps相同的条件下神经元数目越多,对模型性能的影响越小,时间步长较少的条件下即可获得与输入模拟频率近似的性能。本文的研究可以提升转换后SNN的性能,减少转换后SNN对time steps的需求,并为SNN的硬件部署提供了一个切实有效的输入编码方法。

Object detection in remote sensing images is essential research contents of visual image recognition tasks.However,in the remote sensing images of ships,a ship target is small and sparsely distributed,and using a traditional Artificial Neural Network(ANN)for object detection often wastes a considerable amount of computing resources.SpikinG Neural Networks(SNNs)can be applied due to its event-driven and low-power characteristics,greatly saving energy and computing resources.However,training an SNN is difficult because of the complex dynamics and nondifferentiable pulse operation of SNN neurons.Instead,converting a trained ANN into SNN can effectively circumvent training difficulties.For a converted deep SNN,many time steps are often required to maintain its performance.Unfortunately,this process requires a substantial amount of computing resources.This paper studies the reason why a large number of time steps are required to maintain an SNN model’s performance after conversion and proposes a novel conversion method:a layer-by-layer conversion method based on fine-tuning.During conversion,the network is converted layer by layer,and the subsequent unconverted network is fine-tuned,and thus the errors accumulate layer by layer during conversion is prevented.In addition,given the rationality of hardware deployment,we propose Poisson group coding,which uses multiple Poisson coding neurons to encode input images and sends them to the network after average pooling.Compared with Poisson coding,the output of Poisson group coding is less noisy and has less impact on model performance.The fine-tuning transformation method achieves a result(96.9%,70.3%),similar to that of YOLOv3-tiny(97.9%,79.6%)on the SAR ship detection datasets(AIR-SARShip),and 80%of the performance of the preconversion model can be achieved by using few time steps(20 and 80 steps).The method achieves good detection performance(49.2%)on the PASCAL VOC dataset.By contrast,the performance of the conventional conversion method is inferior to that of the fine-tuning conversion method in the same number of time steps and usually requires many time steps(more than 150 time steps),achieving improved detection performance.For Poisson group coding,the impact on model performance under the same time steps decreases with increasing neurons.Performance similar to the input simulation frequency can be achieved with few time steps.The layer-by-layer conversion method based on the proposed fine-tuning method effectively adapts the SNN network for the layer-bylayer conversion of error,preventing the accumulation of error in each layer and reducing SNN.This method can improve the performance of a converted SNN and reduce the time steps.Meanwhile,Poisson group coding provides an effective input coding method for the hardware deployment of an SNN.