softmax变区段非线性双拟合方法及其FPGA实现

Variable section-nonlinear dual fitting method for softmax and its FPGA implementation

针对采用分段非线性拟合逼近法拟合人工神经网络中的softmax时,会出现一些区间误差较大的问题,本文提出了一种基于初次拟合误差的变区段非线性双拟合方法.该拟合方法的基本步骤是:首先,结合softmax的指数函数特性,通过均匀分段和随机分段的非线性拟合找出误差较大的区间;其次,根据区间误差的大小,选择合适的变区段进行第2次非线性拟合,误差大的区间选择分段小的区间,误差小的区间选择分段大的区间;最后,运用python进行softmax的拟合逼近实验,并在FPGA上实现.实验结果表明,该方法不仅解决了使用分段非线性拟合逼近法拟合softmax时会出现一些区间误差较大的问题,整体上还能保持较高的精度,其绝对误差保持在0.015以下,相对误差保持在1.2%以内;同时,运用FPGA实现了纳秒级别的良好实时性,单次运算平均时间约为3.75 ns.

In order to solve the problem of large interval deviation for fitting softmax in artificial neural network with subsection nonlinear fitting approximation method,a variable section-nonlinear dual fitting method based on the initial fitting deviation is proposed.The basic steps of this fitting method are as follows:First of all,the larger deviation is found out by uniform subsection and random subsection nonlinear fitting methods combined with the characteristics of softmax exponential function.Secondly,according to the deviations,the appropriate variable section is selected for the second nonlinear fitting.The small section is chosen for the large deviation,and the large section is applied for the small deviation.Finally,the fitting approximation experiment of softmax is carried out by python and implemented on FPGA.The experimental results demonstrate that this new method not only solves the high deviation with the subsection nonlinear fitting approximation method for fitting softmax but also maintains a high accuracy.The absolute value of error is kept below 0.015,and the relative error is kept within 1.2%.More important thing is that it achieves the good real-time performance with nanosecond level,and about 3.75 ns average time of single calculation by using FPGA.