多模态高精度非线性激活函数协处理器设计

Coprocessor for Multi-Mode High Precision Nonlinear Activation Function

针对片上部署非线性激活函数产生的精度损失以及硬件资源开销大的问题,提出一种基于三分法指数方法的多模态高精度非线性激活函数协处理器设计.首先分析激活函数在不同拟合参数下的近似误差以及运算量,为设计提供指导;然后设计一种模块化的硬件框架,通过复用指数、对数、sigmoid模块并结合浮点计算单元,能够以较低的面积开销部署多种激活函数.在Xilinx的Vertix系列FPGA上完成原型测试,实验结果表明,在仅增加32个查找表的情况下,所提设计tanh和sigmoid的近似误差仅为2项拆分指数方法的65.02%和69.00%,同时拟合范围扩大60%;与高精度分段线性逼近方法相比,该设计在仅用4%的查找表数量的情况下,将近似误差缩小82%.

The deployment of nonlinear activation functions on a chip is suffering from accuracy loss and hardware resource overhead.To address these problems,a multi-mode high-precision coprocessor design framework for nonlinear activation functions is proposed and which is based on the three-split exponential method.In the first stage,the approximation error and the operation workload of nonlinear activation functions on different approximation parameters are analyzed to guide the design.In the second stage,a modular hardware framework is designed to deploy several nonlinear activation functions at a low cost by reusing exponential,logarithmic,and sigmoid modules and combining them with floating-point computation units.The prototype of the proposed framework has been implemented on Xilinx Vertix series FPGA.The experimental results show that with only 32 additional lookup table entries than the two-split exponential methods,the approximation error of tanh and sigmoid is 65.02% and 69.00% of that using the two-split method,and the fitting range is extended by 60%.Compared with the high-precision piecewise linear approximation method,the design has led to an 82% reduction in approximation error with only 4%of the number of lookup tables in use.