一种去注意力机制的动态多层语义感知机

Transformer在大规模数据集上取得了优异效果,但由于使用多头注意力使得模型过于复杂,且在小规模数据集上效果不理想.对于多头注意力替换的研究在图像处理领域已取得一些成果,但在自然语言处理领域还少有研究.为此,首先提出一种去注意力的多层语义感知机(multi-layer semantics perceptron,MSP)方法,其核心创新是使用token序列转换函数替换编码器中的多头注意力,降低模型复杂度,获得更好的语义表达;然后,提出一种动态深度控制框架(dynamic depth control framework,DDCF),优化模型深度,降低模型复杂度;最后,在MSP方法和DDCF的基础上,提出动态多层语义感知机(dynamic multi-layer semantics perceptron,DMSP)模型,在多种文本数据集上的对比实验结果表明,DMSP既能提升模型分类精度,又能有效降低模型复杂度,与Transformer比较,在模型深度相同的情况下,DMSP模型分类精度大幅提升,同时模型的参数量大幅降低.

Control of large angle maneuvers for the flexible solar sail

Solar sail is a new type of spacecraft for deep space exploration,which flies by the pressure of sunlight.The attitude of the sail determines its orbit,so altitude control plays an important role in the mission.However,the large flexible structure leads to some difficulty in attitude control.This paper establishes the reduced dynamic model of a flexible solar sail with foreshortening deformation,and coupling with its attitude and vibration.As usual,large angle maneuvering will lead to the vibration of flexible structure,so the time optimal control of solar sail maneuvering is considered.Bang-Bang control of the solar sail generates large amplitude and sustained vibration,while the combined control based on input shaping can eliminate the vibration efficiently.With the comparison of two reduced models,it is demonstrated that the choice of two models depends on the attention to the stretching deformation.