混合曲率空间中的几何自适应元学习方法

Geometry-Adaptive Meta-Learning in Mixed-Curvature Spaces

元学习通过学习先验知识,能帮助模型快速适应新任务.在适应新任务的过程中,空间几何结构与数据几何结构的匹配程度对模型泛化起着重要作用.现实世界数据具有多样的非欧几何结构,例如自然语言具有非欧层级结构,人脸图像具有非欧环状结构等.已有研究表明,真实数据的非欧结构同黎曼流形的几何结构相匹配,从理论上提供了利用黎曼流形来建模数据的可行性.本文提出了混合曲率空间(mixed-curvature space)中的几何自适应元学习方法,利用多个混合曲率空间来表示数据,并生成与数据非欧结构相匹配的黎曼几何.本文构建了多混合曲率神经网络,将混合曲率空间的几何结构表示为曲率空间的曲率、数量和维度,由此通过梯度下降过程实现对数据非欧结构的几何自适应.本文进一步引入几何初始化生成策略和几何更新策略,通过少数几步迭代,空间几何结构即可快速匹配数据非欧结构,加速了梯度下降过程.本文在小样本分类和小样本回归等任务上进行了实验验证.与欧氏空间的元学习方法相比,本文方法在小样本分类任务上取得了约3%的准确率提升,在小样本回归任务上将均方误差减少了一半,验证了本文方法的有效性.

Meta-learning has shown effectiveness in helping learning models quickly adapt to new tasks by learning prior knowledge.In the process of adaptation to new tasks,the matching degree between the geometric structure of space and the geometric structure of data plays an important role in the generalization ability of the model.In many practical applications,data has diverse non-Euclidean structures.For example,natural language has non-Euclidean hierarchical structures,and face images have non-Euclidean cyclical structures.Existing research has shown that the geometric structure of Riemannian manifolds matches the non-Euclidean structures of real-world data,providing theoretical feasibility for modeling data using Riemannian manifolds.In this paper,we propose a geometry-adaptive meta-learning method in mixed-curvature spaces,which uses multiple mixed-curvature spaces to model data and produces matching Riemannian geometry for non-Euclidean structures.We build a multi-mixed-curvature neural network that represents the geometry of mixed-curvature space as curvature,number,and dimensionality of the curvature spaces,through which the geometry adaptation to non-Euclidean structures is achieved via a gradient descent process.We further introduce a geometry initialization generation scheme and geometry updating scheme.Through only a few optimization steps,the geometric structure of the underlying space can quickly match non-Euclidean structures of data,accelerating the gradient descent process.We conduct experiments on few-shot classification,few-shot regression,and image completion to evaluate the effectiveness of our method.Compared with meta-learning methods in Euclidean space,our method improves the accuracy by 3% in few-shot classification tasks,and reduces mean square error by half in few-shot regression tasks,showing the effectiveness of our method.