可微分抽象机混合编程系统

Hybrid Programming System of Differentiable Abstract Machines

自动化编程是智能软件的核心挑战之一,使用程序执行轨迹或输入输出样例学习程序,是自动化编程的典型研究方法.这些方法无法弥合常规程序元素与神经网络组件间的隔阂,不能吸收经验信息输入、缺乏编程控制能力.给出了一种可无缝结合高级编程语言与神经网络组件的混合编程模型:使用高级编程语言元素和神经网络组件元素混合开发应用程序,其中,编程语言描述程序的框架、提供经验信息,关键复杂部分则用未定、可学习的神经网络组件占位,应用程序在可微分抽象机上运行生成程序的连续可微分计算图表示,然后使用输入输出数据,通过可微分优化方法对计算图进行训练,学习程序的未定部分,自动生成完整的确定性程序.可微分抽象机混合编程模型给出了一种能够将编程经验与神经网络自学习相结合的程序自动生成方法,弥合编程语言元素与神经网络元素间的隔阂,发挥并整合高级过程化编程和神经网络可训练学习编程各自的优势,将复杂的细节交给神经网络未定部分自动生成,降低编程难度或工作量,而适当的经验输入又有助于未定部分的学习,同时,为复用长期积累的宝贵编程经验提供输入接口.

Automated programming is one of the central challenges of intelligent software. Learning program by program execution traces or input output pairs are typical automatic programming research methods, but these methods can not bridge the gap between normal program elements and neural network components, can not absorb programing experience as input, and lack of programming control interface. This paper presents a hybrid programming model that seamlessly combines advanced programming language with neural network components. The program is composed of a mixture of elements from high-level programming language and neural network component, in which the language describes the sketch to provide experience information, with the key complex parts placed with undetermined and learnable neural network components. The program runs on differentiable abstraction machines to generate its continuous differentiable computational graph representation. Then, input-output pairs are used to train the graph by differentiable optimization method to learn to generate the complete program automaticly. This programming model provides an automatic program generation method which can combine programmer experience with neural network self-learning, bridges the gap between elements from programming language and neural network, which integrate the advantages of procedural and neural network programming, the complex details are automatically generated by neural networks to reduce the difficulty or workload of programming. Experience input is heuristichelpful to the learning of undetermined parts and provides an input interface for reusing valuable programming experience accumulated over a long period of time.