Autonomous one-shot on-the-fly learning copes with the high privacy,small dataset,and in-stream data at the edge.Implementing such learning on digital hardware suffers from the well-known von-Neumann and scaling bottl...Autonomous one-shot on-the-fly learning copes with the high privacy,small dataset,and in-stream data at the edge.Implementing such learning on digital hardware suffers from the well-known von-Neumann and scaling bottlenecks.The optical neural networks featuring large parallelism,low latency,and high efficiency offer a promising solution.However,ex-situ training of conventional optical networks,where optical path configuration and deep learning model optimization are separated,incurs hardware,energy and time overheads,and defeats the advantages in edge learning.Here,we introduced a bio-inspired material-algorithm co-design to construct a hydrogel-based optical Willshaw model(HOWM),manifesting Hebbian-rule-based structural plasticity for simultaneous optical path configuration and deep learning model optimization thanks to the underlying opto-chemical reactions.We first employed the HOWM as an all optical in-sensor AI processor for one-shot pattern classification,association and denoising.We then leveraged HOWM to function as a ternary content addressable memory(TCAM)of an optical memory augmented neural network(MANN)for one-shot learning the Omniglot dataset.The HOWM empowered one-shot on-the-fly edge learning leads to 1000boost of energy efficiency and 10boost of speed,which paves the way for the next-generation autonomous,efficient,and affordable smart edge systems.展开更多
基金supported by the National Key R&D Program of China(Grant No.2018YFA0701500)Hong Kong Research Grant Council(Grant No.27206321,17205922)+5 种基金the National Natural Science Foundation of China(Grant Nos.62122004,61874138,61888102,61771176,and 62171173)the Strategic Priority Research Program of the Chinese Academy of Sciences(Grant No.XDB44000000)Research on the GaN Chip for 5G Applications(Grant No:JCYJ20210324120409025)Research on high-reliable GaN power device and the related industrial power system(Grant No:HZQBKCZYZ-2021052)Key Project of Department of Education of Guangdong Province(No.2018KCXTD026)supported by ACCESS-AI Chip Center for Emerging Smart Systems,sponsored by Innovation and Technology Fund(ITF),Hong Kong SAR.
文摘Autonomous one-shot on-the-fly learning copes with the high privacy,small dataset,and in-stream data at the edge.Implementing such learning on digital hardware suffers from the well-known von-Neumann and scaling bottlenecks.The optical neural networks featuring large parallelism,low latency,and high efficiency offer a promising solution.However,ex-situ training of conventional optical networks,where optical path configuration and deep learning model optimization are separated,incurs hardware,energy and time overheads,and defeats the advantages in edge learning.Here,we introduced a bio-inspired material-algorithm co-design to construct a hydrogel-based optical Willshaw model(HOWM),manifesting Hebbian-rule-based structural plasticity for simultaneous optical path configuration and deep learning model optimization thanks to the underlying opto-chemical reactions.We first employed the HOWM as an all optical in-sensor AI processor for one-shot pattern classification,association and denoising.We then leveraged HOWM to function as a ternary content addressable memory(TCAM)of an optical memory augmented neural network(MANN)for one-shot learning the Omniglot dataset.The HOWM empowered one-shot on-the-fly edge learning leads to 1000boost of energy efficiency and 10boost of speed,which paves the way for the next-generation autonomous,efficient,and affordable smart edge systems.
