Visual attention network

While originally designed for natural language processing tasks,the self-attention mechanism has recently taken various computer vision areas by storm.However,the 2D nature of images brings three challenges for applying self-attention in computer vision:(1)treating images as 1D sequences neglects their 2D structures;(2)the quadratic complexity is too expensive for high-resolution images;(3)it only captures spatial adaptability but ignores channel adaptability.In this paper,we propose a novel linear attention named large kernel attention(LKA)to enable self-adaptive and long-range correlations in self-attention while avoiding its shortcomings.Furthermore,we present a neural network based on LKA,namely Visual Attention Network(VAN).While extremely simple,VAN achieves comparable results with similar size convolutional neural networks(CNNs)and vision transformers(ViTs)in various tasks,including image classification,object detection,semantic segmentation,panoptic segmentation,pose estimation,etc.For example,VAN-B6 achieves 87.8%accuracy on ImageNet benchmark,and sets new state-of-the-art performance(58.2%PQ)for panoptic segmentation.Besides,VAN-B2 surpasses Swin-T 4%mloU(50.1%vs.46.1%)for semantic segmentation on ADE20K benchmark,2.6%AP(48.8%vs.46.2%)for object detection on COCO dataset.It provides a novel method and a simple yet strong baseline for the community.The code is available at https://github.com/Visual-Attention-Network.

JNeRF:An efficient heterogeneous NeRF model zoo based on Jittor

Neural radiance fields(NeRFs)for novel-view synthesis have attracted the attention of researchers in computer vision and graphics.Unlike traditional methods using explicit expressions,NeRFs represent a scene as an implicit neural radiance field.When rendering,NeRF queries the color density at every position in the scene through a neural network.NeRF brings a wide range of possibilities for real-world 3D reconstruction and rendering,but problems remain to be solved.Previous works have improved NeRF’s sampling technique,position encoding method,network structure,etc.,but these improvements are difficult to be combined as the different modules are not well decoupled.Recent works have significantly sped up the core GPU computation of NeRF,leaving the deep learning framework as a major computational cost.Thus,it has been suggested to replace the frameworks by pure CUDA programs,but this limits maintainability and extendability.Therefore,we propose JNeRF,a unified,efficient,framework-friendly NeRF model zoo based on Jittor.