Joint user profiling with hierarchical attention networks

User profiling by inferring user personality traits,such as age and gender,plays an increasingly important role in many real-world applications.Most existing methods for user profiling either use only one type of data or ignore handling the noisy information of data.Moreover,they usually consider this problem from only one perspective.In this paper,we propose a joint user profiling model with hierarchical attention networks(JUHA)to learn informative user representations for user profiling.Our JUHA method does user profiling based on both inner-user and inter-user features.We explore inner-user features from user behaviors(e.g.,purchased items and posted blogs),and inter-user features from a user-user graph(where similar users could be connected to each other).JUHA learns basic sentence and bag representations from multiple separate sources of data(user behaviors)as the first round of data preparation.In this module,convolutional neural networks(CNNs)are introduced to capture word and sentence features of age and gender while the self-attention mechanism is exploited to weaken the noisy data.Following this,we build another bag which contains a user-user graph.Inter-user features are learned from this bag using propagation information between linked users in the graph.To acquire more robust data,inter-user features and other inner-user bag representations are joined into each sentence in the current bag to learn the final bag representation.Subsequently,all of the bag representations are integrated to lean comprehensive user representation by the self-attention mechanism.Our experimental results demonstrate that our approach outperforms several state-of-the-art methods and improves prediction performance.

Vision Transformers with Hierarchical Attention

This paper tackles the high computational/space complexity associated with multi-head self-attention(MHSA)in vanilla vision transformers.To this end,we propose hierarchical MHSA(H-MHSA),a novel approach that computes self-attention in a hierarchical fashion.Specifically,we first divide the input image into patches as commonly done,and each patch is viewed as a token.Then,the proposed H-MHSA learns token relationships within local patches,serving as local relationship modeling.Then,the small patches are merged into larger ones,and H-MHSA models the global dependencies for the small number of the merged tokens.At last,the local and global attentive features are aggregated to obtain features with powerful representation capacity.Since we only calculate attention for a limited number of tokens at each step,the computational load is reduced dramatically.Hence,H-MHSA can efficiently model global relationships among tokens without sacrificing fine-grained information.With the H-MHSA module incorporated,we build a family of hierarchical-attention-based transformer networks,namely HAT-Net.To demonstrate the superiority of HAT-Net in scene understanding,we conduct extensive experiments on fundamental vision tasks,including image classification,semantic segmentation,object detection and instance segmentation.Therefore,HAT-Net provides a new perspective for vision transformers.Code and pretrained models are available at https://github.com/yun-liu/HAT-Net.