Self-Attention Mechanism-Based Activity and Motion Recognition Using Wi-Fi Signals

Activity and motion recognition using Wi-Fi signals,mainly channel state information(CSI),has captured the interest of many researchers in recent years.Many research studies have achieved splendid results with the help of machine learning models from different applications such as healthcare services,sign language translation,security,context awareness,and the internet of things.Nevertheless,most of these adopted studies have some shortcomings in the machine learning algorithms as they rely on recurrence and convolutions and,thus,precluding smooth sequential computation.Therefore,in this paper,we propose a deep-learning approach based solely on attention,i.e.,the sole Self-Attention Mechanism model(Sole-SAM),for activity and motion recognition using Wi-Fi signals.The Sole-SAM was deployed to learn the features representing different activities and motions from the raw CSI data.Experiments were carried out to evaluate the performance of the proposed Sole-SAM architecture.The experimental results indicated that our proposed system took significantly less time to train than models that rely on recurrence and convolutions like Long Short-Term Memory(LSTM)and Recurrent Neural Network(RNN).Sole-SAM archived a 0.94%accuracy level,which is 0.04%better than RNN and 0.02%better than LSTM.

Behaviour recognition based on the integration of multigranular motion features in the Internet of Things

With the adoption of cutting-edge communication technologies such as 5G/6G systems and the extensive development of devices,crowdsensing systems in the Internet of Things(IoT)are now conducting complicated video analysis tasks such as behaviour recognition.These applications have dramatically increased the diversity of IoT systems.Specifically,behaviour recognition in videos usually requires a combinatorial analysis of the spatial information about objects and information about their dynamic actions in the temporal dimension.Behaviour recognition may even rely more on the modeling of temporal information containing short-range and long-range motions,in contrast to computer vision tasks involving images that focus on understanding spatial information.However,current solutions fail to jointly and comprehensively analyse short-range motions between adjacent frames and long-range temporal aggregations at large scales in videos.In this paper,we propose a novel behaviour recognition method based on the integration of multigranular(IMG)motion features,which can provide support for deploying video analysis in multimedia IoT crowdsensing systems.In particular,we achieve reliable motion information modeling by integrating a channel attention-based short-term motion feature enhancement module(CSEM)and a cascaded long-term motion feature integration module(CLIM).We evaluate our model on several action recognition benchmarks,such as HMDB51,Something-Something and UCF101.The experimental results demonstrate that our approach outperforms the previous state-of-the-art methods,which confirms its effective-ness and efficiency.

CPG Human Motion Phase Recognition Algorithm for a Hip Exoskeleton with VSA Actuator

Due to the dynamic stiffness characteristics of human joints, it is easy to cause impact and disturbance on normal movements during exoskeleton assistance. This not only brings strict requirements for exoskeleton control design, but also makes it difficult to improve assistive level. The Variable Stiffness Actuator (VSA), as a physical variable stiffness mechanism, has the characteristics of dynamic stiffness adjustment and high stiffness control bandwidth, which is in line with the stiffness matching experiment. However, there are still few works exploring the assistive human stiffness matching experiment based on VSA. Therefore, this paper designs a hip exoskeleton based on VSA actuator and studies CPG human motion phase recognition algorithm. Firstly, this paper puts forward the requirements of variable stiffness experimental design and the output torque and variable stiffness dynamic response standards based on human lower limb motion parameters. Plate springs are used as elastic elements to establish the mechanical principle of variable stiffness, and a small variable stiffness actuator is designed based on the plate spring. Then the corresponding theoretical dynamic model is established and analyzed. Starting from the CPG phase recognition algorithm, this paper uses perturbation theory to expand the first-order CPG unit, obtains the phase convergence equation and verifies the phase convergence when using hip joint angle as the input signal with the same frequency, and then expands the second-order CPG unit under the premise of circular limit cycle and analyzes the frequency convergence criterion. Afterwards, this paper extracts the plate spring modal from Abaqus and generates the neutral file of the flexible body model to import into Adams, and conducts torque-stiffness one-way loading and reciprocating loading experiments on the variable stiffness mechanism. After that, Simulink is used to verify the validity of the criterion. Finally, based on the above criterions, the signal mean value is removed using feedback structure to complete the phase recognition algorithm for the human hip joint angle signal, and the convergence is verified using actual human walking data on flat ground.

Online object detection and recognition using motion information and local feature co-occurrence

An object learning and recognition system is implemented for humanoid robots to discover and memorize objects only by simple interactions with non-expert users. When the object is presented, the system makes use of the motion information over consecutive frames to extract object features and implements machine learning based on the bag of visual words approach. Instead of using a local feature descriptor only, the proposed system uses the co-occurring local features in order to increase feature discriminative power for both object model learning and inference stages. For different objects with different textures, a hybrid sampling strategy is considered. This hybrid approach minimizes the consumption of computation resources and helps achieving good performances demonstrated on a set of a dozen different daily objects.

Multi-scale context-aware network for continuous sign language recognition

The hands and face are the most important parts for expressing sign language morphemes in sign language videos.However,we find that existing Continuous Sign Language Recognition(CSLR)methods lack the mining of hand and face information in visual backbones or use expensive and time-consuming external extractors to explore this information.In addition,the signs have different lengths,whereas previous CSLR methods typically use a fixed-length window to segment the video to capture sequential features and then perform global temporal modeling,which disturbs the perception of complete signs.In this study,we propose a Multi-Scale Context-Aware network(MSCA-Net)to solve the aforementioned problems.Our MSCA-Net contains two main modules:(1)Multi-Scale Motion Attention(MSMA),which uses the differences among frames to perceive information of the hands and face in multiple spatial scales,replacing the heavy feature extractors;and(2)Multi-Scale Temporal Modeling(MSTM),which explores crucial temporal information in the sign language video from different temporal scales.We conduct extensive experiments using three widely used sign language datasets,i.e.,RWTH-PHOENIX-Weather-2014,RWTH-PHOENIX-Weather-2014T,and CSL-Daily.The proposed MSCA-Net achieve state-of-the-art performance,demonstrating the effectiveness of our approach.

Ensemble learning HMM for motion recognition and retrieval by Isomap dimension reduction

Along with the development of motion capture technique, more and more 3D motion databases become available. In this paper, a novel approach is presented for motion recognition and retrieval based on ensemble HMM (hidden Markov model) learning. Due to the high dimensionality of motion’s features, Isomap nonlinear dimension reduction is used for training data of ensemble HMM learning. For handling new motion data, Isomap is generalized based on the estimation of underlying eigen- functions. Then each action class is learned with one HMM. Since ensemble learning can effectively enhance supervised learning, ensembles of weak HMM learners are built. Experiment results showed that the approaches are effective for motion data recog- nition and retrieval.

Study on automatic recognition of the first motion in a seismic event

In this paper, we have studied the waveforms of background noise in a seismograph and set up an AR model to characterize them. We then complete the modeling and the automatic recognition program. Finally, we provide the results from automatic recognition and the manual recognition of the first motion for 25 underground explosions.

Combining Multi-scale Directed Depth Motion Maps and Log-Gabor Filters for Human Action Recognition

Recognition of the human actions by computer vision has become an active research area in recent years. Due to the speed and the high similarity of the actions, the current algorithms cannot get high recognition rate. A new recognition method of the human action is proposed with the multi-scale directed depth motion maps(MsdDMMs) and Log-Gabor filters. According to the difference between the speed and time order of an action, MsdDMMs is proposed under the energy framework. Meanwhile, Log-Gabor is utilized to describe the texture details of MsdDMMs for the motion characteristics. It can easily satisfy both the texture characterization and the visual features of human eye. Furthermore, the collaborative representation is employed as action recognition by the classification. Experimental results show that the proposed algorithm, which is applied in the MSRAction3 D dataset and MSRGesture3 D dataset, can achieve the accuracy of 95.79% and 96.43% respectively. It also has higher accuracy than the existing algorithms, such as super normal vector(SNV), hierarchical recurrent neural network(Hierarchical RNN).

Human Motion Recognition Based on Incremental Learning and Smartphone Sensors

Batch processing mode is widely used in the training process of human motiun recognition. After training, the motion elassitier usually remains invariable. However, if the classifier is to be expanded, all historical data must be gathered for retraining. This consumes a huge amount of storage space, and the new training process will be more complicated. In this paper, we use an incremental learning method to model the motion classifier. A weighted decision tree is proposed to help illustrate the process, and the probability sampling method is also used. The resuhs show that with continuous learning, the motion classifier is more precise. The average classification precision for the weighted decision tree was 88.43% in a typical test. Incremental learning consumes much less time than the batch processing mode when the input training data comes continuously.

Human Motion Recognition Using Ultra-Wideband Radar and Cameras on Mobile Robot

Cameras can reliably detect human motions in a normal environment, but they are usually affected by sudden illumination changes and complex conditions, which are the major obstacles to the reliability and robustness of the system. To solve this problem, a novel integration method was proposed to combine hi-static ultra-wideband radar and cameras. In this recognition system, two cameras are used to localize the object＇s region, regions while a radar is used to obtain its 3D motion models on a mobile robot. The recognition results can be matched in the 3D motion library in order to recognize its motions. To confirm the effectiveness of the proposed method, the experimental results of recognition using vision sensors and those of recognition using the integration method were compared in different environments. Higher correct-recognition rate is achieved in the experiment.

Low-Cost Posture Recognition of Moving Hands by Profile-Mold Construction in Cluttered Background and Occlusion

In this paper, we propose a low-cost posture recognition scheme using a single webcam for the signaling hand with nature sways and possible oc-clusions. It goes for developing the untouchable low-complexity utility based on friendly hand-posture signaling. The scheme integrates the dominant temporal-difference detection, skin color detection and morphological filtering for efficient cooperation in constructing the hand profile molds. Those molds provide representative hand profiles for more stable posture recognition than accurate hand shapes with in effect trivial details. The resultant bounding box of tracking the signaling molds can be treated as a regular-type object-matched ROI to facilitate the stable extraction of robust HOG features. With such commonly applied features on hand, the prototype SVM is adequately capable of obtaining fast and stable hand postures recognition under natural hand movement and non-hand object occlusion. Experimental results demonstrate that our scheme can achieve hand-posture recognition with enough accuracy under background clutters that the targeted hand can be allowed with medium movement and palm-grasped object. Hence, the proposed method can be easily embedded in the mobile phone as application software.

Action recognition using a hierarchy of feature groups

To improve the recognition performance of video human actions,an approach that models the video actions in a hierarchical way is proposed. This hierarchical model summarizes the action contents with different spatio-temporal domains according to the properties of human body movement.First,the temporal gradient combined with the constraint of coherent motion pattern is utilized to extract stable and dense motion features that are viewed as point features,then the mean-shift clustering algorithm with the adaptive scale kernel is used to label these features.After pooling the features with the same label to generate part-based representation,the visual word responses within one large scale volume are collected as video object representation.On the benchmark KTH（Kungliga Tekniska H？gskolan）and UCF （University of Central Florida）-sports action datasets,the experimental results show that the proposed method enhances the representative and discriminative power of action features, and improves recognition rates.Compared with other related literature,the proposed method obtains superior performance.

Multi-Modality Video Representation for Action Recognition

Nowadays,action recognition is widely applied in many fields.However,action is hard to define by single modality information.The difference between image recognition and action recognition is that action recognition needs more modality information to depict one action,such as the appearance,the motion and the dynamic information.Due to the state of action evolves with the change of time,motion information must be considered when representing an action.Most of current methods define an action by spatial information and motion information.There are two key elements of current action recognition methods:spatial information achieved by sampling sparsely on video frames’sequence and the motion content mostly represented by the optical flow which is calculated on consecutive video frames.However,the relevance between them in current methods is weak.Therefore,to strengthen the associativity,this paper presents a new architecture consisted of three streams to obtain multi-modality information.The advantages of our network are:(a)We propose a new sampling approach to sample evenly on the video sequence for acquiring the appearance information;(b)We utilize ResNet101 for gaining high-level and distinguished features;(c)We advance a three-stream architecture to capture temporal,spatial and dynamic information.Experimental results on UCF101 dataset illustrate that our method outperforms other previous methods.

基于Leap Motion手势识别的三维交互系统

随着虚拟交互技术的发展,人们迈入了“体验式经济时代”,消费者越来越关注个性体验,因此,基于Leap Motion手势识别设备,设计了一种三维虚拟室内交互系统。该系统以Unity3D作为开发工具,Leap Motion作为硬件平台,结合C#语言进行脚本的编译,利用3ds Max平台对室内进行场景搭建,通过Unity3D工具将组件整合,设计了七种手势,使用Leap Motion硬件设备对场景中物体进行各种不同的操作。经试验表明,该系统实现了用户与场景中物体的交互能力,可以应用在室内装修和设计等方面,增强人们的体验感与趣味性。

A Fast Statistical Approach for Human Activity Recognition

An essential part of any activity recognition system claiming be truly real-time is the ability to perform feature extraction in real-time. We present, in this paper, a quite simple and computationally tractable approach for real-time human activity recognition that is based on simple statistical features. These features are simple and relatively small, accordingly they are easy and fast to be calculated, and further form a relatively low-dimensional feature space in which classification can be carried out robustly. On the Weizmann publicly benchmark dataset, promising results (i.e. 97.8%) have been achieved, showing the effectiveness of the proposed approach compared to the-state-of-the-art. Furthermore, the approach is quite fast and thus can provide timing guarantees to real-time applications.

Influence of enhancing dynamic scapular recognition on shoulder disability,and pain in diabetics with frozen shoulder:A case report

BACKGROUND Frozen shoulder(FS)is a familiar disorder.Diabetics with FS have more severe symptoms and a worse prognosis.Thus,this study investigated the influence of enhancing dynamic scapular recognition on shoulder disability and pain in diabetics with FS.CASE SUMMARY A Forty-five years-old male person with diabetes mellitus and a unilateral FS(stage II)for at least 3 mo with shoulder pain and limitation in both passive and active ranges of motion(ROMs)of the glenohumeral joint of≥25%in 2 directions participated in this study.This person received dynamic scapular recognition exercise was applied to a diabetic person with a unilateral FS(stage II).The main outcome measures were upward rotation of the scapula,shoulder pain and disability index,and shoulder range of motion of flexion,abduction,and external rotation.The dynamic scapular exercise was performed for 15 min/session and 3 sessions/wk lasted for 4 wk.After 4 wk of intervention,there were improvements between pre-treatment and post-treatment in shoulder pain,shoulder pain and disability index,shoulder ROM,and upward rotation of the scapula.CONCLUSION This case report suggested that enhancing dynamic scapular recognition may improve shoulder pain and disability;upward rotation of the scapula;and shoulder ROM of shoulder abduction,flexion,and external rotation after 4 wk.

Continuous Arabic Sign Language Recognition in User Dependent Mode

Arabic Sign Language recognition is an emerging field of research. Previous attempts at automatic vision-based recog-nition of Arabic Sign Language mainly focused on finger spelling and recognizing isolated gestures. In this paper we report the first continuous Arabic Sign Language by building on existing research in feature extraction and pattern recognition. The development of the presented work required collecting a continuous Arabic Sign Language database which we designed and recorded in cooperation with a sign language expert. We intend to make the collected database available for the research community. Our system which we based on spatio-temporal feature extraction and hidden Markov models has resulted in an average word recognition rate of 94%, keeping in the mind the use of a high perplex-ity vocabulary and unrestrictive grammar. We compare our proposed work against existing sign language techniques based on accumulated image difference and motion estimation. The experimental results section shows that the pro-posed work outperforms existing solutions in terms of recognition accuracy.

基于ME-ResNet人脸微表情识别方法

人脸微表情具有持续时间短、动作幅度小的特点。数据集样本量较少等因素也给微表情识别带来了巨大挑战。针对上述问题,提出一种基于ME-ResNet残差网络的微表情识别方法。首先,在预处理阶段,等间隔提取微表情视频片段起始帧至顶点帧之间的关键帧序列,利用改进Farneback光流法提取微表情关键帧序列的面部光流运动特征;接着,构建基于3D卷积的ResNet50网络,并将空间通道注意力CBAM机制加入网络Bottleneck模块,以增强网络对面部关键运动特征的聚焦学习能力,并构建ME-ResNet网络模型,将所提取的面部光流运动特征送入网络进行训练;最后,使用数据增强增加网络训练样本量,将ME-ResNet网络模型用于微表情识别任务,并在CASMEII,SMIC和SAMM数据集上进行实验验证,所提算法识别率达到了84.42%,72.56%,70.41%,与其他算法相比具有较高的识别能力。

基于Leap Motion的手势识别在虚拟交互中的研究

针对虚拟场景中的自然手势交互进行了研究,提出了基于Leap Motion的动态指尖手势轨迹识别方法。首先借助Leap Motion设备采集指尖在场景中运动时产生的坐标并同时对数据进行预处理,然后从这一系列坐标中找出起始和结束位置并提取出有效的手势轨迹,再进行轨迹优化和手势初步分类,基于加权欧氏距离将轨迹和手势模板进行相似度计算,得到识别结果。采集200组手势数据进行实验,结果证明提出的方法具有很高的识别率,将方法应用在手势交互系统中,实现使用自然手势和虚拟物品进行交互,增加了交互乐趣,改善了交互体验。

基于Leap Motion的虚拟农作物人机交互系统设计

传统的虚拟植物人机交互系统一般运行在特定的操作系统或者移动平台上,而交互方式多是通过鼠标键盘进行人机交互,需要用户输入较为繁琐的参数和命令,导致系统用户缺乏良好的交互体验。针对上述情况,该本文基于云计算和体感交互技术,设计开发了虚拟农作物体感交互系统。系统先在云端计算生成虚拟农作物的三维模型,并将这些模型保存在云端。随后,由系统前端的Leap Motion体感控制器采集用户手部信息数据,通过本地计算机对数据进行处理,识别出各种各样的手势,再与浏览器上Web GL绘制成的虚拟植物进行实时交互,从而实现植物的平移、旋转、生长、形态变化以及三维模型的更新等操作。详细介绍了Leap Motion手势识别的基本原理,虚拟农作物体感交互系统的总体框架,农作物的测量和虚拟模型的生成,数据交互协议的设计原理,Web GL图形绘制以及单双手手势交互库的建立等关键技术。该系统已经由Web GL编程实现,试验结果表明,使用该文研发的系统,用户在浏览器上可以通过不同的手势对虚拟作物模型的形态和生长进行实时调控,具有良好的交互体验。该系统的建立,可为国内外同类研究提供技术上的参考。