Movement Function Assessment Based on Human Pose Estimation from Multi-View

Human pose estimation is a basic and critical task in the field of computer vision that involves determining the position(or spatial coordinates)of the joints of the human body in a given image or video.It is widely used in motion analysis,medical evaluation,and behavior monitoring.In this paper,the authors propose a method for multi-view human pose estimation.Two image sensors were placed orthogonally with respect to each other to capture the pose of the subject as they moved,and this yielded accurate and comprehensive results of three-dimensional(3D)motion reconstruction that helped capture their multi-directional poses.Following this,we propose a method based on 3D pose estimation to assess the similarity of the features of motion of patients with motor dysfunction by comparing differences between their range of motion and that of normal subjects.We converted these differences into Fugl–Meyer assessment(FMA)scores in order to quantify them.Finally,we implemented the proposed method in the Unity framework,and built a Virtual Reality platform that provides users with human–computer interaction to make the task more enjoyable for them and ensure their active participation in the assessment process.The goal is to provide a suitable means of assessing movement disorders without requiring the immediate supervision of a physician.

Overview of 3D Human Pose Estimation

3D human pose estimation is a major focus area in the field of computer vision,which plays an important role in practical applications.This article summarizes the framework and research progress related to the estimation of monocular RGB images and videos.An overall perspective ofmethods integrated with deep learning is introduced.Novel image-based and video-based inputs are proposed as the analysis framework.From this viewpoint,common problems are discussed.The diversity of human postures usually leads to problems such as occlusion and ambiguity,and the lack of training datasets often results in poor generalization ability of the model.Regression methods are crucial for solving such problems.Considering image-based input,the multi-view method is commonly used to solve occlusion problems.Here,the multi-view method is analyzed comprehensively.By referring to video-based input,the human prior knowledge of restricted motion is used to predict human postures.In addition,structural constraints are widely used as prior knowledge.Furthermore,weakly supervised learningmethods are studied and discussed for these two types of inputs to improve the model generalization ability.The problem of insufficient training datasets must also be considered,especially because 3D datasets are usually biased and limited.Finally,emerging and popular datasets and evaluation indicators are discussed.The characteristics of the datasets and the relationships of the indicators are explained and highlighted.Thus,this article can be useful and instructive for researchers who are lacking in experience and find this field confusing.In addition,by providing an overview of 3D human pose estimation,this article sorts and refines recent studies on 3D human pose estimation.It describes kernel problems and common useful methods,and discusses the scope for further research.

3D Human Pose Estimation Using Two-Stream Architecture with Joint Training

With the advancement of image sensing technology, estimating 3Dhuman pose frommonocular video has becomea hot research topic in computer vision. 3D human pose estimation is an essential prerequisite for subsequentaction analysis and understanding. It empowers a wide spectrum of potential applications in various areas, suchas intelligent transportation, human-computer interaction, and medical rehabilitation. Currently, some methodsfor 3D human pose estimation in monocular video employ temporal convolutional network (TCN) to extractinter-frame feature relationships, but the majority of them suffer from insufficient inter-frame feature relationshipextractions. In this paper, we decompose the 3D joint location regression into the bone direction and length, wepropose the TCG, a temporal convolutional network incorporating Gaussian error linear units (GELU), to solvebone direction. It enablesmore inter-frame features to be captured andmakes the utmost of the feature relationshipsbetween data. Furthermore, we adopt kinematic structural information to solve bone length enhancing the use ofintra-frame joint features. Finally, we design a loss function for joint training of the bone direction estimationnetwork with the bone length estimation network. The proposed method has extensively experimented on thepublic benchmark dataset Human3.6M. Both quantitative and qualitative experimental results showed that theproposed method can achieve more accurate 3D human pose estimations.

Building 3-D Human Data Based on Handed Measurement and CNN

3-dimension(3-D)printing technology is growing strongly with many applications,one of which is the garment industry.The application of human body models to the garment industry is necessary to respond to the increasing personalization demand and still guarantee aesthetics.This paper proposes amethod to construct 3-D human models by applying deep learning.We calculate the location of the main slices of the human body,including the neck,chest,belly,buttocks,and the rings of the extremities,using pre-existing information.Then,on the positioning frame,we find the key points(fixed and unaltered)of these key slices and update these points tomatch the current parameters.To add points to a star slice,we use a deep learning model tomimic the form of the human body at that slice position.We use interpolation to produce sub-slices of different body sections based on the main slices to create complete body parts morphologically.We combine all slices to construct a full 3-D representation of the human body.

RFID-based 3D human pose tracking: A subject generalization approach

Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosensory games, and human-computer interaction. However, vision-based pose tracking techniques usually raise privacy concerns, making human pose tracking without vision data usage an important problem. Thus, we propose using Radio Frequency Identification (RFID) as a pose tracking technique via a low-cost wearable sensing device. Although our prior work illustrated how deep learning could transfer RFID data into real-time human poses, generalization for different subjects remains challenging. This paper proposes a subject-adaptive technique to address this generalization problem. In the proposed system, termed Cycle-Pose, we leverage a cross-skeleton learning structure to improve the adaptability of the deep learning model to different human skeletons. Moreover, our novel cycle kinematic network is proposed for unpaired RFID and labeled pose data from different subjects. The Cycle-Pose system is implemented and evaluated by comparing its prototype with a traditional RFID pose tracking system. The experimental results demonstrate that Cycle-Pose can achieve lower estimation error and better subject generalization than the traditional system.

DESIGN AND IMPLEMENTATION OF A 3-DIMENSIONAL COMPUTER AIDED GARMENT DESIGN SYSTEM

A 3-Dimensional computer aided garment design (CAGD) system has been developed andimplemented on a high-performance workstation. We studied various approaches to the func-tional modelling of garment designs for the system. According to the characteristic data of a hu-man body, the models of human body and the garment are displayed on the screen, then we canmodify the garment with various styles and different sizes. The system can transform the 3-Dgarment to the 2-D pieces. The system has improved design efficiency. Various potential alterna-tives and improvement of the system have also been studied and explored.

3-D MODELLING OF COMPUTER AIDED GARMENT DESIGN

This paper describes a method of the computer aided garment design,and discusses 3-D humanbody,wire frame modelling,approaches of expressing and a shading model of the 3-D garment.

Synthesis, Characterization and <i>In Vitro</i>Antitumor Evaluation of New Pyrazolo[3,4-d]Pyrimidine Derivatives

A new series of 3-(methylthio)-1-phenyl-1H-pyrazolo[3,4-d]pyrimidine derivatives was synthesized. The structures of the new derivatives were confirmed by the spectral data and elemental analyses. The antitumor activity of this series against human breast adenocarcinoma cell line MCF7 was evaluated. Out of twenty new derivatives, ten were revealed mild to moderate activity compared with doxorubicin as a reference antitumor. Among this new series N-(2-chlorophenyl)-2-(3-(methylthio)-4-oxo-1-phenyl-1H-pyrazolo[3,4-d]pyrimidin-5(4H)-yl)acetamide (13a) was found the most active one with IC50 equal to 23 μM.

Hourglass-GCN for 3D Human Pose Estimation Using Skeleton Structure and View Correlation

Previous multi-view 3D human pose estimation methods neither correlate different human joints in each view nor model learnable correlations between the same joints in different views explicitly,meaning that skeleton structure information is not utilized and multi-view pose information is not completely fused.Moreover,existing graph convolutional operations do not consider the specificity of different joints and different views of pose information when processing skeleton graphs,making the correlation weights between nodes in the graph and their neighborhood nodes shared.Existing Graph Convolutional Networks(GCNs)cannot extract global and deeplevel skeleton structure information and view correlations efficiently.To solve these problems,pre-estimated multiview 2D poses are designed as a multi-view skeleton graph to fuse skeleton priors and view correlations explicitly to process occlusion problem,with the skeleton-edge and symmetry-edge representing the structure correlations between adjacent joints in each viewof skeleton graph and the view-edge representing the view correlations between the same joints in different views.To make graph convolution operation mine elaborate and sufficient skeleton structure information and view correlations,different correlation weights are assigned to different categories of neighborhood nodes and further assigned to each node in the graph.Based on the graph convolution operation proposed above,a Residual Graph Convolution(RGC)module is designed as the basic module to be combined with the simplified Hourglass architecture to construct the Hourglass-GCN as our 3D pose estimation network.Hourglass-GCNwith a symmetrical and concise architecture processes three scales ofmulti-viewskeleton graphs to extract local-to-global scale and shallow-to-deep level skeleton features efficiently.Experimental results on common large 3D pose dataset Human3.6M and MPI-INF-3DHP show that Hourglass-GCN outperforms some excellent methods in 3D pose estimation accuracy.

MH-HMR:Human mesh recovery from monocular images via multi-hypothesis learning

Recovering 3D human meshes from monocular images is an inherently ill-posed and challenging task due to depth ambiguity,joint occlusion,and truncation.However,most existing approaches do not model such uncertainties,typically yielding a single reconstruction for one input.In contrast,the ambiguity of the reconstruction is embraced and the problem is considered as an inverse problem for which multiple feasible solutions exist.To address these issues,the authors propose a multi-hypothesis approach,multi-hypothesis human mesh recovery(MH-HMR),to efficiently model the multi-hypothesis representation and build strong relationships among the hypothetical features.Specifically,the task is decomposed into three stages:(1)generating a reasonable set of initial recovery results(i.e.,multiple hypotheses)given a single colour image;(2)modelling intra-hypothesis refinement to enhance every single-hypothesis feature;and(3)establishing inter-hypothesis communication and regressing the final human meshes.Meanwhile,the authors take further advantage of multiple hypotheses and the recovery process to achieve human mesh recovery from multiple uncalibrated views.Compared with state-of-the-art methods,the MH-HMR approach achieves superior performance and recovers more accurate human meshes on challenging benchmark datasets,such as Human3.6M and 3DPW,while demonstrating the effectiveness across a variety of settings.The code will be publicly available at https://cic.tju.edu.cn/faculty/likun/projects/MH-HMR.

融合深度学习与外极线约束的三维人体位姿测量方法

为了实现汽车座椅上三维人体姿态的高精度实时测量,基于现有二维人体关节点和三维人体关节点测量方法,提出了一种融合深度学习与外极线约束的三维人体姿态测量方法。该方法将二维人体关节点深度网络提取方法和双目测量系统相结合,采用双通道多阶段迭代网络分别提取左右相机图像中人体二维关节点,结合关节点位置的Brief特征和外极线约束,利用双目相机标定结果将匹配二维关节点信息转换到三维空间中,最终得到三维人体姿态。实验结果表明,文中提出方法在自采测试集中的检测精度可达到98%。通过得到三维关节点计算所得关键位姿角度的偏差小于10°。该文所提出的方法能够满足实际汽车座椅设计的数据采集要求。

MouseVenue3D:A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice

Understanding the connection between brain and behavior in animals requires precise monitoring of their behaviors in three-dimensional(3-D)space.However,there is no available three-dimensional behavior capture system that focuses on rodents.Here,we present MouseVenue3D,an automated and low-cost system for the efficient capture of 3-D skeleton trajectories in markerless rodents.We improved the most time-consuming step in 3-D behavior capturing by developing an automatic calibration module.Then,we validated this process in behavior recognition tasks,and showed that 3-D behavioral data achieved higher accuracy than 2-D data.Subsequently,MouseVenue3D was combined with fast high-resolution miniature two-photon microscopy for synchronous neural recording and behavioral tracking in the freely-moving mouse.Finally,we successfully decoded spontaneous neuronal activity from the 3-D behavior of mice.Our findings reveal that subtle,spontaneous behavior modules are strongly correlated with spontaneous neuronal activity patterns.

Pose estimation based on human detection and segmentation

We address the problem of 3D human pose estimation in a single real scene image. Normally, 3D pose estimation from real image needs background subtraction to extract the appropriate features. We do not make such assumption, In this paper, a two-step approach is proposed, first, instead of applying background subtraction to get the segmentation of human, we combine the segmentation with human detection using an ISM-based detector. Then, silhouette feature can be extracted and 3D pose estimation is solved as a regression problem. RVMs and ridge regression method are applied to solve this problem. The results show the robustness and accuracy of our method.

Three-dimensional structure and function study on the active region in the extracellular ligand-binding domain of human IL-6 receptor

In this study the three-dimensional (3-D) model of the ligand-binding domain (V106-P322) of human interleukin-6 receptor (hlL-6 R) was constructed by computer-guided ho-mology modeling technique using the crystal structure of the ligand-binding domain (K52-L251) of human growth hormone receptor (hGHR) as templet. Furthermore, the active binding region of the 3-D model of hlL-6R with the ligand (hlL-6) was predicted. In light of the structural characteristics of the active region, a hydrophobic pocket shielded by two hydrophilic residues (E115 and E505) of the region was identified by a combination of molecular modelling and the site-directed or double-site mutation of the twelve crucial residues in the ligand-binding domain of hIL-6R (V106-P322). We observed and analyzed the effects of these mutants on the spatial conformation of the pocket-like region of hlL-6 R. The results indicated that any site-directed mutation of the five Cys residues (four conservative Cys residues: Cyst 21, Cys132, Cys165, Cys176; near membrane Cys residue: Cys193) or each double-site mutation of the five residues in WSEWS motif of hIL-6R (V106-P322) makes the corresponding spatial conformation of the pocket region block the linkage between hlL-6 R and hlL-6. However, the influence of the site-directed mutation of Cys211 and Cys277 individually on the conformation of the pocket region benefits the interaction between hlL-6R and hlL-6. Our study suggests that the predicted hydrophobic pocket in the 3-D model of hIL-6R (V106-P322) is the critical molecular basis for the binding of hlL-6R with its ligand, and the active pocket may be used as a target for designing small hlL-6R-inhibiting molecules in our further study.

THREE-DIMENSIONAL MODELLING FOR LUBRICATION WITH REFERENCE TO HUMAN JOINTS

A three-dimensional numerical model on lubrication of human joints is presented in this peper. A simplified constitutive equation for viscoelasic fluid are obtained from the oldroyd's 4-constant model. A comparison between numerical result for a 'long joint' by the authors and analytic result by Manohar and Nigam [4] shows that the results agree.

A study on sampling strategies in the figure cognitive process

This study was aimed at investigating the sampling strategies for 2 types of figures: 3-D cubes and human faces. The research was focused on: (a) from where the sampling process started; (b) in what order the figures' features were sampled. The study consisted of 2 experiments: (a) sampling strategies for 3-D cubes; (b) sampling strategies for human faces. The results showed that: (a), for 3-D cubes, the first sampling was mostly located at the outline parts, rarely at the center part; while for human faces, the first sampling was mostly located at the hair and outline parts, rarely at the mouth or cheek parts, in most cases, the first sampling-position had no significant effects on cognitive performance and that (b), the sampling order, both for 3-D cubes and for human faces, was determined by the degree of difference among the sampled-features.

Virtual Temporal Bone Anatomy

Background The Visible Human Project(VHP) initiated by the U.S. National Library of Medicine has drawn much attention and interests from around the world. The Visible Chinese Human(VCH) project has started in China. The current study aims at acquiring a feasible virtual methodology for reconstructing the temporal bone of the Chinese population, which may provide an accurate 3-D model of important temporal bone structures that can be used in teaching and patient care for medical scientists and clinicians. Methods A series of sectional images of the temporal bone were generated from section slices of a female cadaver head. On each sectional image, SOIs (structures of interest) were segmented by carefully defining their contours and filling their areas with certain gray scale values. The processed volume data were then inducted into the 3D Slicer software(developed by the Surgical Planning Lab at Brigham and Women’s Hospital and the MIT AI Lab) for resegmentation and generation of a set of tagged images of the SOIs. 3D surface models of SOIs were then reconstructed from these images. Results The temporal bone and structures in the temporal bone, including the tympanic cavity, mastoid cells, sigmoid sinus and internal carotid artery, were successfully reconstructed. The orientation of and spatial relationship among these structures were easily visualized in the reconstructed surface models. Conclusion The 3D Slicer software can be used for 3- dimensional visualization of anatomic structures in the temporal bone, which will greatly facilitate the advance of knowledge and techniques critical for studying and treating disorders involving the temporal bone.

Compute extremely low-frequency electromagnetic field exposure by 3-D impendance method

A 3-D impedance method has been introduced to compute the electric currents induced in a human body exposed to extremely low-frequency electromagnetic field. The 3-D impedance method has been deduced from Maxwell equations and is put into the computation and simulation effectively to the visible human body model, which has 196×114×626 cells and more than 40 types of tissues. As the result, two representative cases are investigated. One is exposure of the human body to 100 μT （1 000 mG）, the limit recommended by the International Commission on Non-Ionizing Radiation Protection for the public and the other one is the exposure of human body to 0.4 laT （4 mG）, the level at which a statistical link appears with a doubled risk of development of childhood leukaemia. The distribution of induced current density can be obtained and the maximum of induced current are found to be 16 mA/m^2 and 0.07 mA/m^2.