Visual Motion Segmentation in Crowd Videos Based on Spatial-Angular Stacked Sparse Autoencoders

Visual motion segmentation(VMS)is an important and key part of many intelligent crowd systems.It can be used to figure out the flow behavior through a crowd and to spot unusual life-threatening incidents like crowd stampedes and crashes,which pose a serious risk to public safety and have resulted in numerous fatalities over the past few decades.Trajectory clustering has become one of the most popular methods in VMS.However,complex data,such as a large number of samples and parameters,makes it difficult for trajectory clustering to work well with accurate motion segmentation results.This study introduces a spatial-angular stacked sparse autoencoder model(SA-SSAE)with l2-regularization and softmax,a powerful deep learning method for visual motion segmentation to cluster similar motion patterns that belong to the same cluster.The proposed model can extract meaningful high-level features using only spatial-angular features obtained from refined tracklets(a.k.a‘trajectories’).We adopt l2-regularization and sparsity regularization,which can learn sparse representations of features,to guarantee the sparsity of the autoencoders.We employ the softmax layer to map the data points into accurate cluster representations.One of the best advantages of the SA-SSAE framework is it can manage VMS even when individuals move around randomly.This framework helps cluster the motion patterns effectively with higher accuracy.We put forward a new dataset with itsmanual ground truth,including 21 crowd videos.Experiments conducted on two crowd benchmarks demonstrate that the proposed model can more accurately group trajectories than the traditional clustering approaches used in previous studies.The proposed SA-SSAE framework achieved a 0.11 improvement in accuracy and a 0.13 improvement in the F-measure compared with the best current method using the CUHK dataset.

Prior-Free Dependent Motion Segmentation Using Helmholtz-Hodge Decomposition Based Object-Motion Oriented Map

Motion segmentation in moving camera videos is a very challenging task because of the motion dependence between the camera and moving objects. Camera motion compensation is recognized as an effective approach. However, existing work depends on prior-knowledge on the camera motion and scene structure for model selection. This is not always available in practice. Moreover, the image plane motion suffers from depth variations, which leads to depth-dependent motion segmentation in 3D scenes. To solve these problems, this paper develops a prior-free dependent motion segmentation algorithm by introducing a modified Helmholtz-Hodge decomposition （HHD） based object-motion oriented map （OOM）. By decomposing the image motion （optical flow） into a curl-free and a divergence-free component, all kinds of camera-induced image motions can be represented by these two components in an invariant way. HHD identifies the camera-induced image motion as one segment irrespective of depth variations with the help of OOM. To segment object motions from the scene, we deploy a novel spatio-temporal constrained quadtree labeling. Extensive experimental results on benchmarks demonstrate that our method improves the performance of the state-of-the-art by 10%-20% even over challenging scenes with complex background.

Motion Segmentation Based on Dual Interrelated Models

Motion segmentation plays an important role in many vision applications,yet it is still a challenging problem in complex scenes.The typical conditions in real world scenarios like illumination variations,dynamic backgrounds and camera shaking make negative effects on segmentation performance.In this paper,a newly designed method for robust motion segmentation is proposed,which is mainly composed of two interrelated models.One is a normal random model（N-model）,and the other is called enhanced random model（E-model）.They are constructed and updated in spatio-temporal information for adapting to illumination changes and dynamic backgrounds,and operate in an AdaBoost-like strategy.The exhaustive experimental evaluations on complex scenes demonstrate that the proposed method outperforms the state-of-the-art methods.

Evaluating quality of motion for unsupervised video object segmentation

Current mainstream unsupervised video object segmentation(UVOS) approaches typically incorporate optical flow as motion information to locate the primary objects in coherent video frames. However, they fuse appearance and motion information without evaluating the quality of the optical flow. When poor-quality optical flow is used for the interaction with the appearance information, it introduces significant noise and leads to a decline in overall performance. To alleviate this issue, we first employ a quality evaluation module(QEM) to evaluate the optical flow. Then, we select high-quality optical flow as motion cues to fuse with the appearance information, which can prevent poor-quality optical flow from diverting the network's attention. Moreover, we design an appearance-guided fusion module(AGFM) to better integrate appearance and motion information. Extensive experiments on several widely utilized datasets, including DAVIS-16, FBMS-59, and You Tube-Objects, demonstrate that the proposed method outperforms existing methods.

Charge Couple Device-Based Systemfor3-di mensional Real Ti me Positioning on the Assessment of Segmental Range of Motion of Lumbar Spine

Objective： To observe the tested results of the segmental range of motion （ROM） of lumbar spine by charge couple device （CCD）-based system for 3-dimensional real-time positioning （CCD system）, and to analyze its clinical significance. Methods： Seven patients with lumbar joint dysfunction and 8 healthy subjects were tested twice by the CCD-based system with an interval of 10 min. Results： The ROM of the patients was obviously lesser than that of the healthy subjects. The measuring data of segmental ROM of lumbar spine by CCD system is correlated significantly to the same data checked later on the same subjects in every direction of the movements. The differences between two checks are usually less than 1 degree. Conclusion： Specially designed CCD based system for 3-dimensional real-time positioning could objectively reflect the segmental ROM of lumbar spine. The system would be of great clinical significance in the assessment of the biomechanical dysfunction of lumbar spine and the effect of the treatment applied.

Study on the synthesis of spin labeled poly(styrene-co-maleic acid)s and their segmental motion

Study of the segmental mobility of polymer chains is important when the polymer is used as drag reduction agents of crude oil.Electron spin resonance(ESR)spectroscopy can provide important information about segmental dynamics of polymer chains,which is related to their microenvironment.In this article,we employed an amphiphilic polymer to study the effect of hydrophilic/hydrophobic balance of the polymers on the segmental motion of polymer chains.Poly(styrene-co-maleic acid)(PSMA)was spin labeled with 4-amino TEMPO radicals by increasing the concentration of radical moiety on the polymer chains.The PSMA and spin labeled-PSMAs(SL-PSMAs)were characterized by nuclear magnetic resonance(NMR),Fourier transform infrared(FT-IR)spectroscopy,Cyclic voltammetry(CV)and ESR techniques.Inter-polymer complexes(IPCs)of SL-PSMA-2 were prepared by employing polyethylene glycols(PEGs)of varying molecular weights.The results showed that the increased hydrophobic interactions of nitroxide radicals on the SLPSMAs’chains reduced the rotational mobility of spin labels and the random coil-toglobular transition of polymer chains occurred at higher pH value for SL-PSMAs,which showed a slow motion component in the ESR spectra of SL-PSMAs.Further,by increasing the molecular weight of PEGs in IPCs the complexation was increased,which also reduced the rotational motion of spin labels due to interpolymer hydrogen bonding causing a slow motion component in the ESR spectra.©2021 The Authors.

Review of techniques for motion capture data processing

In order to high reality and efficiency, the technique computer animation. With the development of motion capture, a of motion capture （MoCap） has been widely used in the field of large amount of motion capture databases are available and this is significant for the reuse of motion data. But due to the high degree of freedoms and high capture frequency, the dimension of the mo- tion capture data is usually very high and this will lead to a low efficiency in data processing. So how to process the high dimension data and design an efficient and effective retrieval approach has become a challenge which we can＇t ignore. In this paper, first we lay out some problems about the key techniques in motion capture data processing. Then the existing approaches are analyzed and sum- marized. At last, some future work is proposed.

The use of the greater trochanter marker in the thigh segment model:Implications for hip and knee frontal and transverse plane motion

Background:The greater trochanter marker is commonly used in 3-dimensional(3D) models;however,its influence on hip and knee kinematics during gait is unclear.Understanding the influence of the greater trochanter marker is important when quantifying frontal and transverse plane hip and knee kinematics,parameters which are particularly relevant to investigate in individuals with conditions such as patellofemoral pain,knee osteoarthritis,anterior cruciate ligament(ACL) injury,and hip pain.The aim of this study was to evaluate the effect of including the greater trochanter in the construction of the thigh segment on hip and knee kinematics during gait.Methods:3D kinematics were collected in 19 healthy subjects during walking using a surface marker system.Hip and knee angles were compared across two thigh segment definitions(with and without greater trochanter) at two time points during stance:peak knee flexion(PKF) and minimum knee flexion(Min KF).Results:Hip and knee angles differed in magnitude and direction in the transverse plane at both time points.In the thigh model with the greater trochanter the hip was more externally rotated than in the thigh model without the greater trochanter(PKF:-9.34°± 5.21° vs.1.40°± 5.22°,Min KF:-5.68°± 4.24° vs.5.01°± 4.86°;p < 0.001).In the thigh model with the greater trochanter,the knee angle was more internally rotated compared to the knee angle calculated using the thigh definition without the greater trochanter(PKF:14.67°± 6.78° vs.4.33°± 4.18°,Min KF:10.54°± 6.71° vs.-0.01°± 2.69°;p < 0.001).Small but significant differences were detected in the sagittal and frontal plane angles at both time points(p < 0.001).Conclusion:Hip and knee kinematics differed across different segment definitions including or excluding the greater trochanter marker,especially in the transverse plane.Therefore when considering whether to include the greater trochanter in the thigh segment model when using a surface markers to calculate 3D kinematics for movement assessment,it is important to have a clear understanding of the effect of different marker sets and segment models in use.

A multistimuli-responsive fluorescent hydrogel based on a fluorescence response to macromolecular segmental motion

Fluorescent hydrogels with fast and reversible responses have attracted extensive attention, and it remains a challenge to designmultistimuli-responsive fluorescent hydrogel through a facile and versatile method. Meanwhile, the segmental motion inhydrogels is of significance for the various functions of hydrogels such as chemical reactivity, self-healing, and mechanicalstrength, etc., however, it is difficult and complicated to in situ investigate the segmental motion under different conditions. In thiswork, a multistimuli-responsive fluorescent hydrogel was designed and fabricated by introducing a tetraphenylethylene (TPE)derivative as a nonaggregated crosslinker in the gel network. Since the intermolecular rotation of TPE at the crosslinking pointwas directly integrated with the dynamic conformational transition of the macromolecular network, the mobility of macromolecularsegments can be monitored by the fluorescence intensity of the hydrogel. The prepared hydrogel has promising fluorescenceresponses to temperature, pH, metal ions, and hydrogen bonding agents, and characterization of the fluorescence and the chainsegmental motion showed that the weaker the mobility of the network macromolecular chain is, the stronger the fluorescenceintensity is. Furthermore, due to the multistimuli-responsive fluorescence of the hydrogel, such fluorescent hydrogels can bedesigned as reversible patterning displays and biomimetic color/shape adjustable actuators, with various potential applications.

ClusterSLAM:A SLAM backend for simultaneous rigid body clustering and motion estimation

We present a practical backend for stereo visual SLAM which can simultaneously discover individual rigid bodies and compute their motions in dynamic environments.While recent factor graph based state optimization algorithms have shown their ability to robustly solve SLAM problems by treating dynamic objects as outliers,their dynamic motions are rarely considered.In this paper,we exploit the consensus of 3 D motions for landmarks extracted from the same rigid body for clustering,and to identify static and dynamic objects in a unified manner.Specifically,our algorithm builds a noise-aware motion affinity matrix from landmarks,and uses agglomerative clustering to distinguish rigid bodies.Using decoupled factor graph optimization to revise their shapes and trajectories,we obtain an iterative scheme to update both cluster assignments and motion estimation reciprocally.Evaluations on both synthetic scenes and KITTI demonstrate the capability of our approach,and further experiments considering online efficiency also show the effectiveness of our method for simultaneously tracking ego-motion and multiple objects.

Effect of Static Load on the Nucleus Pulposus of Rabbit Intervertebral Disc Motion Segment in Ex vivo Organ Culture

Background：The development of mechanically active culture systems helps increase the understanding of the role of mechanical stress in intervertebral disc （IVD） degeneration.Motion segment cultures allow for preservation of the native IVD structure,and adjacent vertebral bodies facilitate the application and control of mechanical loads.The purpose of this study was to establish loading and organ culture methods for rabbit IVD motion segments to study the effect of static load on the whole disc organ.Methods：IVD motion segments were harvested from rabbit lumbar spines and cultured in no-loading 6-well plates （control conditions） or custom-made apparatuses under a constant,compressive load （3 kg,0.5 MPa） for up to 14 days.Tissue integrity,matrix synthesis,and the matrix gene expression profile were assessed after 3,7,and 14 days of culturing and compared with those of fresh tissues.Results：The results showed that ex vivo culturing of motion segments preserved tissue integrity under no-loading conditions for 14 days whereas the static load gradually destroyed the morphology after 3 days.Proteoglycan contents were decreased under both conditions,with a more obvious decrease under static load,and proteoglycan gene expression was also downregulated.However,under static load,immunohistochemical staining intensity and collagen Type Ⅱ alpha 1 （COL2A 1） gene expression were significantly enhanced （61.54 ± 5.91,P =0.035） and upregulated （1.195 ± 0.040,P =0.000）,respectively,compared with those in the controls （P 〈 0.05）.In contrast,under constant compression,these trends were reversed.Our initial results indicated that short-term static load stimulated the synthesis of collagen Type Ⅱ alpha l;however,sustained constant compression led to progressive degeneration and specifically to a decreased proteoglycan content.Conclusions：A loading and organ culture system for ex vivo rabbit IVD motion segments was developed.Using this system,we were able to study the effects of mechanical stimulation on the biology of IVDs,as well as the pathomechanics of IVD degeneration.

Detecting interaction/complexity within crowd movements using braid entropy

The segmentation of moving and non-moving regions in an image within the field of crowd analysis is a crucial process in terms of understanding crowd behavior. In many studies, similar movements were segmented according to the location, adjacency to each other, direction, and average speed. However, these segments may not in turn indicate the same types of behavior in each region. The purpose of this study is to better understand crowd behavior by locally measuring the degree of interaction/complexity within the segment. For this purpose, the flow of motion in the image is primarily represented as a series of trajectories. The image is divided into hexagonal cells and the finite time braid entropy(FTBE) values are calculated according to the different projection angles of each cell. These values depend on the complexity of the spiral structure that the trajectories generated throughout the movement and show the degree of interaction among pedestrians. In this study, behaviors of different complexities determined in segments are pictured as similar movements on the whole. This study has been tested on 49 different video sequences from the UCF and CUHK databases.

G2MF-WA:Geometric multi-model fitting with weakly annotated data

In this paper we address the problem of geometric multi-model fitting using a few weakly annotated data points,which has been little studied so far.In weak annotating(WA),most manual annotations are supposed to be correct yet inevitably mixed with incorrect ones.Such WA data can naturally arise through interaction in various tasks.For example,in the case of homography estimation,one can easily annotate points on the same plane or object with a single label by observing the image.Motivated by this,we propose a novel method to make full use of WA data to boost multi-model fitting performance.Specifically,a graph for model proposal sampling is first constructed using the WA data,given the prior that WA data annotated with the same weak label has a high probability of belonging to the same model.By incorporating this prior knowledge into the calculation of edge probabilities,vertices(i.e.,data points)lying on or near the latent model are likely to be associated and further form a subset or cluster for effective proposal generation.Having generated proposals,α-expansion is used for labeling,and our method in return updates the proposals.This procedure works in an iterative way.Extensive experiments validate our method and show that it produces noticeably better results than state-of-the-art techniques in most cases.

Large-scale 3D Semantic Mapping Using Stereo Vision

In recent years, there have been a lot of interests in incorporating semantics into simultaneous localization and mapping （SLAM） systems. This paper presents an approach to generate an outdoor large-scale 3D dense semantic map based on binocular stereo vision. The inputs to system are stereo color images from a moving vehicle. First, dense 3D space around the vehicle is constructed, and tile motion of camera is estimated by visual odometry. Meanwhile, semantic segmentation is performed through the deep learning technology online, and the semantic labels are also used to verify tim feature matching in visual odometry. These three processes calculate the motion, depth and semantic label of every pixel in the input views. Then, a voxel conditional random field （CRF） inference is introduced to fuse semantic labels to voxel. After that, we present a method to remove the moving objects by incorporating the semantic labels, which improves the motion segmentation accuracy. The last is to generate tile dense 3D semantic map of an urban environment from arbitrary long image sequence. We evaluate our approach on KITTI vision benchmark, and the results show that the proposed method is effective.

Segmental dynamics near the chain end of polystyrene in its ultrathin films:a study by single-molecule fluorescence de-focus microscopy

Rotational motion of fluorophores chemically attached to polystyrene chain-ends in ultra-thin films on solid substrates was studied by single-molecule fluorescence de-focus microscopy.The collective feature of the rotational motion was found and evidenced by the sharp change of the population of fluorophores undergoing rotational motion within a very narrow temperature range(named as the changing temperature,T c).The T c value was found to depend on film thickness and interfacial chemistry and the variation of the T c value is also dependent on the molecular weight of the polymer.The results demonstrate that the spatial confinement effect enhances the segmental mobility near the polymer chain-ends while the interfacial attraction restricts the segmental motion inside the thin film.