Towards trustworthy multi-modal motion prediction:Holistic evaluation and interpretability of outputs

Predicting the motion of other road agents enables autonomous vehicles to perform safe and efficient path planning.This task is very complex,as the behaviour of road agents depends on many factors and the number of possible future trajectories can be consid-erable(multi-modal).Most prior approaches proposed to address multi-modal motion prediction are based on complex machine learning systems that have limited interpret-ability.Moreover,the metrics used in current benchmarks do not evaluate all aspects of the problem,such as the diversity and admissibility of the output.The authors aim to advance towards the design of trustworthy motion prediction systems,based on some of the re-quirements for the design of Trustworthy Artificial Intelligence.The focus is on evaluation criteria,robustness,and interpretability of outputs.First,the evaluation metrics are comprehensively analysed,the main gaps of current benchmarks are identified,and a new holistic evaluation framework is proposed.Then,a method for the assessment of spatial and temporal robustness is introduced by simulating noise in the perception system.To enhance the interpretability of the outputs and generate more balanced results in the proposed evaluation framework,an intent prediction layer that can be attached to multi-modal motion prediction models is proposed.The effectiveness of this approach is assessed through a survey that explores different elements in the visualisation of the multi-modal trajectories and intentions.The proposed approach and findings make a significant contribution to the development of trustworthy motion prediction systems for autono-mous vehicles,advancing the field towards greater safety and reliability.

Multipoint Heave Motion Prediction Method for Ships Based on the PSO-TGCN Model

During ship operations,frequent heave movements can pose significant challenges to the overall safety of the ship and completion of cargo loading.The existing heave compensation systems suffer from issues such as dead zones and control system time lags,which necessitate the development of reasonable prediction models for ship heave movements.In this paper,a novel model based on a time graph convolutional neural network algorithm and particle swarm optimization algorithm(PSO-TGCN)is proposed for the first time to predict the multipoint heave movements of ships under different sea conditions.To enhance the dataset's suitability for training and reduce interference,various filter algorithms are employed to optimize the dataset.The training process utilizes simulated heave data under different sea conditions and measured heave data from multiple points.The results show that the PSO-TGCN model predicts the ship swaying motion in different sea states after 2 s with 84.7%accuracy,while predicting the swaying motion in three different positions.By performing a comparative study,it was also found that the present method achieves better performance that other popular methods.This model can provide technical support for intelligent ship control,improve the control accuracy of intelligent ships,and promote the development of intelligent ships.

Polar motion prediction using the combination of SSA and ARMA

High-precision polar motion(PM) prediction is of important significance in astronomy, geodesy, aviation,hydrographic mapping, interstellar navigation, and so on. SSA can effectively extract the trend and period terms of PM,in the process of achieving high-precision medium-and long-term polar motion prediction,it is necessary to solve the end effect problem and overfitting problem of SSA forecasting method;therefore, ARMA was applied to decreasethe end effect, and a suitable combination of reconstructed components was determined to avoid the high variance reaction of SSA overfitting. Based on the decomposition and reconstruction of the PM by SSA, the reconstructed components are determined to participate in the SSA iterative fitting model according to the variance contribution rate. The combination of the reconstructed components representing the polar motion period term and the trend term is determined according to the correlation analysis of the selected reconstructed components. After the above work, the principal component prediction sequence is obtained by fitting the period term and the trend term to convergence, respectively, and then, the SSA end effect is modified, and the residual term is predicted based on ARMA. The test results show that he prediction accuracy of SSA + ARMA at the front of the X and Y directions are improved by 96.90% and 97.53% compared with those of SSA, respectively,and the forecast accuracy of 365 days are improved by 37.93% and 19.53% in the X and Y directions compared with those of Bulletin A, respectively.

Vehicle Motion Prediction at Intersections Based on the Turning Intention and Prior Trajectories Model

Intersections are quite important and complex traffic scenarios,where the future motion of surrounding vehicles is an indispensable reference factor for the decision-making or path planning of autonomous vehicles.Considering that the motion trajectory of a vehicle at an intersection partly obeys the statistical law of historical data once its driving intention is determined,this paper proposes a long short-term memory based(LSTM-based)framework that combines intention prediction and trajectory prediction together.First,we build an intersection prior trajectories model(IPTM)by clustering and statistically analyzing a large number of prior traffic flow trajectories.The prior trajectories model with fitted probabilistic density is used to approximate the distribution of the predicted trajectory,and also serves as a reference for credibility evaluation.Second,we conduct the intention prediction through another LSTM model and regard it as a crucial cue for a trajectory forecast at the early stage.Furthermore,the predicted intention is also a key that is associated with the prior trajectories model.The proposed framework is validated on two publically released datasets,next generation simulation(NGSIM)and INTERACTION.Compared with other prediction methods,our framework is able to sample a trajectory from the estimated distribution,with its accuracy improved by about 20%.Finally,the credibility evaluation,which is based on the prior trajectories model,makes the framework more practical in the real-world applications.

A new polar motion prediction method combined with the difference between polar motion series

After the first Earth Orientation Parameters Prediction Comparison Campaign(1 st EOP PCC),the traditional method using least-squares extrapolation and autoregressive(LS+AR)models was considered as one of the polar motion prediction methods with higher accuracy.The traditional method predicts individual polar motion series separately,which has a single input data and limited improvement in prediction accuracy.To address this problem,this paper proposes a new method for predicting polar motion by combining the difference between polar motion series.The X,Y,and Y-X series were predicted separately using LS+AR models.Then,the new forecast value of X series is obtained by combining the forecast value of Y series with that of Y-X series;the new forecast value of Y series is obtained by combining the forecast value of X series with that of Y-X series.The hindcast experimental comparison results from January 1,2011 to April 4,2021 show that the new method achieves a maximum improvement of 12.95%and 14.96%over the traditional method in the X and Y directions,respectively.The new method has obvious advantages compared with the differential method.This study tests the stability and superiority of the new method and provides a new idea for the research of polar motion prediction.

Improving longitudinal motion prediction of hybrid monohulls with the viscous effect

A new method improves prediction of the motion of a hybrid monohull in regular waves. Stem section hydrodynamic coefficients of a hybrid monohull with harmonic oscillation were computed using the Reynolds Averaged Navier-Stokes Equations （RANSE）. The governing equations were solved using the finite volume method. The VOF method was used for free surface treatment, and RNGK-ε turbulence model was employed in viscous flow calculation. The whole computational domain was divided into many blocks each with structured grids, and the dynamic process was treated with moving grids. Using a 2-D strip method and 2.5D theory with the correction hydrodynamic coefficients allows consideration of the viscous effect when predicting longitudinal motion of a hybrid monohull in regular waves. The method is effective at predicting motion of a hybrid monohull, showing that the viscous effect on a semi-submerged body cannot be ignored.

A Lightweight Temporal Convolutional Network for Human Motion Prediction

A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain the spatial structure information of human motion and extract the correlation in the time series of human motion.The residual structure is applied to the proposed network model to alleviate the problem of gradient disappearance in the deep network.Experiments on the Human 3.6M dataset demonstrate that the proposed method effectively reduces the errors of motion prediction compared with previous methods,especially of long-term prediction.

STTG-net:a Spatio-temporal network for human motion prediction based on transformer and graph convolution network

In recent years,human motion prediction has become an active research topic in computer vision.However,owing to the complexity and stochastic nature of human motion,it remains a challenging problem.In previous works,human motion prediction has always been treated as a typical inter-sequence problem,and most works have aimed to capture the temporal dependence between successive frames.However,although these approaches focused on the effects of the temporal dimension,they rarely considered the correlation between different joints in space.Thus,the spatio-temporal coupling of human joints is considered,to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network(GCN)(STTG-Net).The temporal transformer is used to capture the global temporal dependencies,and the spatial GCN module is used to establish local spatial correlations between the joints for each frame.To overcome the problems of error accumulation and discontinuity in the motion prediction,a revision method based on fusion strategy is also proposed,in which the current prediction frame is fused with the previous frame.The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods.The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.

A HEVC Video Steganalysis Method Using the Optimality of Motion Vector Prediction

Among steganalysis techniques,detection against MV(motion vector)domain-based video steganography in the HEVC(High Efficiency Video Coding)standard remains a challenging issue.For the purpose of improving the detection performance,this paper proposes a steganalysis method that can perfectly detectMV-based steganography in HEVC.Firstly,we define the local optimality of MVP(Motion Vector Prediction)based on the technology of AMVP(Advanced Motion Vector Prediction).Secondly,we analyze that in HEVC video,message embedding either usingMVP index orMVD(Motion Vector Difference)may destroy the above optimality of MVP.And then,we define the optimal rate of MVP as a steganalysis feature.Finally,we conduct steganalysis detection experiments on two general datasets for three popular steganographymethods and compare the performance with four state-ofthe-art steganalysis methods.The experimental results demonstrate the effectiveness of the proposed feature set.Furthermore,our method stands out for its practical applicability,requiring no model training and exhibiting low computational complexity,making it a viable solution for real-world scenarios.

A Hybrid BPNN-GARF-SVR Prediction Model Based on EEMD for Ship Motion

Accurate prediction of shipmotion is very important for ensuringmarine safety,weapon control,and aircraft carrier landing,etc.Ship motion is a complex time-varying nonlinear process which is affected by many factors.Time series analysis method and many machine learning methods such as neural networks,support vector machines regression(SVR)have been widely used in ship motion predictions.However,these single models have certain limitations,so this paper adopts amulti-model prediction method.First,ensemble empirical mode decomposition(EEMD)is used to remove noise in ship motion data.Then the randomforest(RF)prediction model optimized by genetic algorithm(GA),back propagation neural network(BPNN)prediction model and SVR prediction model are respectively established,and the final prediction results are obtained by results of three models.And the weights coefficients are determined by the correlation coefficients,reducing the risk of prediction and improving the reliability.The experimental results show that the proposed combined model EEMD-GARF-BPNN-SVR is superior to the single predictive model and more reliable.The mean absolute percentage error(MAPE)of the proposed model is 0.84%,but the results of the single models are greater than 1%.

Robotic Etiquette:Socially Acceptable Navigation of Service Robots with Human Motion Pattern Learning and Prediction

Nonverbal and noncontact behaviors play a significant role in allowing service robots to structure their interactions withhumans.In this paper, a novel human-mimic mechanism of robot’s navigational skills was proposed for developing sociallyacceptable robotic etiquette.Based on the sociological and physiological concerns of interpersonal interactions in movement,several criteria in navigation were represented by constraints and incorporated into a unified probabilistic cost grid for safemotion planning and control, followed by an emphasis on the prediction of the human’s movement for adjusting the robot’spre-collision navigational strategy.The human motion prediction utilizes a clustering-based algorithm for modeling humans’indoor motion patterns as well as the combination of the long-term and short-term tendency prediction that takes into accountthe uncertainties of both velocity and heading direction.Both simulation and real-world experiments verified the effectivenessand reliability of the method to ensure human’s safety and comfort in navigation.A statistical user trials study was also given tovalidate the users’favorable views of the human-friendly navigational behavior.

Human motion prediction using optimized sliding window polynomial fitting and recursive least squares

Human motion prediction is a critical issue in human-robot collaboration(HRC)tasks.In order to reduce the local error caused by the limitation of the capture range and sampling frequency of the depth sensor,a hybrid human motion prediction algorithm,optimized sliding window polynomial fitting and recursive least squares(OSWPF-RLS)was proposed.The OSWPF-RLS algorithm uses the human body joint data obtained under the HRC task as input,and uses recursive least squares(RLS)to predict the human movement trajectories within the time window.Then,the optimized sliding window polynomial fitting(OSWPF)is used to calculate the multi-step prediction value,and the increment of multi-step prediction value was appropriately constrained.Experimental results show that compared with the existing benchmark algorithms,the OSWPF-RLS algorithm improved the multi-step prediction accuracy of human motion and enhanced the ability to respond to different human movements.

Improvement of the prediction accuracy of polar motion using empirical mode decomposition

Previous studies revealed that the error of pole coordinate prediction will significantly increase for a prediction period longer than 100 days, and this is mainly caused by short period oscillations. Empirical mode decomposition （EMD）, which is increasingly popular and has advantages over classical wavelet decomposition, can be used to remove short period variations from observed time series of pole co- ordinates. A hybrid model combing EMD and extreme learning machine （ELM）, where high frequency signals are removed and processed time series is then modeled and predicted, is summarized in this paper. The prediction performance of the hybrid model is compared with that of the ELM-only method created from original time series. The results show that the proposed hybrid model outperforms the pure ELM method for both short-term and long-term prediction of pole coordinates. The improvement of prediction accuracy up to 360 days in the future is found to be 24.91% and 26.79% on average in terms of mean absolute error （MAE） for the xp and yp components of pole coordinates, respectively.

Design of Human Adaptive Mechatronics Controller for Upper Limb Motion Intention Prediction

Human Adaptive Mechatronics(HAM)includes human and computer system in a closed loop.Elderly person with disabilities,normally carry out their daily routines with some assistance to move their limbs.With the short fall of human care takers,mechatronics devices are used with the likes of exoskeleton and exosuits to assist them.The rehabilitation and occupational therapy equipments utilize the electromyography(EMG)signals to measure the muscle activity potential.This paper focuses on optimizing the HAM model in prediction of intended motion of upper limb with high accuracy and to increase the response time of the system.Limb characteristics extraction from EMG signal and prediction of optimal controller parameters are modeled.Time and frequency based approach of EMG signal are considered for feature extraction.The models used for estimating motion and muscle parameters from EMG signal for carrying out limb movement predictions are validated.Based on the extracted features,optimal parameters are selected by Modified Lion Optimization(MLO)for controlling the HAM system.Finally,supervised machine learning makes predictions at different points in time for individual sensing using Support Vector Neural Network(SVNN).This model is also evaluated based on optimal parameters of motion estimation and the accuracy level along with different optimization models for various upper limb movements.The proposed model of human adaptive controller predicts the limb movement by 96%accuracy.

Characteristics of horizontal ground motion measures along principal directions

Ground motion records are often used to develop ground motion prediction equations （GMPEs） for a randomly oriented horizontal component, and to assess the principal directions of ground motions based on the Arias intensity tensor or the orientation of the major response axis. The former is needed for seismic hazard assessment, whereas the latter can be important for assessing structural responses under multi-directional excitations. However, a comprehensive investigation of the pseudo-spectral acceleration （PSA） and of GMPEs conditioned on different axes is currently lacking. This study investigates the principal directions of strong ground motions and their relation to the orientation of the major response axis, statistics of the PSA along the principal directions on the horizontal plane, and correlation of the PSA along the principal directions on the horizontal plane. For these, three sets of strong ground motion records, including intraplate California earthquakes, inslab Mexican earthquakes, and interface Mexican earthquakes, are used. The results indicate that one of the principal directions could be considered as quasi-vertical. By focusing on seismic excitations on the horizontal plane, the statistics of the angles between the major response axis and the major principal axis are obtained; GMPEs along the principal axes are provided and compared with those obtained for a randomly oriented horizontal component; and statistical analysis of residuals associated with GMPEs along the principal directions is carried out.

Comparison of strong-motion records and damage implications between the 2014 Yunnan M_S6.5 Ludian earthquake and M_S6.6 Jinggu earthquake

Serial destructive earthquakes have caused heavy casualties and economic losses to the city in southwestern of China. The Ludian Ms 6.5 earthquake and the Jinggu Ms 6.6 earthquake occurred in Yunnan province in 2014. There is a question of why the two events with almost the same level of magnitude caused differences in earthquake damage. To understand the uniqueness of the phenomenon, this paper focuses on the characteristics of the ground motions and post-earthquake field investigation for the two events. Firstly, we present an overview of the residuals between the Ludian earthquake and the Jinggu earthquake based on the YW06 Ground Motion Prediction Equation （GMPE）, and explain the unusual destructiveness of the strong ground motion. Then we analyze the ground motion recordings at selected typical station, based on the strong motion parameters： equivalent predominant frequency and Arias intensity. The result exhibits a good agreement with the Chinese seismic intensity scale. This study would be helpful to gain a better knowledge of the characteristics and variability of ground motions for Ms6 class earthquakes in China and to understand the implications to future earthquakes with similar focal mechanism and local condition.

Diving dynamics identification and motion prediction for marine crafts using field data

Ensuring accurate parameter identification and diving motion prediction of marine crafts is essential for safe navigation,optimized operational efficiency,and the advancement of marine exploration.Addressing this,this paper proposes an instrumental variable-based least squares(IVLS)algorithm.Firstly,aiming to balance complexity with accuracy,a decoupled diving model is constructed,incorporating nonlinear actuator characteristics,inertia coefficients,and damping coefficients.Secondly,a discrete parameter identification matrix is designed based on this dedicated model,and then a IVLS algorithm is innovatively derived to reject measurement noise.Furthermore,the stability of the proposed algorithm is validated from a probabilistic point of view,providing a solid theoretical foundation.Finally,performance evaluation is conducted using four depth control datasets obtained from a piston-driven profiling float in Qiandao Lake,with desired depths of 30 m,40 m,50 m,and 60 m.Based on the diving dynamics identification results,the IVLS algorithm consistently shows superior performance when compared to recursive weighted least squares algorithm and least squares support vector machine algorithm across all depths,as evidenced by lower average absolute error(AVGAE),root mean square error(RMSE),and maximum absolute error values and higher determination coefficient(R2).Specifically,for desired depth of 60 m,the IVLS algorithm achieved an AVGAE of 0.553 m and RMSE of 0.655 m,significantly outperforming LSSVM with AVGAE and RMSE values of 8.782 m and 11.117 m,respectively.Moreover,the IVLS algorithm demonstrates a remarkable generalization capability with R2 values consistently above 0.95,indicating its robustness in handling varied diving dynamics.

Engineering Analysis of Strong Motion Data from Recent Earthquakes in Sichuan, China

Recent earthquakes in the Sichuan Province have contributed to significantly expand the existing ground-motion database for China with new,high-quality ground-motion records.This study investigated the compatibility of ground-motion prediction equations(GMPEs)established by the NGA-West2 project in the US and local GMPEs for China,with respect to magnitude scaling,distance scaling,and site scaling implied by recent Chinese strong-motion data.The NGA-West2 GMPEs for shallow crustal earthquakes in tectonically active regions are considerably more sophisticated than widely used previous models,particularly in China.Using a mixed-effects procedure,the study evaluated event terms(inter-event residuals)and intra-event residuals of Chinese data relative to the NGA-West2 GMPEs.Distance scaling was investigated by examining trends of intra-event residuals with source-to-site distance.Scaling with respect to site conditions was investigated by examining trends of intra-event residuals with soil type.The study also investigated other engineering characteristics of Chinese strong ground motions.In particular,the records were analyzed for evidence of pulse-like forward-directivity effects.The elastic median response spectra of the selected stations were compared to code-mandated design spectra for various mean return periods.Results showed that international and local GMPEs can be applied for seismic hazard analysis in Sichuan with minor modification of the regression coefficients related to the source-to-site distance and soil scaling.Specifically,the Chinese data attenuated faster than implied by the considered GMPEs and the differences were statistically significant in some cases.Near-source,pulse-like ground motions were identified at two recording stations for the 2008 Wenchuan earthquake,possibly implying rupture directivity.The median recorded spectra were consistent with the code-based spectra in terms of amplitude and shape.The new ground-motion data can be used to develop advanced ground-motion models for China and worldwide and,ultimately,for advancing probabilistic seismic hazard assessment(PSHA).

Fast Sub-Pixel Motion Estimation Algorithm For H.264

A novel fast sub-pixel search algorithm is proposed to accelerate sub-pixel search. Based on the features of predicted motion vector (PMV) and texture direction observed, the proposed method effectively filters out impossible points and thus decreases 11 searched points in average during the sub-pixel search stage. A threshold is also adopted to early terminate the sub-pixel search. Simulation results show that the proposed method can achieve up to 4.8 times faster than full sub-pixel motion search scheme (FSPS) with less than 0.025 dB PSNR losses and 2.2% bit-length increases.

Online Observability-Constrained Motion Suggestion via Efficient Motion Primitive-Based Observability Analysis

An active perception methodology is proposed to locally predict the observability condition in a reasonable horizon and suggest an observability-constrained motion direction for the next step to ensure an accurate and consistent state estimation performance of vision-based navigation systems. The methodology leverages an efficient EOG-based observability analysis and a motion primitive-based path sampling technique to realize the local observability prediction with a real-time performance. The observability conditions of potential motion trajectories are evaluated,and an informed motion direction is selected to ensure the observability efficiency for the state estimation system. The proposed approach is specialized to a representative optimizationbased monocular vision-based state estimation formulation and demonstrated through simulation and experiments to evaluate the ability of estimation degradation prediction and efficacy of motion direction suggestion.