Research on Anthropomorphic Obstacle Avoidance Trajectory Planning for Adaptive Driving Scenarios Based on Inverse Reinforcement Learning Theory

The forward design of trajectory planning strategies requires preset trajectory optimization functions,resulting in poor adaptability of the strategy and an inability to accurately generate obstacle avoidance trajectories that conform to real driver behavior habits.In addition,owing to the strong time-varying dynamic characteristics of obstacle avoidance scenarios,it is necessary to design numerous trajectory optimization functions and adjust the corresponding parameters.Therefore,an anthropomorphic obstacle-avoidance trajectory planning strategy for adaptive driving scenarios is proposed.First,numerous expert-demonstrated trajectories are extracted from the HighD natural driving dataset.Subsequently,a trajectory expectation feature-matching algorithm is proposed that uses maximum entropy inverse reinforcement learning theory to learn the extracted expert-demonstrated trajectories and achieve automatic acquisition of the optimization function of the expert-demonstrated trajectory.Furthermore,a mapping model is constructed by combining the key driving scenario information that affects vehicle obstacle avoidance with the weight of the optimization function,and an anthropomorphic obstacle avoidance trajectory planning strategy for adaptive driving scenarios is proposed.Finally,the proposed strategy is verified based on real driving scenarios.The results show that the strategy can adjust the weight distribution of the trajectory optimization function in real time according to the“emergency degree”of obstacle avoidance and the state of the vehicle.Moreover,this strategy can generate anthropomorphic trajectories that are similar to expert-demonstrated trajectories,effectively improving the adaptability and acceptability of trajectories in driving scenarios.

HYBRID MULTI-OBJECTIVE GRADIENT ALGORITHM FOR INVERSE PLANNING OF IMRT

The intelligent optimization of a multi-objective evolutionary algorithm is combined with a gradient algorithm. The hybrid multi-objective gradient algorithm is framed by the real number. Test functions are used to analyze the efficiency of the algorithm. In the simulation case of the water phantom, the algorithm is applied to an inverse planning process of intensity modulated radiation treatment （IMRT）. The objective functions of planning target volume （PTV） and normal tissue （NT） are based on the average dose distribution. The obtained intensity profile shows that the hybrid multi-objective gradient algorithm saves the computational time and has good accuracy, thus meeting the requirements of practical applications.

A multiphase direct aperture optimization for inverse planning in radiotherapy

Optimization of the inverse planning becomes critical because it follows the invention of intensity modulated radiotherapy(IMRT) to shorten the previous "trial-and-error" treatment process and increase efficiency.In this paper, the inverse planning is used to direct aperture optimization in the ARTS(Accurate/Advanced Radiotherapy System). The objective function was quadratic, both tolerance and dose-volume constraint types are supported. The memory efficient conjugate gradient algorithm is used to cope with its large data.Furthermore, to fully exploit the solution space, a shortest path sub-procedure is coupled into the whole algorithm, thus giving further possibility decreasing the objective function. Two clinical cases are tested, indicating that the applicability of this algorithm is promising to clinical usage.

Evaluating the influence of 6 MV and 15 MV photon beams on prostate intensity-modulated radiation therapy plans

Objective We aimed to determine the ef ects of low- and high-energy intensity-modulated radiation therapy （IMRT） photon beams on the target volume planning and on the critical organs in the case of prostate can-cer. Methods Thirty plans were generated by using either 6 MV or 15 MV beams separately, and a combination of both 6 and 15 MV beams. Al plans were generated by using suitable planning objectives and dose con-straints, which were identical across the plans, except the beam energy. The plans were analyzed in terms of their target coverage, conformity, and homogeneity, regardless of the beam energy. Results The mean percentage values of V70 Gy for the rectal wal for the plans with 6 MV, 15 MV, and mixed-energy beams were 16.9%, 17.8%, and 16.4%, respectively, while the mean percentage values of V40 Gy were 53.6%, 52.3%, and 50.4%. The mean dose values to the femoral heads for the 6 MV, 15 MV, and mixed-en-ergy plans were 30.1 Gy, 25.5 Gy, and 25.4 Gy, respectively. The mean integral dose for the 6 MV plans was 10% larger than those for the 15 MV and mixed-energy plans.Conclusion These preliminary results suggest that mixed-energy IMRT plans may be advantageous with respect to the dosimetric characteristics of low- and high-energy beams. Although the reduction of dose to the organs at risk may not be clinical y relevant, in this study, IMRT plans using mixed-energy beams exhibited better OAR sparing and overal higher plan quality for deep-seated tumors.

Recognition and interfere deceptive behavior based on inverse reinforcement learning and game theory

In real-time strategy(RTS)games,the ability of recognizing other players’goals is important for creating artifical intelligence(AI)players.However,most current goal recognition methods do not take the player’s deceptive behavior into account which often occurs in RTS game scenarios,resulting in poor recognition results.In order to solve this problem,this paper proposes goal recognition for deceptive agent,which is an extended goal recognition method applying the deductive reason method(from general to special)to model the deceptive agent’s behavioral strategy.First of all,the general deceptive behavior model is proposed to abstract features of deception,and then these features are applied to construct a behavior strategy that best matches the deceiver’s historical behavior data by the inverse reinforcement learning(IRL)method.Final,to interfere with the deceptive behavior implementation,we construct a game model to describe the confrontation scenario and the most effective interference measures.

Dosimetric consequences of tumor volume changes after kilovoltage cone-beam computed tomography for non-operative lung cancer during adaptive intensity-modulated radiotherapy or fractionated stereotactic radiotherapy

Objective The aim of this study was to investigate tumor volume changes with kilovoltage cone-beam computed tomography （kV-CBCT） and their dosimetric consequences for non-operative lung cancer during intensity-modulated radiotherapy （IMRT） or fractionated stereotactic radiotherapy. Methods Eighteen patients with non-operative lung cancer who received IMRT consisting of 1.8-2.2 Gy/fraction and five fractions per week or stereotactic radiotherapy with 5-8 Gy/fraction and three fractions a week were studied, kV-CBCT was performed once per week during IMRT and at every fraction during stereotactic radiotherapy. The gross tumor volume （GTV） was contoured on the kV-CBCT images, and adaptive treatment plans were created using merged kV-CBCT and primary planning computed tomogra- phy image sets. Tumor volume changes and dosimetric parameters, including the minimum dose to 95% （D95） or 1% （D1） of the planning target volume （PTV）, mean lung dose （MLD）, and volume of lung tissue that received more than 5 （Vs）, 10 （Vl0）, 20 （V20）, and 30 （V30） Gy were retrospectively analyzed. Results The average maximum change in GTV observed during IMRT or fractionated stereotactic radio- therapy was -25.85% （range, -13.09% --56.76%）. The D95 and Dr of PTV for the adaptive treatment plans in all patients were not significantly different from those for the initial or former adaptive treatment plans. In patients with tumor volume changes of 〉20% in the third or fourth week of treatment during IMRT, adap- tive treatment plans offered clinically meaningful decreases in MLD and V5, V10, V20, and V30; however, in patients with tumor volume changes of 〈 20% in the third or fourth week of treatment as well as in patients with stereotactic radiotherapy, there were no significant or clinically meaningful decreases in the dosimetric parameters. Conclusion Adaptive treatment planning for decreasing tumor volume during IMRT may be beneficial for patients who experience tumor volume changes of 〉20% in the third or fourth week of treatment.

A Path Planning Algorithm Based on Improved RRT Sampling Region

For the problem of slow search and tortuous paths in the Rapidly Exploring Random Tree(RRT)algorithm,a feedback-biased sampling RRT,called FS-RRT,is proposedbasedon RRT.Firstly,toimprove the samplingefficiency of RRT to shorten the search time,the search area of the randomtree is restricted to improve the sampling efficiency.Secondly,to obtain better information about obstacles to shorten the path length,a feedback-biased sampling strategy is used instead of the traditional random sampling,the collision of the expanding node with an obstacle generates feedback information so that the next expanding node avoids expanding within a specific angle range.Thirdly,this paper proposes using the inverse optimization strategy to remove redundancy points from the initial path,making the path shorter and more accurate.Finally,to satisfy the smooth operation of the robot in practice,auxiliary points are used to optimize the cubic Bezier curve to avoid path-crossing obstacles when using the Bezier curve optimization.The experimental results demonstrate that,compared to the traditional RRT algorithm,the proposed FS-RRT algorithm performs favorably against mainstream algorithms regarding running time,number of search iterations,and path length.Moreover,the improved algorithm also performs well in a narrow obstacle environment,and its effectiveness is further confirmed by experimental verification.

Motion Planning Algorithms of Redundant Manipulators Based on Self-motion Manifolds

The current motion planning approaches for redundant manipulators mainly includes two categories： improved gradient-projection method and some other efficiency numerical methods. The former is excessively sensitive to parameters, which makes adjustment difficult; and the latter treats the motion planning as general task by ignoring the particularity, which has good universal property but reduces the solving speed for on-line real-time planning. In this paper, a novel stepwise solution based on self-motion manifold is proposed for motion planning of redundant manipulators, namely, the chief tasks and secondary tasks are implemented step by step. Firstly, the posture tracking of end-effector is achieved accurately by employing the non-redundant joint. Secondly, the end-effector is set to keep stationary. Finally, self-motion of manipulator is realized via additional work on the gradient of redundant joint displacement. To verify this solution, experiments of round obstacle avoiding are carried out via the planar 3 degree-of-~eedom manipulator. And the experimental results indicate that this motion planning algorithm can effectively achieve obstacle avoiding and posture tracking of the end-effector. Compared with traditional gradient projection method, this approach can accelerate the problem-solving process, and is more applicable to obstacle avoiding and other additional work in displacement level.

Algebraic Method‑Based Point‑to‑Point Trajectory Planning of an Under‑Constrained Cable‑Suspended Parallel Robot with Variable Angle and Height Cable Mast

To avoid impacts and vibrations during the processes of acceleration and deceleration while possessing flexible working ways for cable-suspended parallel robots(CSPRs),point-to-point trajectory planning demands an under-constrained cable-suspended parallel robot(UCPR)with variable angle and height cable mast as described in this paper.The end-effector of the UCPR with three cables can achieve three translational degrees of freedom(DOFs).The inverse kinematic and dynamic modeling of the UCPR considering the angle and height of cable mast are completed.The motion trajectory of the end-effector comprising six segments is given.The connection points of the trajectory segments(except for point P3 in the X direction)are devised to have zero instantaneous velocities,which ensure that the acceleration has continuity and the planned acceleration curve achieves smooth transition.The trajectory is respectively planned using three algebraic methods,including fifth degree polynomial,cycloid trajectory,and double-S velocity curve.The results indicate that the trajectory planned by fifth degree polynomial method is much closer to the given trajectory of the end-effector.Numerical simulation and experiments are accomplished for the given trajectory based on fifth degree polynomial planning.At the points where the velocity suddenly changes,the length and tension variation curves of the planned and unplanned three cables are compared and analyzed.The OptiTrack motion capture system is adopted to track the end-effector of the UCPR during the experiment.The effectiveness and feasibility of fifth degree polynomial planning are validated.

Planar Serial Manipulator Motion Synthesis

： This paper deals with the universal serial manipulator on the inverse kinematics problem of plane type, the fast working space solution method, and the obstacle avoidance path planning method. With the vector projection as the main constraint condition of the target, it proposes a general form of the inverse kinematics solution which does not depend on the robot configuration of freedom degree. By identifying the target vector direction maximum and minimum workspace boundary and determining the destination vector by thick search on the workspaee boundary method, an expressing method of the polar coordinate form of work space is then introduced. Finally, according to the form of plane trajectory planning for obstacle avoidance problem, the method of solving the inverse kinematics solution of the concave and convex forms of the safe obstacle avoidance area is improved. The simulation results verify that the proposed method has feasibility and generality.

Comparison between 4D robust optimization methods for carbon-ion treatment planning

Intensity-modulated particle therapy(IMPT)with carbon ions is comparatively susceptible to various uncertainties caused by breathing motion,including range,setup,and target positioning uncertainties.To determine relative biological effectiveness-weighted dose(RWD)distributions that are resilient to these uncertainties,the reference phase-based four-dimensional(4D)robust optimization(RP-4DRO)and each phase-based 4D robust optimization(EP-4DRO)method in carbon-ion IMPT treatment planning were evaluated and compared.Based on RWD distributions,4DRO methods were compared with 4D conventional optimization using planning target volume(PTV)margins(PTV-based optimization)to assess the effectiveness of the robust optimization methods.Carbon-ion IMPT treatment planning was conducted in a cohort of five lung cancer patients.The results indicated that the EP-4DRO method provided better robustness(P=0.080)and improved plan quality(P=0.225)for the clinical target volume(CTV)in the individual respiratory phase when compared with the PTV-based optimization.Compared with the PTV-based optimization,the RP-4DRO method ensured the robustness(P=0.022)of the dose distributions in the reference breathing phase,albeit with a slight sacrifice of the target coverage(P=0.450).Both 4DRO methods successfully maintained the doses delivered to the organs at risk(OARs)below tolerable levels,which were lower than the doses in the PTV-based optimization(P<0.05).Furthermore,the RP-4DRO method exhibited significantly superior performance when compared with the EP-4DRO method in enhancing overall OAR sparing in either the individual respiratory phase or reference respiratory phase(P<0.05).In general,both 4DRO methods outperformed the PTV-based optimization in terms of OAR sparing and robustness.

一种用于冗余机械臂解析逆运动学的多目标优化方法

为实现工程机械的多自由度臂架智能运动控制,本文结合参数化关节方法,改进MOEA/D形成:IM-IK算法(Improved MOEA/D-inverse kinematics),来解决冗余机械臂逆运动学问题,该算法在MOEA/D算法框架中加入自适应区间搜索策略以提升算法局部搜索能力,提高求解速度。本文将IM-IK算法应用到8自由度冗余机械臂上,与NSGA-II、MOEA/D及T-IK算法进行了试验对比,各项试验指标均证明了IM-IK算法在处理冗余机械臂的逆运动学问题上具有优于其他算法的综合性能,可在满足机械臂末端执行器位姿精度要求的前提下,同时保障机械臂关节运动连续性和有效避开关节极限。

基于梯度下降的串联机械臂逆运动学求解

将梯度下降法用于机械臂的逆运动学求解,解决了不同自由度串联机械臂逆运动学求解的问题;将求解过程中间解利用起来,在保证了系统性能的同时,完成了轨迹规划任务;针对使用梯度下降得到的轨迹较差的问题,采用改进的梯度下降对轨迹进行了优化。用OpenGL和C++搭建了仿真平台,对六自由度和七自由度机械臂进行了仿真,算法收敛精度可达到0.01 mm,实验结果表明将梯度下降用于机械臂逆运动学求解具有实用性。

Multi-objective optimization of inverse planning for accurate radiotherapy

The multi-objective optimization of inverse planning based on the Pareto solution set, according to the multi-objective character of inverse planning in accurate radiotherapy, was studied in this paper. Firstly, the clinical requirements of a treatment plan were transformed into a multi-objective optimization problem with multiple constraints. Then, the fast and elitist multi-objective Non-dominated Sorting Genetic Algorithm （NSGA-） was introduced to optimize the problem. A clinical example was tested using this method. The results show that an obtained set of non-dominated solutions were uniformly distributed and the corresponding dose distribution of each solution not only approached the expected dose distribution, but also met the dosevolume constraints. It was indicated that the clinical requirements were better satisfied using the method and the planner could select the optimal treatment plan from the non-dominated solution set.

Real-time trajectory planning for UCAV air-to-surface attack using inverse dynamics optimization method and receding horizon control

This paper presents a computationally efficient real-time trajectory planning framework for typical unmanned combat aerial vehicle （UCAV） performing autonomous air-to-surface （A/S） attack. It combines the benefits of inverse dynamics optimization method and receding horizon optimal control technique. Firstly, the ground attack trajectory planning problem is mathematically formulated as a receding horizon optimal control problem （RHC-OCP）. In particular, an approximate elliptic launch acceptable region （LAR） model is proposed to model the critical weapon delivery constraints. Secondly, a planning algorithm based on inverse dynamics optimization, which has high computational efficiency and good convergence properties, is developed to solve the RHCOCP in real-time. Thirdly, in order to improve robustness and adaptivity in a dynamic and uncer- tain environment, a two-degree-of-freedom （2-DOF） receding horizon control architecture is introduced and a regular real-time update strategy is proposed as well, and the real-time feedback can be achieved and the not-converged situations can be handled. Finally, numerical simulations demon- strate the efficiency of this framework, and the results also show that the presented technique is well suited for real-time implementation in dynamic and uncertain environment.

多机协调吊运系统点到点轨迹规划策略研究

针对速度与加速度不连续的被吊运物轨迹不能直接运用于多机器人协调吊运系统的逆运动学求解问题,提出一种基于点到点轨迹规划的逆运动学求解策略。首先,建立了该系统运动学与动力学模型。然后,对系统的逆运动学进行了分析,并对不同解的情况给出了解决办法。随后,提出了基于点到点轨迹规划求解可行逆解的策略,利用三种代数规划方法对被吊运物轨迹进行规划,在保证规划后的速度与加速度连续的基础上,结合逆运动学不同解的处理办法,得到更具有普适意义的可行逆解。最后,通过两个实例仿真验证了该策略的可行性,且只有五次多项式规划法符合拉力条件,研究结果为系统张力优化分布和控制奠定了基础。

基于残差BP神经网络的Baxter机器人逆运动学分析方法

提出1种基于残差BP(back propagation)神经网络的自适应逆运动学分析方法,围绕数据采集至实时控制的整个运动规划流程,采集140组位置和欧拉角数据,利用残差BP神经网络对Baxter机械臂进行逆运动学分析,拟合得到机械臂7个关节角度;将训练好的关节角度以话题的形式发布,通过在抓取物体的脚本中订阅该话题实现通讯;结合Rviz进行可视化展示和实物双臂协同实验,对4种物体模型分别用残差BP神经网络和普通BP神经网络进行抓取实验,验证所提方法的有效性。结果表明:所提方法的计算单点时间约8.1 ms,远小于机械臂的控制周期,可实现实时性的要求;在进行1500次训练的情况下,残差BP神经网络模型的均方误差为0.006,相比普通BP神经网络模型,误差降低0.077,提高了模型的准确性;所提方法的抓取成功率为87.5%,比普通BP神经网络提高了22.5%,验证了本文所提方法的有效性和实用性。

基于多自由度机械臂的乒乓球平衡控制算法研究

以板球系统模型为基础,提出了使用多自由度串联机械臂控制乒乓球平衡的方法。首先,从板球控制系统出发,分析这类平衡算法的优缺点;其次,对六自由度串联机械臂进行运动学分析;最后,通过Matlab软件和Webots软件对机械臂进行仿真试验,验证其准确性。仿真结果表明:在一般的应用场景下,使用多自由度串联机械臂控制乒乓球平衡的方法具有一定的通用性。

基于动作捕捉系统对下肢外骨骼运动分析及步态仿真

结合生物医学工程,设计出一种人体下肢外骨骼。通过光学动作捕捉系统采集人体正常行走时的位置坐标,分析采集数据,从而获得人体正常行走时的髋、膝和踝关节的角度变化;建立自由度为5的下肢外骨骼运动学模型,随后通过逆运动学求解出各个关节的角度变化,运用于步态规划中;在SolidWorks中建立下肢外骨骼三维模型,运用ADAMS进行动力学仿真,其中对仿真参数的设定为步态规划中求解的各关节角度,通过虚拟样机完成仿真动作,最后将所得的仿真数据与光学动作捕捉系统采集的数据进行对比。数据对比结果显示,下肢外骨骼能基本实现人体的正常行走,确定了模型建立的可适用性、运动学分析与动力学仿真准确性,为后期的研究提供基准。

基于CNN的康复机器人逆运动学分析与轨迹规划

为提高脑卒中患者上肢康复训练的运动舒适性,设计一种具有运动依从性的上肢康复机器人。用D-H参数法建立机器人模型,进行正运动学分析,针对传统逆运动学求解方法的高复杂性,搭建一种卷积神经网络进行逆运动学求解,96.24%的预测角度所计算得到的关节位置,与标签位置误差在0~0.025m范围内。采用解析法分析机器人的工作空间,使用七次多项式插值法规划了一组康复训练动作,仿真结果符合患者的康复运动要求,能够提供高质量康复训练。