Balance Control of a Biped Robot on a Rotating Platform Based on Efficient Reinforcement Learning

In this work,we combined the model based reinforcement learning(MBRL)and model free reinforcement learning(MFRL)to stabilize a biped robot(NAO robot)on a rotating platform,where the angular velocity of the platform is unknown for the proposed learning algorithm and treated as the external disturbance.Nonparametric Gaussian processes normally require a large number of training data points to deal with the discontinuity of the estimated model.Although some improved method such as probabilistic inference for learning control(PILCO)does not require an explicit global model as the actions are obtained by directly searching the policy space,the overfitting and lack of model complexity may still result in a large deviation between the prediction and the real system.Besides,none of these approaches consider the data error and measurement noise during the training process and test process,respectively.We propose a hierarchical Gaussian processes(GP)models,containing two layers of independent GPs,where the physically continuous probability transition model of the robot is obtained.Due to the physically continuous estimation,the algorithm overcomes the overfitting problem with a guaranteed model complexity,and the number of training data is also reduced.The policy for any given initial state is generated automatically by minimizing the expected cost according to the predefined cost function and the obtained probability distribution of the state.Furthermore,a novel Q(λ)based MFRL method scheme is employed to improve the policy.Simulation results show that the proposed RL algorithm is able to balance NAO robot on a rotating platform,and it is capable of adapting to the platform with varying angular velocity.

Biped Walking Robot Based on a 2-UPU+2-UU Parallel Mechanism

Existing biped robots mainly fall into two categories： robots with left and right feet and robots with upper and lower feet. The load carrying capability of a biped robot is quite limited since the two feet of a walking robot supports the robot alternatively during walking. To improve the load carrying capability, a novel biped walking robot is proposed based on a 2-UPU＋2-UU parallel mechanism. The biped walking robot is composed of two identical platforms（feet） and four limbs, including two UPU（universal-prismatic-universal serial chain） limbs and two UU limbs. To enhance its terrain adaptability like articulated vehicles, the two feet of the biped walking robot are designed as two vehicles in detail. The conditions that the geometric parameters of the feet must satisfy are discussed. The degrees-of-freedom of the mechanism is analyzed by using screw theory. Gait analysis, kinematic analysis and stability analysis of the mechanism are carried out to verify the structural design parameters. The simulation results validate the feasibility of walking on rugged terrain. Experiments with a physical prototype show that the novel biped walking robot can walk stably on smooth terrain. Due to its unique feet design and high stiffness, the biped walking robot may adapt to rugged terrain and is suitable for load-carrying.

Biped Robot with Triangle Configuration

A new biped robot with a triangle configuration is presented and it is a planar closed chain mechanism. The scalability of three sides of the triangle is realized by three actuated prismatic joints. The three vertexes of the triangle are centers of three passive revolute joints coincidently. The biped mechanism for straight walking is proposed and its walking principle and mobility are explained. The static stability and the height and span of one step are analyzed. Kinematic analysis is performed to plan the gaits of walking on an even floor and going upstairs. A prototype is developed and experiments are carried out to validate the straight walking gait. Two additional revolute joints are added to form a modified biped robot which can follow the instruction of turning around. The turning ability is verified by experiments. As a new member of biped robots, its triangle configuration is used to impart geometry knowledge. Because of its high stiffness, some potential applications are on the way.

Dynamical analysis and performance evaluation of a biped robot under multi-source random disturbances

During bipedal walking,it is critical to detect and adjust the robot postures by feedback control to maintain its normal state amidst multi-source random disturbances arising from some unavoidable uncertain factors.The radical basis function（RBF）neural network model of a five-link biped robot is established,and two certain disturbances and a randomly uncertain disturbance are then mixed with the optimal torques in the network model to study the performance of the biped robot by several evaluation indices and a specific Poincar′e map.In contrast with the simulations,the response varies as desired under optimal inputting while the output is fluctuating in the situation of disturbance driving.Simulation results from noise inputting also show that the dynamics of the robot is less sensitive to the disturbance of knee joint input of the swing leg than those of the other three joints,the response errors of the biped will be increasing with higher disturbance levels,and especially there are larger output fluctuations in the knee and hip joints of the swing leg.

Adsorption Performance of Sliding Wall-Climbing Robot

Sliding wall-climbing robot （SWCR） is applied worldwide for its continuous motion, however, considerable air leakage causes two problems： great power consumption and big noise, and they constraint the robot＇s comprehensive performance. So far, effective theoretical model is still lacked to solve the problems. The concept of SWCR＇s adsorption performance is presented, and the techniques of improving utilization rate of given adsorption force and utilization rate of power are studied respectively to improve SWCR＇s adsorption performance. The effect of locomotion mechanism selection and seal＇s pressure allocation upon utilization rate of given adsorption force is discussed, and the theoretical way for relevant parameters optimization are provided. The directions for improving utilization rate of power are pointed out based on the detail analysis results of suction system＇s thermodynamics and hydrodynamics. On this condition, a design method for SWCR-specific impeller is presented, which shows how the impeller＇s key parameters impact its aerodynamic performance with the aid of computational fluid dynamics （CFD） simulations. The robot prototype, BIT Climber, is developed, and its functions such as mobility, adaptability on wall surface, payload, obstacle ability and wall surface inspection are tested. Through the experiments for the adhesion performance of the robot adsorption system on the normal wall surface, at the impeller＇s rated rotating speed, the total adsorption force can reach 237.2 N, the average effective negative pressure is 3.02 kPa and the design error is 3.8% only, which indicates a high efficiency. Furthermore, it is found that the robot suction system＇s static pressure efficiency reaches 84% and utilization rate of adsorption force 81% by the experiment. This thermodynamics model and SWCR-specific impeller design method can effectively improve SWCR＇s adsorption performance and expand this robot applicability on the various walls. A sliding wall-climbing robot with high adhesion efficiency is developed, and this robot has the features of light body in weight, small size in structure and good capability in payload.

Development and control of flexible pneumatic wall-climbing robot

A new kind of flexible pneumatic wall-climbing robot,named WALKMAN-I,was proposed. WALKMAN-I is basically composed of a flexible pneumatic actuator (FPA),a flexible pneumatic spherical joint and six suction cups. It has many characteristics of low-cost,lightweight,simple structure and good flexibility. Its operating principle was introduced. Then three basic locomotion modes,which are linear motion,curvilinear motion and crossing the orthogonal planes,were presented. The safety conditions of WALKMAN-I were discussed and built. Finally,the control system was designed and experiments were carried out. Experimental results show that WALKMAN-I is able to climb on the vertical wall surface along a straight line or a curved path,and has the ability of crossing orthogonal planes and obstacles. The maximum rotation angle reaches 90°,the maximum velocity reaches 5 mm/s,and the rotation angle and the moving velocity of WALKMAN-I can be easily controlled.

Stability and control of dynamic walking for a five-link planar biped robot with feet

During dynamic walking of biped robots, the underactuated rotating degree of freedom （DOF） emerges between the support foot and the ground, which makes the biped model hybrid and dimension-variant. This paper addresses the asymptotic orbit stability for dimension-variant hybrid systems （DVHS）. Based on the generalized Poincare map, the stability criterion for DVHS is also presented, and the result is then used to study dynamic walking for a five-link planar biped robot with feet. Time-invariant gait planning and nonlinear control strategy for dynamic walking with fiat feet is also introduced. Simulation results indicate that an asymptotically stable limit cycle of dynamic walking is achieved by the proposed method.

STUDIES ON MECHANICS PROBLEM OF DYNAMIC WALKING OF ANTHROPOMORPHIC BIPED ROBOTS

This paper deals with the mechanics problem of dynamic walking of anthropomorphic biped robots. Through analysing the mechanics system of this kind of robots in detail, the motion constraint equations are established, three mechanics laws describing the r

Input torque sensitivity to uncertain parameters in biped robot

Input torque is the main power to maintain bipedal walking of robot, and can be calculated from trajectory planning and dynamic modeling on biped robot. During bipedal walking, the input torque is usually required to be adjusted due to some uncertain parameters arising from objective or subjective factors in the dynamical model to maintain the pre-planned stable trajectory. Here, a planar 5-link biped robot is used as an illustrating example to investigate the effects of uncertain parameters on the input torques. Kine-matic equations of the biped robot are firstly established by the third-order spline curves based on the trajectory planning method, and the dynamic modeling is accomplished by taking both the certain and uncertain parameters into account. Next, several evaluation indices on input torques are intro-duced to perform sensitivity analysis of the input torque with respect to the uncertain parameters. Finally, based on the Monte Carlo simulation, the values of evaluation indices on input torques are presented, from which all the robot param-eters are classified into three categories, i.e., strongly sensi-tive, sensitive and almost insensitive parameters.

Influence of Slope Angle on the Walking of Passive Dynamic Biped Robot

In this paper, we modeled a simple planer passive dynamic biped robot without knee with point feet. This model has a stable, efficient and natural periodic gait which depends on the values of parameters like slope angle of inclined ramp, mass ratio and length ratio. The described model actually is an impulse differential equation. Its corresponding poincare map is discrete case. With the analysis of the bifurcation properties of poincare map, we can effectively understand some feature of impulse model. The ideas and methods to cope with this impulse model are common. But, the process of analysis is rigorous. Numerical simulations are reliable.

ZMP-based Gait Optimization of the Biped Robot

The gait of the biped robot is described using six parameters such as stature,velocity,length of the step,etc.The algorithm of the Newton-Euler is actualized by object-oriented idea,and then the zero moment point (ZMP) of the dynamically walking biped is calculated.Finally,the gait of biped is optimized using gene algorithm,and the optimized result prove the correctness of the algorithm.

Walking Stability Control Method for Biped Robot on Uneven Ground Based on Deep Q-Network

A gait control method for a biped robot based on the deep Q-network (DQN) algorithm is proposed to enhance the stability of walking on uneven ground. This control strategy is an intelligent learning method of posture adjustment. A robot is taken as an agent and trained to walk steadily on an uneven surface with obstacles, using a simple reward function based on forward progress. The reward-punishment (RP) mechanism of the DQN algorithm is established after obtaining the offline gait which was generated in advance foot trajectory planning. Instead of implementing a complex dynamic model, the proposed method enables the biped robot to learn to adjust its posture on the uneven ground and ensures walking stability. The performance and effectiveness of the proposed algorithm was validated in the V-REP simulation environment. The results demonstrate that the biped robot's lateral tile angle is less than 3° after implementing the proposed method and the walking stability is obviously improved.

Robust State Feedback H_∞Control for Dynamic Biped Robot Based on T-S Fuzzy Model

T-S fuzzy model was applied to describe nonlinear system and global fuzzy model was expressed by the form of uncertain system.Based on robust state feedback H_∞control strategy,designed a global asymptotic steady fuzzy model.This control system can use the experimental input-output data pairs for the biped robot learning and walking with dynamic balance.It is proved by simulation result that robust state feedback H_∞control method based on T-S fuzzy model can effectively restrain the effect of model uncertainties and external disturbance acting on biped robot.From these works,we showed the satisfactory performance of joint tracking without any chattering.

Study and Development of the Biped Ice Skating Robot with Passive Wheels

A new passive wheel type of biped ice-skating robot(BISR)which was able to imitate human skating motion was developed. Firstly, the characteristics of two types of human skating gait were introduced; secondly, after simplifying the kinematical model, the BISR's motion principle was presented; then the construction and control system of BISR were proposed; at last, the skating experiment of the BISR in a symmetric gait mode was conducted and some conclusions were drawn.

A NEW METHOD OF MECHANISM SYNTHESIS FOR BIPED ROBOTS

ＡＮＥＷＭＥＴＨＯＤＯＦＭＥＣＨＡＮＩＳＭＳＹＮＴＨＥＳＩＳＦＯＲＢＩＰＥＤＲＯＢＯＴＳ①ＴａｎＧｕａｎｚｈｅｎｇＩｎｓｔｉｔｕｔｅｏｆＲｏｂｏｔｉｃｓ，ＣｅｎｔｒａｌＳｏｕｔｈＵｎｉｖｅｒｓｉｔｙｏｆＴｅｃｈｎｏｌｏｇｙ，Ｃｈａｎｇｓｈａ４１００８...

A novel compound biped locomotion algorithm for humanoid robots to realize biped walking

In this paper, a compound biped locomotion algorithm for a humanoid robot under development is presented. This paper is organized in two main parts. In the first part, it mainly focuses on the structural design for the humanoid. In the second part, the compound biped locomotion algorithm is presented based on the reference motion and reference Zero Moment Point （ZMP）. This novel algorithm includes calculation of the upper body motion and trajectory of the Center of Gravity （COG） of the robot. First, disturbances from the environment are eliminated by the compensational movement of the upper body; then based on the error between a reference ZMP and the real ZMP as well as the relation between ZMP and CoG, the CoG error is calculated, thus leading to the CoG trajectory. Then, the motion of the robot converges to its reference motion, generating stable biped walking. Because the calculation of upper body motion and trajectory of CoG both depend on the reference motion, they can work in parallel, thus providing double insurances against the robot＇s collapse. Finally, the algorithm is validated by different kinds of simulation experiments.

A New Method of Mechanism Synthesis for Biped Robots

For a biped robot,the most essential components are its joints,including hip,knee,and ankle joints.In this paper,the construction method of a new type of joint is proposed.The main feature of this joint is that a combined transmission consisting of harmonic geardrive and planet gear drive is adopted in its system,and it possesses smaller volume,greaterdriving torque,and fine-looking appearance.Using this type of joint,a practical biped robotnamed NAIWR-I was constructed.The computer simulations and experiments with NAI-WR-I robot show that the method of mechanism synthesis proposed in this paper is reason-able and feasible.

Balance recovery control for biped robot based on reaction null space method

A biped walking robot should be able to keep balance even in the presence of disturbing forces. This paper presents a step strategy concept of biped walking robot that is stabilized by using reaction null space method. The called ＂step strategy＂ can be modeled by means of the reaction null space method that introduced earlier to tackle dynamic interaction problems of free-floating robots, or moving base robots in general. 6-DOF biped robot model simulations are used to confirm the validity.

LIE SYMMETRIES AND CONSERVED QUANTITY OF A BIPED ROBOT

For a better understanding of the dynamic principles governing biped locomotion, the Lie symmetries and conservation laws of a biped robot are studied. In Lie theory, Lie sym- metries and conservation laws can be derived from the form invariance of di?erential equations undergoing in?nitesimal transformation. By introducing in?nitesimal transformations including time and spatial coordinates, the determining equations of a biped robot are established. Then the necessary and su?cient conditions for a biped robot to have conserved quantities are obtained. For the lateral-plane dynamical model of a biped robot, a Lie conserved quantity is found.

A New Method of Desired Gait Synthesis for Biped Walking Robot Based on Ground Reaction Force

A new method of desired gait synthesis for biped walking robot based on the ground reaction force was proposed. The relation between the ground reaction force and joint motion is derived using the D’Almbert principle. In view of dynamic walking with high stability, the ZMP(Zero Moment Point)stability criterion must be considered in the desired gait synthesis. After that, the joint trajectories of biped walking robot are decided by substituting the ground reaction force into the aforesaid relation based on the ZMP criterion. The trajectory of desired ZMP is determined by a fuzzy logic based upon the body posture of biped walking robot. The proposed scheme is simulated and experimented on a 10 degree of freedom biped walking robot. The results indicate that the proposed method is feasible.