Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Adaptive Optimal Discrete-Time Output-Feedback Using an Internal Model Principle and Adaptive Dynamic Programming

In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed method, termed as IMP-ADP, does not require complete state feedback-merely the measurement of input and output data. More specifically, based on the IMP, the output control problem can first be converted into a stabilization problem. We then design an observer to reproduce the full state of the system by measuring the inputs and outputs. Moreover, this technique includes both a policy iteration algorithm and a value iteration algorithm to determine the optimal feedback gain without using a dynamic system model. It is important that with this concept one does not need to solve the regulator equation. Finally, this control method was tested on an inverter system of grid-connected LCLs to demonstrate that the proposed method provides the desired performance in terms of both tracking and disturbance rejection.

Shrek:a dynamic object-oriented programming language

From a perspective of theoretical study, there are some faults in the models of the existing object-oriented programming languages. For example, C# does not support metaclasses, the primitive types of Java and C# are not objects, etc. So, this paper designs a programming language, Shrek, which integrates many language features and constructions in a compact and consistent model. The Shrek language is a class-based purely object-oriented language. It has a dynamical strong type system, and adopts a single-inheritance mechanism with Mixin as its complement. It has a consistent class instantiation and inheritance structure, and the ability of intercessive structural computational reflection, which enables it to support safe metaclass programming. It also supports multi-thread programming and automatic garbage collection, and enforces its expressive power by adopting a native method mechanism. The prototype system of the Shrek language is implemented and anticipated design goals are achieved.

Residential Energy Scheduling for Variable Weather Solar Energy Based on Adaptive Dynamic Programming

The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.

Inertia Matching During Dynamic Simulation of Vehicle Drive System on Test Bench

Framework and basic parameters of a test bench for motor drive system of electric vehicle (EV) are illuminated. Two kinds of electric drive models, one was for the electric vehicle drived on real road, the other was for that on test bench, are put forward. Then, dynamic analysis of these models is made in detail. Inertia matching method of the test bench is researched and some useful formulas and graphs are brought forward. The experiment of an electric bus is introduced in order to explain the usage of this inertia matching method.

Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets

This paper researches the adaptive scheduling problem of multiple electronic support measures(multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain environment. For adaptive selection of appropriate ESMs, we generalize an approximate dynamic programming(ADP) framework to the dynamic case. We define the environment model and agent model, respectively. To handle the partially observable challenge, we apply the unsented Kalman filter(UKF) algorithm for belief state estimation. To reduce the computational burden, a simulation-based approach rollout with a redesigned base policy is proposed to approximate the long-term cumulative reward. Meanwhile, Monte Carlo sampling is combined into the rollout to estimate the expectation of the rewards. The experiments indicate that our method outperforms other strategies due to its better performance in larger-scale problems.

UAV flight strategy algorithm based on dynamic programming

Unmanned aerial vehicles(UAVs) may play an important role in data collection and offloading in vast areas deploying wireless sensor networks, and the UAV’s action strategy has a vital influence on achieving applicability and computational complexity. Dynamic programming(DP) has a good application in the path planning of UAV, but there are problems in the applicability of special terrain environment and the complexity of the algorithm.Based on the analysis of DP, this paper proposes a hierarchical directional DP(DDP) algorithm based on direction determination and hierarchical model. We compare our methods with Q-learning and DP algorithm by experiments, and the results show that our method can improve the terrain applicability, meanwhile greatly reduce the computational complexity.

Dynamic Matching-Based Spectrum Detection in Cognitive Radio Networks

Cognitive Radio Network provides an opportunity to reduce the spectrum resource crisis by allowing secondary users to access the idle spectrum allocated to primary users. The precondition of spectrum sharing is to obtain the Spectrum Availability Information(SAI). Energy detection is a typical technology to get SAI. With the mobility of primary users, the energy received by secondary users varies greatly with the distance from the target primary users. Most of the existing energy detection algorithms that use fixed thresholds are not suitable. We propose a Dynamic Matching-Based Spectrum Detection(DMBSD) scheme which can detect sensing data, reduce the impact of malicious data and make final sensing results more accurate with dynamic threshold setting and data matching. Simulation results show that the proposed scheme can detect tempered data, and increase detection probability by decreasing false alarm probability and missed detection probability.

A COMBINED PARAMETRIC QUADRATIC PROGRAMMING AND PRECISE INTEGRATION METHOD BASED DYNAMIC ANALYSIS OF ELASTIC-PLASTIC HARDENING/SOFTENING PROBLEMS

The objective of the paper is to develop a new algorithm for numerical solution of dynamic elastic-plastic strain hardening/softening problems. The gradient dependent model is adopted in the numerical model to overcome the result mesh-sensitivity problem in the dynamic strain softening or strain localization analysis. The equations for the dynamic elastic-plastic problems are derived in terms of the parametric variational principle, which is valid for associated, non-associated and strain softening plastic constitutive models in the finite element analysis. The precise integration method, which has been widely used for discretization in time domain of the linear problems, is introduced for the solution of dynamic nonlinear equations. The new algorithm proposed is based on the combination of the parametric quadratic programming method and the precise integration method and has all the advantages in both of the algorithms. Results of numerical examples demonstrate not only the validity, but also the advantages of the algorithm proposed for the numerical solution of nonlinear dynamic problems.

Optimal Control for a Class of Complex Singular System Based on Adaptive Dynamic Programming

This paper presents a new design approach to achieve decentralized optimal control of high-dimension complex singular systems with dynamic uncertainties. Based on robust adaptive dynamic programming(robust ADP) method, controllers for solving the singular systems optimal control problem are designed. The proposed algorithm can work well when the system model is not exactly known but the input and output data can be measured. The policy iteration of each controller only uses their own states and input information for learning,and do not need to know the whole system dynamics. Simulation results on the New England 10-machine 39-bus test system show the effectiveness of the designed controller.

Genetic programming for predictions of effectiveness of rolling dynamic compaction with dynamic cone penetrometer test results

Rolling dynamic compaction (RDC),which employs non-circular module towed behind a tractor,is an innovative soil compaction method that has proven to be successful in many ground improvement applications.RDC involves repeatedly delivering high-energy impact blows onto the ground surface,which improves soil density and thus soil strength and stiffness.However,there exists a lack of methods to predict the effectiveness of RDC in different ground conditions,which has become a major obstacle to its adoption.For this,in this context,a prediction model is developed based on linear genetic programming (LGP),which is one of the common approaches in application of artificial intelligence for nonlinear forecasting.The model is based on in situ density-related data in terms of dynamic cone penetrometer (DCP) results obtained from several projects that have employed the 4-sided,8-t impact roller (BH-1300).It is shown that the model is accurate and reliable over a range of soil types.Furthermore,a series of parametric studies confirms its robustness in generalizing data.In addition,the results of the comparative study indicate that the optimal LGP model has a better predictive performance than the existing artificial neural network (ANN) model developed earlier by the authors.

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.

Approximate Dynamic Programming for Stochastic Resource Allocation Problems

A stochastic resource allocation model, based on the principles of Markov decision processes(MDPs), is proposed in this paper. In particular, a general-purpose framework is developed, which takes into account resource requests for both instant and future needs. The considered framework can handle two types of reservations(i.e., specified and unspecified time interval reservation requests), and implement an overbooking business strategy to further increase business revenues. The resulting dynamic pricing problems can be regarded as sequential decision-making problems under uncertainty, which is solved by means of stochastic dynamic programming(DP) based algorithms. In this regard, Bellman’s backward principle of optimality is exploited in order to provide all the implementation mechanisms for the proposed reservation pricing algorithm. The curse of dimensionality, as the inevitable issue of the DP both for instant resource requests and future resource reservations,occurs. In particular, an approximate dynamic programming(ADP) technique based on linear function approximations is applied to solve such scalability issues. Several examples are provided to show the effectiveness of the proposed approach.

Dynamic programming methodology for multi-criteria group decision-making under ordinal preferences

A method of minimizing rankings inconsistency is proposed for a decision-making problem with rankings of alternatives given by multiple decision makers according to multiple criteria. For each criteria, at first, the total inconsistency between the rankings of all alternatives for the group and the ones for every decision maker is defined after the decision maker weights in respect to the criteria are considered. Similarly, the total inconsistency between their final rankings for the group and the ones under every criteria is determined after the criteria weights are taken into account. Then two nonlinear integer programming models minimizing respectively the two total inconsistencies above are developed and then transformed to two dynamic programming models to obtain separately the rankings of all alternatives for the group with respect to each criteria and their final rankings. A supplier selection case illustrated the proposed method, and some discussions on the results verified its effectiveness. This work develops a new measurement of ordinal preferences’ inconsistency in multi-criteria group decision-making （MCGDM） and extends the cook-seiford social selection function to MCGDM considering weights of criteria and decision makers and can obtain unique ranking result.

Approach of service recovery decision-making based on Bellman dynamic programming

Based on service-oriented architecture（SOA）,a Bellman-dynamic-programming-based approach of service recovery decision-making is proposed to make valid recovery decisions.Both the attribute and the process of services in the controllable distributed information system are analyzed as the preparatory work.Using the idea of service composition as a reference,the approach translates the recovery decision-making into a planning problem regarding artificial intelligence （AI） through two steps.The first is the self-organization based on a logical view of the network,and the second is the definition of evaluation standards.Applying Bellman dynamic programming to solve the planning problem,the approach offers timely emergency response and optimal recovery source selection,meeting multiple QoS （quality of service）requirements.Experimental results demonstrate the rationality and optimality of the approach,and the theoretical analysis of its computational complexity and the comparison with conventional methods exhibit its high efficiency.

Policy iteration optimal tracking control for chaotic systems by using an adaptive dynamic programming approach

A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.

CONTROLLING ROBOT MANIPULATORS BY DYNAMIC PROGRAMMING

A certain number of considerations should be taken into account in the dynamic control of robot manipulators as highly complex non-linear systems.In this article,we provide a detailed presentation of the mechanical and electrical impli- cations of robots equipped with DC motor actuators.This model takes into account all non-linear aspects of the system.Then,we develop computational algorithms for optimal control based on dynamic programming.The robot's trajectory must be predefined,but performance criteria and constraints applying to the system are not limited and we may adapt them freely to the robot and the task being studied.As an example,a manipulator arm with 3 degrees of freedom is analyzed.

Novel algorithm for distributed replicas management based on dynamic programming

Replicas can improve the data reliability in distributed system. However, the traditional algorithms for replica management are based on the assumption that all replicas have the uniform reliability, which is inaccurate in some actual systems. To address such problem, a novel algorithm is proposed based on dynamic programming to manage the number and distribution of replicas in different nodes. By using Markov model, replicas management is organized as a multi-phase process, and the recursion equations are provided. In this algorithm, the heterogeneity of nodes, the expense for maintaining replicas and the engaged space have been considered. Under these restricted conditions, this algorithm realizes high data reliability in a distributed system. The results of case analysis prove the feasibility of the algorithm.

Multi-view video color correction using dynamic programming

Color inconsistency between views is an important problem to be solved in multi-view video systems. A multi-view video color correction method using dynamic programming is proposed. Three-dimensional histograms are constructed with sequential conditional probability in HSI color space. Then, dynamic programming is used to seek the best color mapping relation with the minimum cost path between target image histogram and source image histogram. Finally, video tracking technique is performed to correct multi-view video. Experimental results show that the proposed method can obtain better subjective and objective performance in color correction.