Switched adaptive active disturbance rejection control of variable structure near space vehicles based on adaptive dynamic programming

Choyng DONG ,Chen LIU ,b,*,Qing WANG ,Ligng GONG

a School of Aeronautic Science and Engineering,Beihang University,Beijing 100083,China

b Beijing Institute of Electric System Engineering,Beijing 100854,China

c School of Automation Science and Electrical Engineering,Beihang University,Beijing 100083,China

KEYWORDS Adaptive control;ADP;ADRC;Attitude control;Switched nonlinear system;VSNSV

Abstract Based on the switched nonlinear system,a switched adaptive Active Disturbance Rejection Control(ADRC)law is proposed for the Variable Structure Near Space Vehicle(VSNSV)with unknown uncertainties and external disturbances.The reduced-order Extended State Observers(ESOs)are constructed for the attitude angle system and the angular rate system to estimate the total disturbance in real time.With the extended state introduced to counteract the effects of uncertainties and disturbances,a systematic procedure is presented for the synthesis of the switched adaptive ADRC strategy.Rigorous proof shows that the estimation errors of the reduced-order ESOs would converge to a small neighborhood of zero in finite time,and that the output of the closedloop system can track a given signal stably for a class of switching signals with average dwell time via the proposed approach.The variable gain control strategy based on Adaptive Dynamic Programming(ADP)with the actor-critic structure is also designed to improve the dynamic performance of the system.Simulation results verify the effectiveness and advantage of the proposed control scheme.

1.Introduction

Near Space Vehicles(NSVs)are a kind of novel aerospace vehicles.Compared with traditional aircrafts,the NSV has higher mobility,wider flight envelope,lower launching costs,shorter preparation period,stronger penetration ability,and richer task modes;therefore,NSVs have been widely applied in both civilian and military fields.1-4However,in the near space5,6(20-100 km above the earth), there exist strong wind disturbance,low air pressure,complicated variations of temperature and flight environments so the model of NSVs suffers from highly nonlinearity,fast time-variation,strong coupling and serious uncertainties.Thus,the control of NSVs has become extremely difficult.In Ref.7,an adaptive neural tracking controller was proposed for NSVs with unknown parametric uncertainty,external disturbances,input saturation and dead zones.Ref.8constructed an adaptive neural network observer for the reentry model of NSVs,and designed a backstepping controller.Due to their large flight envelope and multiple mission modes,NSVs have different flight characteristics in different flight stages.Drawing on the advantages of morphing aircraft,Variable Structure Near Space Vehicles(VSNSVs)have been applied to improve flight performance.In spite of benefits, configuration transformation causes difficulty in approximating the transition dynamics more precisely.To address this problem,a switched system is usually established to describe the flight characteristics of variable structure vehicles.The changes of vehicle configuration however increase the challenge of controller design for VSNSVs.Ref.9proposed a switching controller to stabilize the vehicle based on the switched nonlinear system.In Ref.10,smooth switched linear parameter-varying controllers were designed for hypersonic vehicles.These studies focus on the steady-state performance,designing fixed gain controllers;however,a fixed gain controller cannot ensure high and smooth dynamic performance when switching happens because at the switching instant,the aerodynamic parameters of the vehicle change in a large scale.

Another difficulty in the controller design for VSNSVs lies in external disturbances and parameter uncertainty caused by complex environments of the vehicle.A powerful tool to suppress the influence of all the uncertainties is H∞control,which has been widely applied in practical systems.11In Ref.12,a design of non-fragile linear parameter-varying H∞controller was proposed to investigate the stabilization problem of morphing aircraft with asynchronous switching.Introducing a prediction mechanism,Ref.13developed a novel sampled-data control strategy to solve the H∞control problem of the attitude stabilization of a rigid spacecraft with external disturbances.Ref.14studied the problem of the mixed H∞and of passive control for flexible spacecrafts subject to nonuniform sampling and input delays.Since the changes of sweep angles can induce the changes of aerodynamic coefficients and the center of gravity,parameter uncertainty is also involved.In the complex near space environment,the uncertainties are so large that they cannot be regarded as small disturbances.In this case,the H∞control method may have certain conservation since it can only attenuate the effect of disturbances instead of eliminating them.An effective control strategy for the effect of uncertainties and disturbances is active disturbance rejection control(ADRC),which has attracted considerable attention in both academia15-17and industries.18-20Treating the total disturbance as an extended state and estimating this state in real time can compensate the total disturbance in the controller.Ref.21proposed a control strategy of nonlinear fractional order proportion-integral-derivative active disturbance rejection based on particle swarm algorithm for attitude control of hypersonic vehicles.By constructing a reduced-order ESO,Ref.22proposed an integrated guidance and control law for missiles to intercept unknown targets with uncertainties and control constraints.Ref.23applied ADRC to hydraulic servo fields with uncertain mismatched input problems,and introduced the singular value perturbation theory.Ref.24developed a flight control approach by modifying ADRC for quadrotors with input delays and external disturbances.Nevertheless,ADRC has seldom been used in the switching control field.It still remains challenging to construct switched ESOs,design the corresponding switched control law,and analyze system stability.

Moreover, a data-driven learning approach, Adaptive Dynamic Programming(ADP),has advantages in tuning controller parameters so that the system can adapt to the changes of external environments.25,26In Ref.27,a supplementary control approach based on ADP was proposed for the tracking control of air-breathing hypersonic vehicles. Using ADP,Ref.28designed a novel control strategy for a class of continuous-time nonaffine nonlinear systems with unknown dynamics. Ref.29devised an incremental Approximate Dynamic Programming(iADP)method,and a model-free control scheme for a class of nonlinear systems based on iADP.In Ref.30,an online adaptive control law was provided to solve the infinite-horizon optimal control problem of uncertain nonlinear systems with input saturation constraints.

Based on the studies mentioned above,a switched adaptive ADRC strategy is proposed in this paper for VSNSVs with parameter uncertainty and external disturbances.This paper estimates extended states in real time by constructing switched reduced-order ESOs for the attitude angle system and angular rate system.Then,a switched controller is designed for the vehicle to track a given signal stably for a class of switching signals with average dwell time.This paper also proposes a gain scheme for the variable controller based on ADP with the actor-critic structure.The stability criteria of the method proposed are less conservative than those of the design method for the switched adaptive controller based on the common Lyapunov approach in Ref.9,and the control strategy proposed presents a better dynamic performance when switching occurs or when the system suffers from external disturbances.The proposed controller features a simple structure,which is valuable for practical engineering.

The rest of the paper is organized as follows.Section 2 describes the attitude tracking problem of VSNSVs,and Section 3 proposes a systematic procedure for the synthesis of controller design and analyses system stabilization under the switched adaptive ADRC law proposed.In Section 4,a gain strategy is developed for the variable controller based on ADP with the actor-critic structure.Section 5 provides a simulation example to verify the effectiveness and advantage of the proposed method.

Notation:Ris the set of real numbers.R+denotes the interval[0,∞)of R.||·||refers to the Euclidean vector norm or the induced matrix 2-norm.A function α:R+→R+is of class K if α is continuous, strictly increasing, and α(0)=0. If unbounded,α is also of class K∞.λmax(·)and λmin(·)denote the largest and smallest eigenvalue of a matrix,respectively.In×nis an identity matrix of n dimensions.

2.Mathematical model of VSNSV

As shown in Fig.1,the sweep angle of a VSNSV varies according to flight conditions.For example,the sweep angle is 60°during a supersonic flight,and increases to 75°during a hypersonic flight.The parameters of NSVs,including the wing area and dynamic coefficients,change as the sweep angle varies;thus,it is difficult to describe the dynamic coefficients of different structures with a precise nonlinear function.To represent the flight characteristics in different flight conditions, a multi-model system has been introduced.

The kinematical and dynamical equations of the VSNSV are the switched nonlinear system9,as shown below:

where Ω=[α β μ]T,standing for the attitude angle vector with an attack angle α,sideslip angle β and bank angle μ.ω=[p q r]T,which is the angular rate vector with the roll rate p, pitch rate q and yaw rate r.σ(t):[0,+∞)→S={1,2,···,s},which is the switching signal dependent on the sweep angle Λ,and in which S is the set of switching signals consisting of all the right-continuous piecewise constant functions.dsandare external disturbances.andare introduced to represent parameter uncertainty,which will be elaborate in the assumption part.The symbols in Eq.(1)are given as

where M and V are the mass and the velocity of the VSNSV,respectively.Geis the gravitational constant of Earth,rethe distance from the VSNSV to the center of Earth,the dynamic pressure,γ the flight-path angle,Sσ(t)the wing area,andare aerodynamic coefficients,andandare the roll,pitch and yaw moments of inertia,respectively.T is the engine thrust.δ=[δeδαδr]T,which is the control surface vector,andis the control moment vector induced by the control surface.The thrust is considered as an ordinary variable in attitude control problems.9

Each subsystem of the above switched system describes the unique flight dynamics of NSVs when the sweep angle varies.For example,model j represents the flight dynamic at the sweep angle of 60°-65°and model k at the sweep angle of 65°-70°.As the sweep angle increases from 60°to 70°,the flight dynamic switches from model j to model k.

In this paper,a switched controller is designed to ensure that the attitude system is stable and that the output Ω can track a reference output Ωrefwhen the structure of the VSNSV changes.It is assumed that aerodynamic coefficients and the moments of inertia are unknown or contain uncertainties.Then,the following assumptions are made for the VSNSV.

Assumption 1.The reference signal Ωrefis continuously differentiable.Ωrefand its derivative ˙Ωrefare bounded.

Assumption 2.The total disturbance in Eq.(1)and its derivatives are bounded,satisfying

Assumption 3.The control gain matrix gsis invertible and there exist two known positive definite matrixes gf0and ψfsuch that

Remark 1.The reference signal is generated by the guidance loop to ensure flight safety,so Assumption 1 is necessary and can be satisfied in practice.22Assumption 2 means that the class of uncertainties and disturbances considered in this paper is continuous and bounded,though the upper bounds can be unknown.Further,moments of inertia are always positive and bounded when the structure of the VSNSV changes;thus,is a positive definite matrix and also bounded.9Therefore,Assumption 3 is reasonable.

In addition,the following definition is provided,which is useful in the sequel.

Definition 1.31.For a switching signal σ(t) and each t2≥t1≥0,let Nσ(t2,t1)be the number of discontinuities of σ(t)in the open interval(t1,t2).If there exist two positive numbers N0(the chatter bound)and τa,σ(t)has average dwell time τasuch that

3.Controller design

3.1.Controller design for attitude angle system

According to the multiple-time-scale features,the dynamics of the VSNSV can be divided into the attitude angle system and angular rate system,and the controller for each system can be designed separately.32A virtual control for the attitude angle system is designed with the reduced-order ESOs constructed to estimate the unknown disturbances and uncertainties.

First,the extended state is introduced,and each subsystem of the attitude angle system is described as

According to the ADRC strategy, the corresponding switched reduced-order ESOs are designed as

where C=diag{c1,c2,c3}.c1,c2,and c3are the observer bandwidths,satisfying c1,c2,c3＞0.22

Therefore,the dynamics of the observer estimation error is

Since C ＞0,there would exist a positive matrix P1such that CP1+P1C=Q1for any positive definite matrix Q1.A Lyapunov function candidate is defined as

Then,the derivative of Eq.(8)along the trajectory of Eq.(7)can be obtained as

where i=1,2,3.Therefore,there would exist a constant τΩsuch that

where ZΩ=Ω-Ωref,representing the tracking error of the reference signal.κ1=diag{κ11,κ12,κ13}＞0,which is a variable gain matrix and would be given later.

Remark 2.In this paper,the reduced-order ESOs are used to eliminate the switching function.Different from the conventional reduced-order ESOs,is contained in Eq. (6).Usually, Young's inequality is used to design a positive smooth function to deal with the discontinuous switching function,9,33increasing the complexity of the virtual controller.In this paper,all the switching functions are approximated in the reduced-order ESOs instead of appearing explicitly in the virtual control.Therefore,the time derivative of the virtual controller would always exist.Due to its simpler structure,this virtual controller is easy to implement in practical engineering.

A Lyapunov function candidate for the kth attitude angle system is set as

where Pk2is a definite positive matrix,and.Combined with Eq.(1),Eq.(11)and Young's inequality,the time derivative of Eq.(12)can be computed as

3.2.Controller design for angular rate system

According to the ADRC strategy, the corresponding switched reduced-order ESOs are designed as

where D=diag{d1,d2,d3},in which d1,d2,d3are the observer bandwidths and d1,d2,d3＞0.

Based on Eqs.(15)and(16),the dynamics of the ESO estimation error of the observer are obtained as follows:

Since D ＞0,there exists a positive matrix P3such that DP3+P3D=Q3for any positive definite matrix Q3.A Lyapunov function candidate is defined as

The derivative of Eq.(18)along the trajectory of Eq.(17)is

where i=1,2,3.Therefore,there would exist a constant τωsuch that

Then,with the reduced-order ESOs Eq.(15),the controller can be designed as

where Zω=ω-ωref,representing the tracking error of the virtual control signal.κ2=diag{κ21,κ22,κ23}＞0,which is a gain matrix of the variable controller to be designed in the next section.

A Lyapunov function candidate for the angular rate system is set as

where Pk4is a positive definite matrix and satisfiesCombined with Eq.(1),Eq.(21)and Young's inequality,the time derivative of Eq.(22)becomes

Remark 3.The observer convergence time is relevant to observer gains.As long as observer gains are set sufficiently large,the convergence time can always be shorter than the dwell time of each subsystem,which means that observer estimation can always converge before the next switch.Therefore,the Zeno phenomenon can be avoided effectively.In addition,since ? is a designed parameter,andcan always be positive.

3.3.Stability analysis

The main result of this paper is stated in the following theorem.

Theorem 1.For the closed-loop VSNSV system (1) with Assumptions 1-3,the control law(21)with the reduced-order ESOs(16)and the virtual control signals(11)with the reducedorder ESOs(6)can guarantee that all the signals in the closedloop system are semi-globally uniformly ultimately bounded under a class of switching signals with average dwell time,given any bounded initial conditions.

Proof.For any k ∈S,define a Lyapunov function candidate for the kth subsystem of Eq.(1)as

It is obvious that there existssuch that

Based on Eqs.(14)and(24),the derivative of Eq.(25)yields that

For any T ＞0,let t0=0 and t1,t2,···,tNσ(T,0)denote the switching time on the interval [0,T].We set

which is piecewise differentiable along solutions of Eq.(1).For any t ∈[tj,tj+1),suppose that subsystem k is active,and one has

For any T ≥t0=0, iterating Eq. (31) from j=0 to j=Nσ(T,0)-1,one has

If τa＞(lnζ/κ),then for any δ ∈(0,κ-(lnζ/τ)),there is τa＞(lnζ/κ-δ). It is obvious that Nσ(T,t)-j ≤1+Nσ(T,tj+1), j=0,1,···,Nσ(T,0), which givesIn addition,since δ ＜κ,there is

Combining Eqs.(32)and(33),one obtains

which implies

From Eq.(34),it can be seen that the tracking errors are bounded for any bounded initial condition if τa＞lnζ/κ.Therefore,the control law Eq.(20)guarantees that the tracking error ZΩis semi-globally uniformly ultimately bounded for a class of switching signals with average dwell timewhich completes the proof.

Remark 4.Ref.9proves that the closed-loop VSNSV system is uniformly ultimately bounded under arbitrary switching laws by utilizing the backstepping controller design procedure and the common Lyapunov function approach.The conservation of the control strategy in Ref.9mainly lies in three aspects.First, limited by the actuator capability, the sweep angle changes with time limitation.Namely,the characteristics of the flight dynamic of VSNSVs should be studied under limited switching laws.Second,a common Lypanov function has to be constructed for all subsystems of a switched system;however,this function cannot always be found.Third,common virtual control functions need to be found before the common Lyapunov function is constructed,which increases the complexity of the application of the common Lypanov function.34

All these shortcomings can be overcome in the present study.First,for the control scheme proposed in this paper,the switching law with average dwell time limitation is accordant with the practical circumstance of the flight control system of VSNSVs.Second,the controller design procedure is simplified so that a common virtual control law is not necessary anymore.Third,the controller and Lyapunov function for each subsystem are designed separately;thus,the sufficient condition for stabilizing the closed loop system can be found more easily.

Remark 5.In this paper,the average dwell time method has been applied to meet the need of multiple switching of VSNSV flight dynamics in a short time.Although the minimum dwell time method is frequently used in the stability analysis of switched systems,it is difficult for multiple Lyapunov function method to estimate the minimum dwell time for a switched system.35Meanwhile,the maximum dwell time method is conservative and time-consuming when activating all the subsystems.The average dwell time method however can solve the fast switching problem in a short time,and is the most suitable candidate for the switched nonlinear system of VSNSVs to establish the stability criteria.

4.Variable controller gain based on ADP

To improve the dynamic performance of the system,this study designs the for the variable gain control strategy based on ADP with the actor-critic structure. The structure of the designed control system is shown in Fig.2.The variable gain strategy based on ADP consists of three parts:the critic network,actor network and Stochastic Action Modifier(SAM).The roles of the three parts will be described later.

To implement the variable gain strategy,the utility function in ADP at time t is defined as

where K1and K2are positive definite matrixes.Give a discount factor β,0 ＜β ＜1,and the minimization of the cost function can be formulated as

where Δκ1,Δκ2are adaption values of controller gains.The goal of the variable gain policy is to get Δκ1(t),Δκ2(t),Δκ1(t+Δt),Δκ2(t+Δt),···,ensuring the minimization of the cost function and the optimal dynamic performance of the VSNSV during the flight.

The critic network is then introduced to get an appropriate value ofThe structure of the critic network is shown in Fig.3.

The input and output vectors of the critic network are expressed as

There is only one hidden layer,which contains 10 neurons,in the network;thus,the function of the critic network can be expressed as

Fig.2 Structure of variable gain control strategy based on ADP.

Fig.3 Structure of critic network.

Define the approximation error of the cost function as

and the updating law of the critic network can be given as

Next,the actor network is applied to obtain an appropriate value of the deviation of controller gains.The structure of the actor network is shown in Fig.4.

The input and output vectors of the actor network are

Fig.4 Structure of actor network.

The actor network contains only one hidden layer consisting of eight neurons;thus,the function of the actor network is clear,given as

where H(t)is a matrix of the dimension 8×2,in which each component is given asj=1,2,···,6, k=1,2,···,6,is the learning rate of the actor network.

Based on the critic and actor networks designed above,the steps of the proposed control strategy are given as follows:

Algorithm 1.Implementation steps of the designed control strategy

The stopping condition is set when the tracking performance is achieved with the current controller or when the maximum iteration step is reached.Since κ1,κ2are positive definite and ? is a designed parameter,Theorem 1 is still tenable with the variable gain strategy proposed.

Until here,the variable gain scheme based on ADP is completed.

Remark 6.To provide better generalization ability,SAM is used to obtain a stochastic deviation of controller gains.The output of the actor network does not change controller gains directly.The actor network is applied to map the tracking errors to Δκ=[Δκ1,Δκ2]T.Then,stochastic modification of controller gains is carried out by SAM according to the estimate of cost value by the critic network.

5.Simulation

Numerical simulation is conducted to verify the design of the control scheme. The aerodynamic coefficients of VSNSVs can be found in Ref.36which applied the set of coefficients at hypersonic speed.The attitude tracking control scheme is considered when the wing sweep angle Λ of the VSNSV changes from 60°to 75°.Assume that the VSNSV flies with the velocity of 1.8 km/s and at the flight height of 30 km,and the initial conditions are Λ=60°, α=1°, β=1°, μ=1°,p=q=r=0(°/s ),and T=208.4 kN.Model 1 and the model 2 are used to describe the flight characteristics when Λ=60°-67°,and Λ=67°-75°,respectively.

The reference outputs are αref=5°,βref=0°,and μref=0°.Suppose that there exist 30%uncertainties of the aerodynamic coefficients,and the disturbances upon model 1 and model 2 of the VSNSV are

According to the design procedure,the initial controller parameters are set as κ1=diag{10,8,8} and κ2=diag{8,7,7}. The observer gains are C=diag{10,10,10}Tand D=diag{20,20,20}T. The initial conditions of the reduced-order ESOs are given as Ω=[1,1,1]T,Ωe=[0,0,0]T,ω=[0,0,0]Tand ωe=[0,0,0]T.

The initial and final learning rates of the critic network are σc(0)=0.2 and=0.005,respectively,and of the actor network are σa(0)=0.2 and=0.005,respectively.The maximal iteration steps of the critic and actor networks are nc=150 and na=100,respectively.The tolerance errors of two networks are both 10-6,and the numbers of the hidden nodes of the critic and actor networks are Nc=Na=9.The discount factor β is set as 0.95.The power matrixes in the utility function are K1=K2=I3×3.The outputs of SAM are bounded by 6 ≤κ11,κ12,κ13≤12,5 ≤κ21,κ22,κ23≤10 with the range of control variables taken into consideration.The simulation time interval is t=0.005 s and the weights of the critic and actor networks are randomly initialized in[-2,2].

According to the range of controller gains,the ADT condition can be calculated as

Then,let the model switching occur at t=3 s and t=10 s:at t=3 s,model 2 is active,and switches to model 1 at t=10 s.It is assumed that the mode switching of the vehicle is a transition process,and the sweep angle changes at a fixed rate.The method proposed in Ref.9is then applied for comparison.The simulation results are shown in Figs.5-11.

Fig.5 Attack angle response.

Fig.6 Sideslip angle response.

Fig.7 Bank angle response.

Fig.8 Control signal.

Fig.9 Extended states and estimates of attitude angle system.

Fig.10 Extended states and estimates of angular rate system.

Fig.11 Weight updating of wc1.

The attitude angle curves of VSNSVs are shown in Figs.5-7.It can be seen that the adaptive controller designed here enables the system to track the given signals stably when switching happens.When the attitude angle is disturbed or switching happens, the controller proposed can guarantee better flight performance than the controller in Ref.9.Fig.8 presents the curves of control inputs which are all in a reasonable range.Figs.9 and 10 show the estimates of the total disturbance in the attitude angle system and angular rate system,respectively.The reduced-order ESOs designed can track system states,and its high tracking performance is observed.Figs.11 and 12 show the weight updating of the critic and actor networks,marked with wc1and wa1,respectively.wc1and wa1represent the weights from the first input to all the hidden nodes of the critic and actor networks, respectively.wc1=[wc11,wc12,···,wc19]T,and wa1=[wa11,wa12,···,wa19]T.It can be seen from Figs.11 and 12 that the weights are adjusted during the control.Fig.13 shows the curve of cost function.Based on the control law and variable gain strategy proposed,the cost function rapidly decreases to zero when the system is disturbed or switching happens,which further verifies the effectiveness of the ADP strategy.Furthermore,the ADT condition gives time limitation to the switching law,so the finding of this study is accordant with practical flight dynamics of VSNSVs. Fast multiple switching in a short time is also allowed as long as average activation time of each subsystem is longer than 1.3863 s. Above all, the simulation results demonstrate the effectiveness and advantage of the control scheme.

Fig.12 Weight updating of wa1.

Fig.13 Cost function.

6.Conclusions

(1)A switched adaptive ADRC strategy is proposed for the attitude dynamics of VSNSVs with unknown uncertainties and external disturbances.All the uncertainties and disturbances are treated as extended states,and the corresponding reduced-order ESOs are constructed for the attitude angle system and the angular rate system to estimate the total disturbance.Switched adaptive ADRC law is designed and extended states are introduced to eliminate the effect of uncertainties and disturbances.It is proved that the output of the closed-loop system can stably track a given signal for a class of switching signals with average dwell time by means of the proposed approach.

(2)A variable gain scheme based on ADP is developed.With the actor-critic structure,two neural networks are designed and an algorithm is given for the updated law of the network weight and for the generation of controller gains.

(3)The variable gain control strategy proposed is applied to VSNSVs,and its high flight performance and effectiveness are demonstrated via numerical simulation.

Acknowledgements

This work was supported by the National Natural Science Foundation of China(Nos.61374012,61403028)and Aeronautical Science Foundation of China(No.2016ZA51011).