An aligned mixture probabilistic principal component analysis for fault detection of multimode chemical processes☆

Yawei Yang,Yuxin Ma,Bing Song,Hongbo Shi*

Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education,East China University of Science and Technology,Shanghai 200237,China

Keywords:Multimode process monitoring Mixture probabilistic principal component analysis Model alignment Fault detection

ABSTRACT A novel approach named aligned mixture probabilistic principal component analysis(AMPPCA)is proposed in this study for fault detection of multimode chemical processes.In order to exploit within-mode correlations,the AMPPCA algorithm first estimates a statistical description for each operating mode by applying mixture probabilistic principal component analysis(MPPCA).As a comparison,the combined MPPCA is employed where monitoring results are softly integrated according to posterior probabilities of the test sample in each local model.For exploiting the cross-mode correlations,which may be useful but are inadvertently neglected due to separately held monitoring approaches,a global monitoring model is constructed by aligning all local models together.In this way,both within-mode and cross-mode correlations are preserved in this integrated space.Finally,the utility and feasibility of AMPPCA are demonstrated through a non-isothermal continuous stirred tank reactor and the TE benchmark process.

1.Introduction

In modern industries,it is no longer a trivial task to improve process operation standards and product quality levels,especially to reduce the variation of the main quality features around their target values.With the increasing availability of data through faster and more informative sensors,multivariate statistical process monitoring(MSPM)has been applied to various processes and gained great success,among which principal component analysis(PCA)is the most everlasting algorithm used[1].Due to various market demands for different products and the changes in feedstock or manufacturing strategies,data characteristics among different operating modes are easily distinguished[2].Hence,evaluated confidence levels of traditional monitoring statistics such as Hotelling's T2and squared prediction error(SPE)will be drastically skewed,and the monitoring performance will be discounted and more seriously,the monitoring approach will completely lose effect.In order to deal with multimode characteristics,some progresses have been made in three different modeling manners.

Intuitively,several individual monitoring models can be built for multiple operating modes,but two essential problems need to be addressed.One is how to cluster the training data according to operating modes.Obviously,traditional clustering algorithms such as k-nearest neighbors and fuzzy c-means method can be directly employed.To improve the clustering accuracy,Zhu et al.[3]utilized a bilayer clustering approach,which consists of an ensemble moving window strategy and an ensemble clustering solutions strategy to obtain partitioned subgroups.Zhao[4]proposed a concurrent phase partition and between mode statistical modeling strategy for batch processes with multiple phases and operating modes.The other problem is how to perform the online monitoring approach.If only one sub-model is applied at one sampling moment,specific rules must be developed to determine the most suitable sub-model.The distance between the sample and the center of local models can be used as an indicator[5]and the minimum SPE value is also effective for choosing a proper local PCA model[6].Feital et al.[7]proposed a component-wise identification approach to estimate the current operating condition through optimizing a maximum likelihood objective function.If all sub-models are trustworthy,their monitoring results can be involved with respect to their posterior probabilities.Based on Bayesian inference,Yu and Qin[8]developed a global probabilistic index while Xie and Shi[9]developed an integrated Mahalanobis distance monitoring statistic.Similarly,Ge and Song[10,11]combined the results achieved by some mixture modeling strategies such as MPPCA and mixture factor analysis.

Meanwhile,some adaptive methods have been improved so that they can tackle both time-varying and multimode behavior.Jin et al.[12]added a setofif–then rules into the monitoring scheme of recursive PCA from some extracted process factors for online identification of mode changes.With sufficient data of possible operating modes collected beforehand,Ge and Song[13]proposed a real-time modeling procedure with the online regression model which is built based on the relevant data set picked out from the historical database.Xie and Shi[14]and Yu[15]built Gaussian mixture models in the offline phase and performed the online updating in two different dynamic fashions with a moving window strategy and a particle filter re-sampling approach.By considering the non-Gaussian characteristic,Ma et al.[16]proposed a moving window local outlier factor algorithm.

Recently,building a global model has been proven to be another effective way to cope with multimode problems.Ma et al.[17]introduced a local standardization strategy into the modeling phase to normalize the distance between samples and its neighbors.Ghosh and Srinivasan[18]put forward a process monitoring scheme based on the immune-inspired negative selection principle where samples from normal operating conditions and known faults are utilized to construct a non-self-space in the form of a collection of spherical detectors.

In the existing approaches,within-mode correlations have been intensively analyzed,so that a high resolution can be offered by the constructed local models.However,cross-mode correlations are often inadvertently neglected.Zhang et al.[19,20]have claimed that this significant issue should be taken into consideration.As effective improvements,they have proposed two approaches to extract the common information between different modes and calculated the left specific information of each individual mode accordingly.Nevertheless,the question which monitoring model is adopted online still must be answered because multiple models are built offline.

Inspired by the work of Teh and Roweis[21],correlations among different operating modes can be preserved by integrating all local models together into a global one.In this study,to take full account of within-mode and cross-mode correlations for multimode process monitoring,a novel method named aligned mixture probabilistic principal component analysis is proposed.The main contribution of this paper is that a new angle of solving the multimode problem is introduced through aligning all constructed models.First,multiple local probabilistic models are constructed by using MPPCA and responsible levels of each sample are estimated with respect to each local model.Then,by minimizing the information loss of the alignment,all local models are merged into a low-dimensional coordinated space.Finally,remaining monitoring procedures including the definition of statistics and the online evaluation of new samples are conducted in the coordinated space.

2.Fault Detection Based on AMPPCA

For exploiting both within-mode and cross-mode correlations,MPPCA is employed first to build multiple local probabilistic models corresponding to the known operating modes.Then an automatic alignment procedure is conducted,in which a global coherent coordinate system for the original data is learned by mapping the disparate local representations together.

2.1.Construction of multiple local models based on MPPCA

To achieve the final low-dimensional coordination Y=[y1,y2,…,yN]∈Rd×N,MPPCA is first employed to automatically estimate several local models based on training data X=[x1,x2,…,xN]∈RD×Ncollected from the known operating modes.Let the number of operating modes be K and the overall distribution of the mixture latent variable models should take the following form.

2.1.1.E-step

The posterior mixing probability p(ω|xi,)and the latent variable posterior probability p(zi|xi,ω,)of each sample can be determined as follows.

where p(xi|zi,ω,)follows the Gaussian distribution N(+,()2I)and thus the expectation of latent variables and its second moment can be calculated.

2.1.2.M-step

First,the prior mixing probability of a local model can be updated by summing up the posterior mixing probabilities of all samples relative to this model.

By setting the derivative of the expectation of complete-data log likelihood to be zero with respect to μω,Λωand,their updated values can be calculated.

2.2.Alignment of local latent subspaces

As pointed out in[21],rather than considering the collected variables from different clusters to be single quantities,such as performing density estimation or dimension reduction separately,it is possible to view them as networks that convert high-dimensional inputs into a vector of internal coordinates from each sub-model,accompanied by responsibilities.Hence,the final coordination can be obtained by averaging the guesses using the responsibilities as weights Y=LTU,where the integrated matrix U∈RKm×Nand its general element is Ui,ω=p(ω|xi,Θ) ?E(zi|xi,ω,Θ).To evaluate the mapping matrix L,a specific cost function is constructed by minimizing the reconstruction error in the original data space.

where p represents the number of neighbors of xiand Wijrepresents the construction weights of xjwith constraints=1(i≠j).The weights summarize the local geometrics relating the data points to their neighbor.Therefore,to preserve the same relationships among the coordinates,the same cost function is utilized.Hence,the optimization problem can be finalized.

In order to remove an arbitrary scaling factor in the projection,a constraint is enforced as follows.

The minimization problem in Eq.(11)reduces to

Finally,the mapping matrix L that minimizes the objective function is given by solving the following generalized eigenvalue problem.

where L comprises the eigenvectors corresponding to d smallest eigenvalues.For fault detection,the popular Hotelling's T2is employed as the monitoring statistic.

According to[10],the control limit of T2in each local PPCA model can be determined by the χ2distribution.Because each latent variable subspace of MPPCA follows a standard Gaussian distribution and the coordinate space can be recognized as a linear combination of latent subspaces,the control limit of T2for the global model should similarly be determined by the χ2distribution.

Remark 1.The representations of a sample in different local models may be various,but after the alignment,only one vector in the final coordination represents this sample,so that cross-mode correlations are largely preserved.When optimizing the information loss,nearby samples in the original data space are constrained to be close to each other in the coordinated space,so the local structures of original data can also be preserved.

Remark 2.The sub-models are merged together in a linear form but no more information is extracted through the alignment approach.The dimension of latent variable subspaces m should be in the range d≤m≤D,where d and D are the dimensions of final coordination and original data,respectively.

2.3.Monitoring scheme of AMPPCA

Of fline modeling:

(1)Collect equal numbers of samples from each operating mode in the historical database and standardize the training data to eliminate the difference in scales.

(2)Apply the MPPCA algorithm to obtain individual parameter sets Θω(ω =1,2,…,K),latent variable subspaces E(Z|X,ω,Θω),and posterior probabilities p(ω|X,Θω).

(3)Determine the mapping matrix L by solving the eigenvalue problem,Eq.(14).

(4)Calculate T2values for the training data and estimate a control limitwith a given confidence level α.

Online monitoring:

(1)For a new data sample xnew,use the mean and variance attained in the first step of offline modeling for standardization.

(2)Compute its latent subspaces E(|xnew,ω,Θω)and post probabilities p(ω|xnew,Θω)with respect to each local sub model through the E-step based on the parameter sets Θω(ω =1,2,…,K).

(3)Construct new integrated matrix Unewand compute the aligned vector through ynew=LTUnew.

3.Case Study

3.1.Non-isothermal CSTR

A non-isothermal continuous stirred tank reactor(CSTR)is simulated,as shown in Fig.1.The process model and simulation conditions are similar to that introduced by Yoon and MacGregor[22].The simulation is based on three assumptions:perfect mixing,constant physical properties and negligible shaft work.

The dynamic features of CSTR process are described by the component material balance on the reactant and the energy balance.

It should be noted that only the PI control loop for outlet temperature T is active in this simulation.The key parameter settings for three operating modes are listed in Table 1.More details about the simulation parameters can be referred to[16,22].

Set the sampling interval to be 1 min.In the training period,500 data samples are collected under each operating mode.In the test stage,two fault scenarios are designed but the transient stages are not considered.(1)The process initially operates under mode 3,and after 500 min,the process switches to mode 2.At 1000 min,a step bias is introduced in cooling water temperature sensor with a magnitude of 4 K.(2)The process initially operates under mode 2,and after 500 min,the process switches to mode 1.At 1000 min,a drift is introduced in the sensor of inlet solute concentration CAAwith a slope d CAA/d t=0.005 kmol·m?3·min?1.

Fig.1.Diagram of CSTR process.

Table 1 Three operating modes of CSTR

Asa comparison,the combined MPPCA(CMPPCA)algorithm utilized in this paper is similar to that in[10].The only difference between the proposed method and CMPPCA is that after applying EM algorithm,AMPPCA employs the alignment approach to integrate local sub models while MPPCA combines the monitoring results of these submodels.In the proposed method,the latent subspace dimension is d ≤ m ≤ D,so two representative conditions are tested here.‘AMPPCAF’assumes m=D while ‘AMPPCAR’assumes m=d.The proper dimension of latent subspace for CSTR is 6 and d=6 in this simulation[16].The number of local models is assumed to be known and chosen to be 3.20 nearest neighbors are selected for describing the local structure.The confidence limit is set to be 99%.The modeling development process is run for 50 times and the best results of the three methods are shown in Table 2.Type II error of AMPPCA is better than that of CMPPCA,because cross-mode correlations are not involved in the monitoring model of CMPPCA.Furthermore,AMPPCAFgets superior results than AMPPCAR,because more information is included by employing more latent variables in each subspace.Besides,a satisfactory Type I error can be achieved.

Table 2 Monitored results of CSTR

Fig.2 illustrates the effectiveness of the proposed method in tackling the fault described in Case 2.The fault occurs due to a drift in the sensor of inlet solute concentration CAAand it sequentially results in drifting behavior of outlet temperature T and outlet concentration C.Owing to the PID control loop,the drift in T will lead to a drift in the manipulated variable FC.This fault is not significant at the beginning,so there are some detection delays with the three monitoring methods.By including more latent variables and cross-mode correlations,AMPPCAFtakes the lead in alarming the fault around 1070 min.After about 30 min,AMPPCARdistinguishes the faulty behavior.It takes another 50 min for CMPPCA to detect the fault.Therefore,a conclusion can be drawn that the proposed method is more practical than CMPPCA.

3.2.TE benchmark process

To validate the effectiveness of the proposed method over MPPCA,the popular TE benchmark process is employed[8,10,17].With different G/H ratios,there are six operating modes but only modes 1 and 3 are simulated here.The simulink programs are downloaded from http://depts.washington.edu/control/LARRY/TE/download.html.TE process contains 22 continuous process variables,12 manipulated variables and 19 composition variables,but only the 22 continuous process variables and 9 of 12 manipulated variables are selected in this paper.The steady-state values of the recycle valve and steam valve in mode 1 always equal 1,and the agitator rates in the two modes are 100 throughout the simulation.1000 samples are collected under each operating mode for offline training of the monitoring model.The number of nearest neighbors is set to be 20.The confidence limit is set to be 99%.Then,for the testing purpose,three cases with various types of process faults are designed to compare different fault detection methods.Similarly,the transient stages are not considered.

Case 1.1–100 samples are under the normal operating condition of mode 3.From the 101st to 300th sample,a random variation is introduced in condenser cooling water inlet temperature.301–400 samples are under the normal operating condition of mode 1.From the 401st to 600th sample,the reactor cooling water valve is stuck.

Case 2.1–100 samples are under the normal operating condition of mode 1.From the 101st to 300th sample,a slow drift is introduced in the reaction kinetics.301–400 samples are under the normal operating condition of mode 3.From the 401st to 600th sample,a random variation is introduced in A+C feed composition(steam 4).

Case 3.1–100 samples are under the normal operating condition of mode 3.From the 101st to 300th sample,a random variation is introduced in reactor cooling water inlet temperature.301–400 samples are under the normal operating condition of mode 1.From the 401st to 600th sample,an unknown disturbance is introduced.

In Case 1,a random variation fault and a stuck valve fault are introduced.The monitoring results of CMPPCA,AMPPCARand AMPPCAFare illustrated in Fig.3.The amplified parts show that without cross-mode correlations,CMPPCA can be easily deceived by random variations,so its two monitoring statistics miss more alarms from the 120th to the 135th sample.With the valve of reactor cooling water stuck,CMPPCA cannot give a persistent alarm and misses many alarms.In contrast,the proposed AMPPCA method performs superiorly,attributed to its inherent feature accounting for both within-mode and cross-mode correlations.AMPPCAFgets the best detection results,because more information may be included by employing more latent variables.

Fig.2.Monitoring results of Case 2.

Fig.3.Monitoring results of Case 1 of TE process.

For the second case,the normal operation of the plant is mixed with another two faults,which are a slow drift in reaction kinetics and a random variation in A+C feed composition.It can be readily seen from the amplified parts of Fig.4 that CMPPCA has a delay of at least 30 samples in alarming the drifting fault while SPE gets an even longer detection time.Meanwhile,CMPPCA still misses too many alarms.Consequently,Type II errors of T2and SPE in Table 3 are as high as 23.75%and 33.75%,respectively.Since the drifting error is not significant at the beginning,AMPPCAFcannot detect this fault from the 140th sample but it can maintain its effectiveness in dealing with random variations,so its Type II error is only 14%.For AMPPCAR,the performance is again middling.The above monitoring results manifest that considering cross-mode correlations and employing more latent variables are valid to improve the model accuracy and fault detection rates.

4.Conclusions

In the present article,an online monitoring scheme based on AMPPCA is proposed for fault detection of multimode processes.MPPCA is first performed to account for the within-mode correlations and an alignment optimization approach is introduced to merge all local sub-models together by minimizing the reconstruction error.In this way,cross-mode correlations are preserved in the final coordination and the global model is more dependable than any local model.The most important contribution of the paper is that it provides a different viewpoint for solving the multimode problem with multiple models.To validate the efficiency of the proposed monitoring scheme,the simulation and analysis of recorded data from a non-isothermal CSTR process and the TE benchmark process are presented.All designed scenarios confirm that the proposed method presents superior performance compared with CMPPCA in terms of Type II error and detection delay.Furthermore,employing more latent variables is feasible.Further efforts can be devoted to deal with strong nonlinear behavior and new operating modes.

Fig.4.Monitoring results of Case 2 of TE process.

Table 3 Monitored results of TE process