Local component based principal component analysis model for multimode process monitoring

Yuan Li,Dongsheng Yang

Department of Information Engineering,Shenyang University of Chemical Technology,Shenyang 110142,China

Keywords: Principal component analysis Finite Gaussian mixture model Process monitoring Tennessee Eastman (TE) process

ABSTRACT For plant-wide processes with multiple operating conditions,the multimode feature imposes some challenges to conventional monitoring techniques.Hence,to solve this problem,this paper provides a novel local component based principal component analysis (LCPCA) approach for monitoring the status of a multimode process.In LCPCA,the process prior knowledge of mode division is not required and it purely based on the process data.Firstly,LCPCA divides the processes data into multiple local components using finite Gaussian mixture model mixture (FGMM).Then,calculating the posterior probability is applied to determine each sample belonging to which local component.After that,the local component information(such as mean and standard deviation)is used to standardize each sample of local component.Finally,the standardized samples of each local component are combined to train PCA monitoring model.Based on the PCA monitoring model,two monitoring statistics T2 and SPE are used for monitoring multimode processes.Through a numerical example and the Tennessee Eastman (TE) process,the monitoring result demonstrates that LCPCA outperformed conventional PCA and LNS-PCA in the fault detection rate.

1.Introduction

In the industrial process,process monitoring technique is a tool that can be used to enhance the overall efficiency of industrial equipment [1,2].In recent years,multivariate statistical process monitoring (MSPM) methods have gained more consideration since the increasing availability of process data [3,4].The most popular method applied to MSPM is the principal component analysis (PCA).Nevertheless,PCA is inappropriate for monitoring processes with complex features.Thus,the PCA extension methods have been developed,including dynamic PCA [5,6],kernel PCA[7,8],multiway PCA [9,10],and other methods [11]to solve the problem of dynamic,nonlinearity,and multi-batch in the processes.For processes with multiple operating conditions,multiblock PCA [12,13],distributed PCA [14,15]and local PCA [16,17]methods have been used for monitoring the status of a multimode process.

The processes with multiple operating conditions have drawn increasing interest in modern industries.Unfortunately,the global PCA is inappropriate for monitoring the multimode processes since it ignores the local process information and the monitoring results are also difficult to interpret [18].Hence,in multimode processes,the multi-block approach is proposed to reduce process monitoring complexity and provide accurate monitoring results.MacGregoret al.proposed a multi-block partial least square (PLS) method and established monitoring charts for the individual process subsections as well as the entire process [19].Qinet al.analyzed several multi-block PCA and PLS methods in detail [20].Zhaoet al.proposed a multiple PCA monitoring method,which used the principal angle to measure the similarities between any PCA models and then determined squared prediction error (SPE) to facilitate process monitoring [21].Jianget al.developed a multimode process monitoring method based on mutual information-based multi-block PCA,joint probability,and Bayesian inference [22].However,the multi-block method ignores the process information during the transition period of multiple operating modes.Besides,the knowledge of manually segmented process is always unavailable practically.To solve these issues,mixture models are applied to monitor the status of multimode processes[23].Yuet al.developed a finite Gaussian mixture model (FGMM) to automatically identify the number of Gaussian components.Then the Bayesian inference is applied to derive an integrated global probabilistic index for fault detection [24].Yuet al.proposed a principal components-based Gaussian mixture model (PC-GMM),which is used multiple Gaussian components for handling complex processes with nonlinearity or multimode features [25].Zhuet al.proposed a distributed GMM method to divide sub-block and operating mode recognition[26].Jianget al.proposed a variational Bayesian Gaussian mixture model with canonical correlation analysis strategy[27].In practice,the above multimode process monitoring methods have been proved to be effective.However,the establishment of these models depends on the reliable mode recognition performance of the mixture model.

Hence,local modeling methods have been developed to monitor the status of a multimode process.Heet al.proposed a K nearest neighbor-based local modeling method,which uses local distance for monitoring processes [28,29].However,this method only considers the location information of data and ignores the data direction variation.Maet al.applied the local neighborhood standardization (LNS) strategy to eliminate the influence of multimode features on process control,and then built a PCA model(LNS-PCA)to monitor the entire process[30].Denget al.proposed a local neighborhood similarity analysis method for monitoring processes [31].Unfortunately,these methods require numerous computations to search for local information.Meanwhile,the prior knowledge to select the number of neighbors is difficult to obtain.

Basically,a good local model requires local data to be as simple as possible,while the entire process can be described by all neighbor information.Based on these analysis,this paper provides a novel local component based PCA monitoring model (LCPCA).The main contributions of this paper are as follows.First,this paper uses local component standardization to describe the whole multimode processes,which eliminates the impact of multimode properties on the conventional PCA method.Second,by FGMM,LCPCA can automatically identify the number of local components and be used to standardize the historical process data.Third,LCPCA can effectively avoid the problem of local dataset including different mode data and provide suitable data foundation for monitoring process.

The rest of this study is structured as follows.Firstly,PCA and LNS-PCA method are provided in Section 2.Then,the LCS-PCA monitoring method is proposed in Section 3.In Section 4 and Section 5,two case studies are presented through a numerical example and a benchmark of the Tennessee Eastman process.Ultimately,conclusions are made in Section 6.

2.Traditional Method: PCA and LNS-PCA

Suppose that X ∈Rd×Nis a data matrix,whereNis the number of samples anddis the number of variables.At first,the covariance of X can be calculated as follows:

where C represents the covariance matrix of X.Then,PCA obtains the loading matrix P through eigenvalue decomposition of C.After that,PCA decomposes X as the score T ∈Rκ×N,loading P ∈Rd×κ,and residual E ∈Rd×N,as follows:

where κ ≤dis the number of principal components (PCs).In general,the number of κ can be calculated through cumulative percentage of variance (CPV) method.The columns of P are actually eigenvectors,which determined through the first κ eigenvalues.

For online process monitoring,T2andSPEstatistics are used to monitor the principal component subspace (PCS) and the residual subspace (RS),individually.Consider a new sample xnew∈Rd,theT2statistic can be calculated as:

where Λ ∈Rκ×κis a diagonal matrix indicating the variances of the retained principal component scores.The SPE statistic can be calculated as:

To solve the problem caused by the multimode features,Maet al.developed the LNS-PCA monitoring method,which utilized neighborhood mean and standard deviation to standardize the samples [30,33].The detail of LNS-PCA is as follows.

For a new samplexi,the π nearest neighbors of xiin training data X can be determined as:

As the local neighborhood dataset become available,the LNS method is performed as follows:

whereme（N（xi） andst（N（xi）） denote the mean and standard deviation of neighborhood dataset.

In comparison with PCA,using LNS strategy makes PCA monitoring the multimode process effectively.However,the number of neighbors affects the monitoring performance of LNS-PCA in the multimode processes.Meanwhile,LNS-PCA utilizes the Euclidean distance to find the local neighborhood,which may cause the neighborhood dataset to still present the multimode features.

3.LCPCA for Process Monitoring

In this section,we will introduce a novel LCPCA method,which is a powerful monitoring technique.Firstly,LCPCA does not required the process prior knowledge of mode division and it is based on the process data,purely.Secondly,the proposed method can avoid the problem which the local neighborhood data includes the multiple mode.Thirdly,LCPCA provides a suitable data foundation for calculating monitoring statistics.The main idea of LCPCA is represented as follows.

Suppose X ∈Rd×Nis a data matrix withNsamples anddvariables from multimode processes.Firstly,using the FGMM,the multimode process data is divided into the local components in LCPCA.The probability density of X can be determined as:

whereKdenotes the number of local components in FGMM.ωkrepresents the weight of each local component,and it satisfiesindicates the Gaussian probability function.μkand Σkare a mean vector and a covariance matrix of thek-th local component,respectively.The unknown parametersshould be estimated to establish the FGMM model.The procedure of parameter estimation is as follows.

The first step is:

wherep（s）（|xi） is the posterior probability ofi-th sample within thek-th local componentat thes-th iteration.

The second step is:

Next,LCPCA computes the posterior probability for each sample x belonging to each Gaussian component as follows:

According to the posterior probability,the process data X can be divided into multiple local componentn（xk） as follows:

wheren（xk） represents thek-th local component.In this step,the samples of local component overlap region can be divided by calculating the posterior probability.We take one of the mode as an example.As shown in Fig.2,each sample in local component A and B is distinguished by different symbols.It can be clearly seen that each sample can be described using to the corresponding local component.

Then,LCPCA standardizes each sample of local componentn（xk）as follows:

Fig.1.Scatter plots of the local component.

Fig.2.Scatter plots of the sample contained in each local component.

where（xk） denotes the local component,which has been standardized.x（k）irepresents thei-th sample in thek-th local component.m［n（xk）］ ands［n（xk）］ are a mean vector and a standard deviation vector of thek-th local component.Thus,the process data X can be reconstructed as:

Take two components have different distribution as example,as shown in Fig.3.Through component information(mean and standard deviation),LCPCA make the standardized component samples（xk） close to the origin.

Finally,the PCA model is built to monitor the standardized process data.According to Fig.4,the process datahave been already normalized to a single distribution and eliminated the multimode features through Eq.(14).Furthermore,LCPCA makes the PCA monitoring model robust performance increasing.

For a new observation sample x*,LCPCA is first to compute its posterior probability,and determines it belongs to which local componentn（xk）.Then,the local information ofn（xk） is used to standardize x*through Eq.(14).Finally,LCPCA used PCA model to monitor the change in principal component subspace (PCS) and residual subspace (RS) of.LCPCA includes two parts,the details of offline modeling and online monitoring are as follows.

Fig.3.Scatter plots of the standardized samples of A and B local component.

Fig.4.Scatter plots of standardized samples and its variable distribution histogram in LCPCA.

Part 1: Offline modeling

(1) Collect the process data X under normal multimode operation conditions;

(2) Decompose X into multiple local componentn（xk） using FGMM technique;

(3) Use the local information(such asm［n（xk）］ ands［n（xk）］) to standardize each sample inn（xk）;

(4)Acquire standardizedprocess data;

(5)Build the PCA modelon the processdata matrix;

(6) Calculate theT2and SPE statistics in PCS and RS,respectively;

(7) Determine the control limitsCLT2andCLSPEusing K DE.

Part 2:Online monitoring

(1) Acquire a new sample xnew,calculate it posterior probability to determine it belongs to which local component;

(2) Use the local information to standardize xnew,and get;(3) Mapto the PCS and RS,and then calculate theand SPEnewstatistics;

Fig.5.Scatter plots of the numerical example data.

4.Numerical Case Study

In this section,a simple numerical example is used to the simulation experiment.The simulation data are generated from the following process system:

Fig.6.Scatter plots of the samples in PCS.

Fig.7.Monitoring results for the numerical example using PCA.

Fig.8.Scatter plots of the neighborhood of the fault sample in LNS-PCA.

where s1,s2,and s3represent the signal sources.Through adjusting the signal sources s1，s2，s3to generate different operating modes as follows:

Fig.9.Scatter plots of the standardized samples in PCS of LNS-PCA.

Fig.10.Monitoring results for the numerical example using LNS-PCA.

Fig.11.Scatter plots of the sample of the a local component in LCPCA.

In this simulation,we used this process system to generate 100 samples under each mode,and a total of 200 samples are used for off-line modeling.Then,two datasets are generated as the testing data,and the 100 fault samples are generated as follows:

In the end,a total of 300 testing samples containing mode 1,mode 2 and fault mode are used to verify PCA,LNS-PCA and the proposed LCPCA method.All of the testing data are shown in Fig.5.

Fig.12.Scatter plots of the standardized samples in PCS of LCPCA.

Fig.13.Monitoring results for the numerical example using LCPCA.

Fig.14.Tennessee Eastman Benchmark process.

In PCA,the number of PCs is calculated as 2 through the 85%CPV.In LNS-PCA,the number of neighbors is chosen as 15,and the number of PCs is calculated as 2.In LCPCA,the number of local component is automatic determined by FGMM,and the number of PCs is calculated as 2 through 85% CPV.Moreover,we specify the control limits as 99% for each method.In each monitoring result figure,the red-solid lines denote the fault samples and the reddotted lines reflect the 99% control limits.

According to Fig.6,PCA cannot extract multimode features in this process system.In comparison with the 99%probability ellipse of normal distribution,it can be clearly seen that all fault samples are still contained in the ellipse.Hence,based on Fig.7,the monitoring results shows that all fault samples are not detected.

In LNS-PCA,the standardized results of LNS will be affected by the neighborhood of fault sample.If the number of neighbors are small,the neighborhood data may unable to describe the actual process information.In the other hand,if the number of neighbors are huge,the neighborhood data may include the multiple modes,which makes the neighborhood standard deviation inflated as shown in Fig.8.Meanwhile,according to the Fig.9,the most stan-dardized fault samples cannot be separated from the normal samples.This is because the fault samples minus the neighborhood means,then divides by the inflated standard deviation of its neighborhood is around 0.Hence,as shown in Fig.10,most fault samples are not detected in LNS-PCA.

In LCPCA,according to Fig.11,a probability model is used to divide the data into multiple local component and find the corre-sponding component for each sample.Meanwhile,this finding indicates that all fault samples belong to the same local component.Since each sample can only be described by one local component,which can avoid the problem of the neighborhood data including multiple modes in LNS-PCA.In comparison with LNSPCA,LCPCA can successfully maintain the deviation degree among the normal and fault samples using the local component standardization,as shown in Fig.12.Hence,LCPCA can detect most fault samples effectively as presented in Fig.13.The monitoring results of PCA,LNS-PCA,and LCPCA are listed in Table 1.The numerical case illustrates LCPCA possesses significant fault detection performance in multimode process.

Table 1Fault detection rates of PCA,LNS-PCA,and LCPCA

Table 2Six operation modes in TE process

Table 3Process variables used for process monitoring in TE process

Table 4Induced process faults in TE process

5.Tennessee Eastman Process

As a famous plant-wide process,the Tennessee Eastman (TE)benchmark is widely used for evaluating process monitoring methods.Downs and Vogel proposed a prototype problem based on the Tennessee Eastman chemical company [35].The TE process comprises five basic units:a reactor,a condenser,a vapor–liquid separator,a recycle compressor,and a product stripper,the systematic illustration as shown in Fig.14.The TE process includes 12 manipulated variables and 41 measurement variables.There is composed of six different operating modes that have different G/H mass ratios.A detailed description of the modes is listed in Table 2.Additional details on the TE process can be found in Ref [35].

In this simulation,33 process variables are selected for monitoring,as illustrated in Table 3.The operating Mode 1 and Mode 3 are used to simulate multimode processes.A set of different types of 21 faults is simulated in each mode of TE process as listed in Table 4.To establish the monitoring model,961 normal samples under each mode are collected.Furthermore,160 normal samples and 801 fault samples under each mode compose the testing data,and a total of 1922 samples are used to evaluate the efficiency of the proposed monitoring method.In PCA,the number of PCs is determined as 1 through the 85% CPV.In LNS-PCA,the number of neighbors is set as 30 according to the Ref [30],and the number of PCs is set as 18 through the 85% CPV.In LCPCA,the number of local component is automatically determined using the FGMM,and the number of PCs is set as 23 through the 85% CPV.The confidence level of each method is set as 99%.Table 5 illustrates the fault detection results of PCA,LNS-PCA,and LCPCA.It can be found that LCPCA outperforms conventional PCA and LNS-PCA technique in the performance of monitoring.The Fault 10 is selected to demonstrate the effectiveness of the proposed method.

Table 5The fault detection rates of PCA,LNS-PCA,and LCPCA

Fault 10 is generated by random variation in C feed temperature.Fig.15(a) represents the probability plots of the first PCs in the PCA method.This finding indicates that the shifts in the operating mode cause significant differences in process information and then make the process data include multiple modes features.Meanwhile,the conventional PCA method based on the single distribution assumption performs poorly in this case.Hence,as shown in Fig.15(b),PCA cannot detect most of fault samples.According to the Fig.16(a),through the LNS-PCA method,the process variables basically follow a Gaussian distribution.However,as mentioned above,LNS-PCA cannot avoid the problem caused by the neighborhood data including multiple modes.Thus,a part of fault samples is not detected as shown in Fig.16(b).Compared with PCA and LNS-PCA,the LCPCA method can use local component information to standardize process data and make the process variables follow a Gaussian distribution,as shown in Fig.17(a).Therefore,accord-ing to the Fig.17(b) the monitoring results indicate the LCPCA plays a more effective role in monitoring the multimode process.

Fig 17.(a) Probability plots of the first PCs in LCPCA (b) Montitroing results for TE process using LCPCA.

Fig.15.(a) Probability plots of the first PCs in PCA (b) Montitroing results for TE process using PCA.

Fig.16.(a) Probability plots of the first PCs in LNS-PCA (b) Montitroing results for TE process using LNS-PCA.

6.Conclusions

In this paper,the local component based principal component analysis method is proposed to eliminate the multimode features and then monitor the status of processes.The three features of LCPCA are summarized as follows.First,LCPCA can eliminate the impact of multimode properties in PCA.Second,LCPCA can efficiently divide the process data,and automatically select the number of the local component without prior knowledge.Third,proposed method can avoid the problem of the local neighborhood data including multiple modes.The validity and superiority of the LCPCA are verified by a multimode numerical case and a benchmark of the Tennessee Eastman process.The application results demonstrate the superior performance of LCPCA compared with PCA and LNS-PCA in the multimode process.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The authors would like to acknowledge that the National Natural Science Foundation of China (61673279).