Development cost prediction of general aviation aircraft projects with parametric modeling

Xiaonan CHEN, Jun HUANG, Mingxu YI

School of Aeronautic Science and Technology, Beihang University, Beijing 100083, China

KEYWORDS BP neural network;Development cost;General aviation aircraft;Gray correlation analysis;Linear regression;P value analysis;Parametric modeling;Preliminary prediction;Sensitivity analysis

Abstract The study of the development cost of general aviation aircraft is limited by small samples with many cost-driven factors.This paper investigates a parametric modeling method for prediction of the development cost of general aviation aircraft.The proposed technique depends on some principal components,acquired by utilizing P value analysis and gray correlation analysis.According to these principal components,the corresponding linear regression and BP neural network models are established respectively.The feasibility and accuracy of the P value analysis are verif ied by comparing results of model f itting and prediction.A sensitivity analysis related to model precision and suitability is discussed in detail.Results obtained in this study show that the proposed method not only has a certain degree of versatility, but also provides a preliminary prediction of the development cost of general aviation aircraft.

1. Introduction

The development and production of aircraft are considered complex systematic engineering,characterized by a long development cycle, technical diff iculties, complex system composition, uncertain factors, etc. Thus, the development and manufacturing costs of aircraft are huge.1-5However,successful aircraft development depends on the f inancial viability of a business and its operation.Accordingly,effective prediction of the development cost is crucial.

General aviation aircraft are now the most versatile type of aircraft on earth,accounting for nearly 70%of the total number of aircraft worldwide and 90% of the civil aircraft.6The study of general aviation aircraft development had a relatively late start,and no cost evaluation model has been suitable for a small sample.As the general aviation industry develops continuously, the development cost has increased substantially,becoming a heavy burden for aircraft manufacturers.Relevant cost estimation methods are limited by small samples and poor information factors due to the short development time of general aviation aircraft, the lack of relevant parameter information, and diff iculties in data collection. Hence, the method proposed in this paper starts from the development cost of general aviation aircraft projects to f ind an appropriate parametric model in the case of small samples.

Existing aircraft cost prediction methodologies are primarily either cost accounting methods7,8or parametric techniques optimized for the minimum cost.9-14The accuracy of methods for aircraft development cost prediction is a problem of increasing importance, in terms of both f itting and predicting the cost. This is because a good cost prediction must involve a good estimation of how much an aircraft will cost on delivery.15The parameter approach is now the most commonly used cost estimation method. Besides, it is broadly applied in the international f ield, such as the engineering estimation method, analogy methodology, extrapolation scheme, etc.16-18Ben-Arieh19predicted the manufacturing cost by focusing on equipment installation, processing, and raw material costs.Yui and Egbelu20summarized a framework to predict the manufacturing cost, and Kulkarni and Bao21developed a model for predicting the process cost. The Back-Propagation(BP) neural network is a comparatively successful predictive model.22-26Curran et al.3presented a general model for cost prediction that could be applied in the design f ield. Liu and Xie27incorporated the GM(0,N)model with a neural network algorithm, and a neural network was applied to optimize the predicted value of the GM(0,N) model. Gu¨nayd?n and Dogˇan28investigated the utility of neural network methodology to overcome cost estimation problems in early phases of a building design process. They observed that as the number of samples was increased,the building cost was estimated more accurately. The most important advantage of neural networks is that they can recognize the relationship between the development cost and parameters.However,the neural network model has poor predictability when data is limited.28-31Sonmez and Ontepeli30combined BP neural network techniques with regression model analysis to get a suitable model for cost estimation of urban railways.Through comparison and prediction results analysis,the applicability of linear regression analysis in urban railway cost estimation was determined.Compared with other urban railway cost prediction methods, this method has a certain degree of generalization signif icance.Considering the lack of a corresponding domestic general aviation aircraft cost prediction model, and because relevant cost parameters are diff icult to obtain, this paper addresses the current reality,and inspired by Sonmez and Ontepeli,30proposes to establish a cost prediction model that is suitable for small samples and poor information available for general aviation aircraft.

The main task of this work is to establish a general aviation aircraft cost prediction model that is suitable for domestic situations.Not all characteristic parameters that might affect the development cost will have a def inite and obvious effect.Therefore, the popular gray relational degree analysis method and a proposed P value analysis method are analyzed simultaneously in this paper. Taking into account the diff iculty in obtaining data costs for general aviation aircraft and the actual status of long-term small samples, linear regression analysis and BP neural network technology are compared to select the most promising parametric model.

2. Parameter description

To establish a development cost model, 17 aircraft projects27were collected, and nine random projects utilized to select the principal components and build the cost model (see Table 1), while the other eight projects were used to test the model. Previously reported data was used to support this study, and is available at Ref.27. These prior studies are cited at relevant places within the text as references.

Seven factors have been identif ied as the main factors inf luencing the development cost of general aviation aircraft,which include the maximum take-off weight (Mw), Mach number(Ma),maximum range(MR),maximum thrust(MT),maximum ceiling(ML),maximum oil load(Mo),and length of the fuselage(Lf). This selection is based on engineering parts, because the factors are primarily controlled by the design rather than by economic f luctuations.The gained parameters should be reconciled with general aviation aircraft design standards.

3. PCA (Principle Component Analysis)

Due to small samples,various cost-driven factors,and the existence of multi-collinearity,the conventional regression method has diff iculties predicting the development cost of general aviation aircraft accurately.Principle Component Analysis(PCA)is a multivariate statistical analysis method specially designed to deal with small samples,multiple variables,and high correlation among variables.

Therefore, the popular gray correlation analysis method and the proposed P value analysis method are analyzed simultaneously in this work.At the same time,the most appropriate PCA method is determined through a comparison between model results. The design parameters selected as cost-driven factors should satisfy the following requirements:

(1) The selected parameters and the predicted cost are strongly correlated.

(2) Changes in the characteristic parameters comply with changes in the development cost caused by these parameters.

(3) Characteristic parameter values are easy to determine.

Table 1 Development cost and related parameters.

3.1. Gray correlation analysis

The theory of gray relational analysis applies to small samples.The basic idea of this theory is to analyze and compare the similarity between system data's column geometry and curve geometry. By using gray correlation analysis, cost characteristic parameters with a high correlation degree (ρ)can be determined, and characteristic parameters with a low correlation can be eliminated.

The degree of gray correlation between relevant factors and the development cost is listed in Table 2.The criterion for not incorporating a parameter in the f inal cost model is that the gray correlation ρ ＜0.6, which will allow only highly correlated parameters to be involved in the analysis. Thus, Ma,MR,and MLwith a gray correlation of smaller than 0.6 should be removed. Next, the remaining Mw, MT, Mo, and Lfshould be employed to build the development cost prediction model.

3.2. P value analysis

All of the seven factors are included as independent variables in the beginning linear regression model A1. The predicted development cost of general aviation aircraft (cpre) is used as the dependent variable in the model,λi(i=0,1,···,7) represents the regression coeff icient, the regression model written as

By analyzing the P value of the correlation coeff icient, the importance of every factor in the linear regression model is obtained.The bigger the P value is,the less signif icant the correlation parameter is to the model. Including these insignif icant factors in the model might result in a poor estimation effect. Consequently, eliminating these insignif icant factors can improve the estimation effect of the model. The P value of λ2is 0.98 in the linear regression model A1. Factor Ma is removed with the largest P value.

The rest of the parameters are included in the next linear regression model A2, which is written as

In the second linear regression model, parameter MT, with a P value of 0.75,indicates that MTdoes not have a signif icant contribution to the model; thus, MTis eliminated.

In a similar way,MLis the third parameter to be removed,with the largest P value of 0.49.MRand Moare the fourth and f ifth parameters to be removed. Factors Mwand Lfare included in the sixth linear regression model A6, the P values of which are 0.001 and 0.0003, respectively. Both parameters thus have signif icant contributions, because their P values are smaller than 0.05.

Through the P value analysis, the remaining Mwand Lfshould be employed to build the development cost prediction model.

Through the two different PCA methods above, the number and type of parameters retained are different. Using the principle components obtained by these two methods,the corresponding linear regression models and BP neural network models are established respectively, which will be introduced in the next section.

4. Development cost modeling

The f inal selected cost-driven factors are listed in Table 3.

Based on the results of these two analyses, the corresponding linear regression models and BP neural network models are established respectively.

A6 is the f inal linear regression model established through the P value analysis, in the following form:

At the same time, the linear regression model evaluated by gray correlation analysis is set as A7,and its model equation is as follows:

There is only one linear relation included in the linear regression model between the relevant parameters and the development cost. Including nonlinear terms may signif icantly inf luence model results. However, decisions related to relational categories become challenging when considerable amounts of factors are included. In view of this, two BP neural network models are developed. There are only two parameters included in the f irst BP neural network model B1, which are determined by P value analysis. In the second BP neural network model B2, all of the four relevant parameters that are determined by gray correlation analysis are included.

Table 3 Cost-driven factors.

Table 2 Gray correlation analysis.

5. Comparison and verif ication

5.1. Prediction performance

By comparing the prediction effects of these four models A6,A7, B1, and B2, the most appropriate model was selected to predict the development cost of general aviation aircraft. At the same time, the better principal component selection method can be determined.

The differences between the actual cost (cact) and the predicted cost (cpre) for general aviation aircraft should be investigated in the initial sample (see Table 1) before the model is applied to general aviation aircraft development cost prediction. These initial nine items were utilized to create the models. Meanwhile, the remaining eight projects were selected as a test sample.

The f itting accuracy of the linear regression models was tested. e is the error between the actual and predicted cost,eprepresents the error percentage. Table 4 lists f itting results.It is suggested from the results in Table 4 that the average f itting error percentage (ep) is 0.06% by using the linear regression model A6, which is much smaller than that of model A7.

The validity of the linear regression model was also tested by introducing it to the additional cost data that was not involved in the initial sample. Some more general aviation aircraft cost data and parameter information were collected and sorted in Table 5.32Previously reported data was used to support this study.32These prior studies are cited at relevant places within the text as references.

Predicted results are listed in Table 6. It is suggested from Table 6 that the average error percentages of the linear regression models A6 and A7 are 6.72% and 16.21%,respectively.

It is suggested from the results in Tables 4 and 6 that the linear regression model A6 is superior to A7 in both f itting and prediction performance. To compare the f itting and prediction performances of the two models more directly, the errors (e) in Tables 4 and 6 are expressed as absolute errors(|e|), and then a line chart is established between the two regression models.The absolute errors between actual and predicted costs are shown in Fig. 1. Meanwhile, a comparison between cactand cpreis shown in Fig. 2.

It is suggested from Figs. 1 and 2 that model A6 proposed in this paper has a relatively smooth trend of error curve,while the curve trend of model A7 is comparatively steep. That is,model A6 presented here overcomes the shortcomings manifested by model A7, such as unstable predicted results and large individual errors. On the other hand, this may be due to the difference in the principal component selection. To verify this view, the results of models B1 and B2 are predicted and proven in this paper.

Table 7 lists f itting and predicting results of models B1 and B2.It is suggested from the results in Table 7 that the average f itting error percentage(ep)is 0.44%by utilizing the BP neural network model B1, which is much smaller than that of model B2. Meanwhile, the average error percentages (ep) of the predicted results of neural network models B1 and B2 are 5.53% and 6.88%, respectively.

To compare the f itting and prediction performances of the two BP neural network models more directly, the absolute errors between actual and predicted costs are shown in Fig. 3.

It is suggested from Fig. 3 that model B1 is superior to model B2 in both f itting and prediction performances.

Table 4 Fitting results.

Table 5 Additional cost and parameter information.

Table 6 Predicted results.

Fig. 1 Absolute errors between actual and predicted costs.

Fig. 2 Comparison between actual and predicted costs.

Models A6 and B1 with two signif icant parameters have a slightly better performance than models A7 and B2 with four signif icant parameters.This also indicates that the P value analysis method introduced in this paper can be applied to the selection of main components of the model. Compared to the gray correlation analysis method,a better prediction is obtained.

Table 7 Fitting and predicting results.

Fig. 3 Absolute errors between actual and predicted costs.

5.2. Sensitivity analysis

Sensitivity analyses of linear regression model A6 and BP neural network model B1 are performed by using parameters Mwand Lf, which are obtained by utilizing the P value analysis method. The 3D plot of A6 is shown in Fig. 4.

The 3D plot of A6 shows a linear surface and indicates that the development cost increases with increases of parameters.Meanwhile, the development cost has remained positive. On the other hand,it is suggested from Fig.4 that the data points for the development cost are mostly located on the 3D plane.Therefore, A6 will be considered as the f inal candidate development cost prediction model.

Fig. 5 shows a comparison between actual and predicted costs of the BP neural network model B1.It is suggested from this f igure that model B1 here can get a better f itting and predicting effect on the data of the projects. Simultaneously,the 3D plot of B1 is given in Fig. 6. The BP neural network model B1 illustrates a nonlinear surface. The model suggests that after Mwreaches 500 t,and reducing the Lfvalue to about 100 ft,the predicted cost would be negative.This relation suggested by the BP neural network B1 is not suff icient, because the development cost must have a positive value. Although the BP neural network model B1 can provide a preferable f itting and predicting effect(see Fig.5),the sensitivity analysis(see Fig.6)reveals that model B1 is not suitable for predicting in the case of small sample data. Finally, A6 is chosen as the most suitable parametric model.

Fig. 4 3D plot of A6.

Fig. 5 Comparison between actual and predicted cost of B1.

Fig. 6 3D plot of B1.

6. Conclusions

P value analysis and gray correlation analysis have been implemented for determining the type and number of principal components. Through f itting and prediction results of linear regression models and BP neural network models,the feasibility of selecting principal components by the P value analysis method is verif ied. A sensitivity analysis reveals that the exact relationship between the development cost and parameters cannot be fully demonstrated by BP neural network models.Thus,the linear regression model A6 is determined as the f inal general aviation aircraft development cost prediction model.

This approach, to a certain extent, overcomes the shortage of research on cost prediction of general aviation aircraft.Although there are still some areas that need improvement,it can play a preliminary role in predicting the development cost of general aviation aircraft. The method proposed in this paper is, to a certain extent, universal. Thus, it can also provide some guidance for cost prediction of commercial aircraft.

Acknowledgements

The research was supported by the National Postdoctoral Program for Innovative Talents,Postdoctoral Science Foundation of China (No. 2017M 610740). In addition, the authors would like to acknowledge the supports from Hefei General Aviation Research Institute, Beihang University.