Multi-model ensemble forecasting of 10-m wind speed over eastern China based on machine learning optimization

Ting Lei ,Jingjing Min ,Cho Hn ,Chen Qi,＊ ,Chenxi Jin ,Shunglin Li

a Beijing Meteorological Service Center, Beijing, China

b Climate Change Research Center, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China

c Department of Atmospheric Science, China University of Geosciences, Wuhan, China

Keywords: Wind speed Machine learning optimization Ensemble forecast Ridge regression

ABSTRACT Wind substantially impacts human activity and electricity generation.Thus,accurately forecasting the short-term wind speed is of profound societal and economic significance.Based on 100 weather stations in eastern China,the authors first evaluate the performance of the 10-m wind forecast products from five operational forecast models.Among them,the Japan Meteorological Agency (JMA) model performs best in reducing the forecasting errors.Then,the authors establish a 10-m wind speed multimodel ensemble forecast based on the five numerical models’ outputs and machine learning methods,combining dynamic and statistical methods.Feature engineering and machine learning algorithm optimization are conducted for each site separately.The forecast performance of this method is compared to the JMA model and multimodel ensemble forecast by ridge regression at lead times of 24–96 h.The results demonstrate that the multimodel ensemble method based on machine learning optimization can reduce the forecast error of JMA by more than 39%,and the improvement in forecast skill is most evident in November.In addition,it performs better than the ensemble forecast by ridge regression.

1.Introduction

Energy security and environmental protection are issues of general concern to the global community today.Wind energy is favored as clean and efficient renewable energy and is increasingly important in the electric energy sector (Yu et al.,2016;Jung et al.,2019).According to the Global Electricity Review 2022,the share of electricity generated from wind reached approximately 6.59% (17.22%) of the global total(renewable) electricity in 2021 (https://ember-climate.org/insights/r esearch/global-electricity-review-2022/).Prior estimates of wind power from forecast models is critical for reducing the problem of grid integration and energy price trading;short-term forecasts are used by transmission system operators and energy traders (Georgilakis,2008;Makarov et al.,2009;Bianco et al.,2016;Mehr et al.,2021).However,the output power of wind turbines is sensitive to the wind speed.Low wind speeds mean it is hard to drive wind turbines,while high wind speeds can cause operational failures.Therefore,reliable short-term wind speed forecasting is urgently needed.

Various methods have been employed to forecast short-term wind speeds,including statistical approaches (traditional methods and machine learning methods) (Wang et al.,2018;?zen et al.,2019;Mehr et al.,2021) and numerical weather forecasting (Wyszogrodzki et al.,2013;Yao et al.,2018;Shi et al.,2022).Although the numerical weather forecasting is more skillful than pure statistical methods for forecasts at lead times of 48–72 h(Cassola and Burlando,2012),numerous forecast errors still exist because of the errors in the initial and boundary conditions of the model and the approximations of physical parameterization schemes (Rabier et al.,1996;Buizza et al.,1999).Therefore,producing forecasts with coupled dynamical–statistical algorithms is desirable for better accuracy(Zjavka,2015;Rodrigues et al.,2018;Sun et al.,2019;Xu et al.,2020).Recently,machine learning algorithms have been introduced into meteorological applications and have demonstrated better performance than traditional statistical methods in correcting the results of numerical forecasts(Sun et al.,2019;Peng et al.,2020;Fang et al.,2023).Along with the different methods of the various algorithms involved,the distinct features and uncertainties of individual models has led to the multimodel ensemble forecasting approach being proposed and preferred.Such an approach combines the advantages of several individual models and produces better results than any single model (Alessandrini et al.,2011;Zhao et al.,2016;Siuta and Stull,2018).Common multimodel ensemble methods include simple ensemble averaging (Feng et al.,2016),the super ensemble mean (Zhi et al.,2009),bias-corrected ensemble averaging (Zhi et al.,2012),partial least-squares regression (Li et al.,2018),and neural networks(Wang et al.,2023).

In terms of previous research on short-term forecasting of wind speeds in China,some studies have applied dynamical–statistical algorithms for short-term wind speed forecasting in recent years (e.g.,Shi et al.,2017;Ren et al.,2020;Wu et al.,2022).However,most of these focused on regional (and even individual wind farm) wind speed forecasts within 72 h,and using one statistical method.Considering different statistical methods possess different advantages and limitations,selecting one final“good enough”model is necessary.In addition,another key question is how the forecast skill is affected when the forecast lead time reaches 96 h,which is lead time urgently needed by the power sector.Accordingly,the aim of this study was to address these two important issues.

The rest of the paper is organized as follows:Section 2 describes the data and methods employed in our study.Section 3 evaluates the forecast accuracy of pure dynamical and dynamical–statistical forecasting.Finally,Section 4 provides a summary and some further discussion.

2.Data and methods

2.1.Data

The numerical forecast products used in this paper are from the European centre for Medium-Range Weather Forecasts (EC),the US National Centers for Environmental Prediction global forecast system(GFS),the Japan Meteorological Agency (JMA),and the China Meteorological Administration(CMA).This study uses two prediction products from the EC Integrated Forecasting System(IFS),the EC-IFS cycle 43r1 and 48r1.The former has a relatively coarser horizontal resolution of 0.125?(EC0.125),while the latter has a finer horizontal resolution of 0.1?(EC0.1).The variables include wind speed and its meridional and latitudinal components at 10 m,80 m,and 100 m (U10,V10,WS10,U80,V80,WS80,U100,V100,WS100),2-m air temperature(T2m),2-m relative humidity (RH) or dewpoint temperature (Td),and surface pressure (SP).Brief details of the models and variables used from each model are provided in Table S1.The observed hourly 10-m wind speeds at 100 automated weather stations in China are used in this study (htt p://data.cma.cn/en/?r=data/detail&dataCode=A.0012.0001),the geographical distribution of which is shown in Fig.S1.Referring to the observational data’s location and temporal resolution,respectively,the inverse distance weight and cubic spline interpolation method are used to interpolate the model forecast.

For numerical forecast results,we use the results predicted starting at 2000 CST (China Standard Time) every day,taking the forecast period from 0800 CST on the second day to 0800 CST on the fifth day(96 h in total)for analysis.The results with the forecast lead time at 24 h,48 h,72 h,and 96 h are referred to as the results of DAY1,DAY2,DAY3,and DAY4,respectively.

2.2.Methods

Two multimodel ensemble methods are adopted.One is ridge regression (RR),in which the 10-m wind speed forecasts of five numerical models and their time series are selected as features.The RR method represents an improvement compared with standard linear regression (Men et al.,2019).The optimal parameters are selected at each site by comparing the mean absolute error (MAE) under different regularization coefficients,and regression models are built for different sites.

The other method is machine learning optimization (MLO).Here,besides the 10-m wind speed forecasts of the single numerical models and time series,the air temperature,humidity,pressure,and upper-level winds from different models (rightmost column of Table S1) and the averaged 10-m wind speed of the five models are also selected as the original features.To obtain useful features,the feature engineering was conducted using the SelectFromModel algorithm,an important approach for feature selection based on feature importance.For a detailed description,readers may refer to the software package website(https://scikitlearn.org/stable/auto_examples/feature_selection/plo t_select_from_model_diabetes.html#sphx-glr-auto-examples-featureselection-plot-select-from-model-diabetes-py).The features with higher absolute coefficient values are considered more important,and features are considered unimportant and removed if the corresponding importance of the feature values is below the provided threshold parameter,which is set here to the median of the coefficients.Then,we select the 10 most important features according to the coefficients as the input at different stations.On this basis,multimodel ensemble forecast models are constructed by candidate machine learning methods,including Random Forest for regression,Gradient Boosting for regression,and Multi-layer Perceptron regressor.To implement the machine learning algorithms,this study uses the software package Scikit-learn developed by Pedregosa et al.(2011),with default parameter selection (software website:https://scikit-learn.org/stable/).The optimum one that has the smallest MAE among the three models at each station is selected as the final ensemble method.That is,the algorithms used for the ensemble forecast at different stations may differ.

The combination of forecast and observed data is used to construct the original dataset,covering the period of 1 January 2021 to 31 August 2022,70% of which are used for training while the rest are used for testing and evaluation.The above process of constructing multimodel ensemble methods is completed based on the training dataset.A completely independent testing dataset is used to evaluate the skill of the multimodel ensemble methods.In this paper,the evaluation of 10-m wind speed forecasts is mainly carried out using the MAE,root-meansquare error(RMSE),and correlation coefficient (CC).

3.Results

3.1.Wind speed forecast performance of different single models

Fig.1 shows the DAY1 forecast performance of different models based on the original dataset.In terms of MAE,most stations have values between 0.5 and 2 m s-1.The number of stations with an MAE of less than 1 m s-1for EC0.1,EC0.125,JMA,GFS,and CMA is 28,27,49,20,and 17,respectively.The stations with an MAE of more than 1 m s-1are mainly located in Mongolia,Jilin,Liaoning,Henan,Anhui,and Hubei.The RMSE at most stations ranges from 1 to 2 m s-1.The number of stations with an RMSE of less than 1.5 m s-1for EC0.1,EC0.125,JMA,GFS,and CMA is 48,51,60,32,and 36,respectively.The stations with an RMSE of more than 1.5 m s-1are mainly located in Northeast China,including eastern Mongolia,Jilin,and Liaoning.The CC of EC0.1,EC0.125,JMA,and GFS at most stations is greater than 0.6,while that of Gansu,Fujian,Chongqing,and Guizhou is less than or equal to 0.6.Compared to the above four models,the CMA model has fewer stations with a CC of more than 0.6.As the lead time increases,the forecast performances are significantly reduced,with a decreased CC and increased RMSE and MAE at almost all stations(figure omitted).

Fig.1.The MAE(left-hand column;units:m s-1),RMSE(middle column;units:m s-1)and CC(right-hand column)of 10-m wind speed forecasts by different models.Rows 1 to 5 are for EC0.1,EC0.125,JMA,GFS,and CMA,respectively.

The averaged results of all stations are shown in Table 1.The DAY1 MAE (RMSE,CC) of each station is calculated and then averaged,and then the same is done for DAY2 to DAY4.EC0.1,EC0.125,and JMA perform better than GFS and CMA.EC0.1 and EC0.125 show almost no difference.The RMSE and MAE of the JMA model are smaller than those of the EC models,although the CC is slightly lower.This result is consistent with Gong et al.(2015),who demonstrated that the surface wind forecast results of JMA have smaller errors than those of EC and GFS in eastern China.Therefore,JMA is selected as the reference model in the following evaluation.The following analysis is based on the testing dataset.

Table 1 10-m wind speed forecast evaluation using the original dataset.

3.2.Wind speed forecast performance based on multimodel ensemble methods

3.2.1.Forecast performance at different forecast lead times

The spatial distribution of the percentage decrease in the MAE with the RR method relative to the JMA model is shown in Fig.2(a).The MAE decreases at all stations.The rate of reduction on DAY1 is 0–40% for most stations,and the number of stations with MAE reduction rates of 0–20%and 20%–40%is similar.The rates of reduction are 40%–60%for some stations in Mongolia,Ningxia,Beijing,and Yunnan,and more than 60% at the individual stations of Guizhou and Shanghai.The spatial distributions of the percentage decrease in MAE for DAY 2 to DAY4 are similar to that of DAY1 but with a relatively weaker intensity.

For the MLO method (Fig.2(b)),the MAE decreases at all stations,and the improvement in forecast skill is more evident than with RR.More than 99%of stations have an MAE reduction rate of over 20%.In areas with poor forecast skill,such as Mongolia,Gansu,and Ningxia,the forecast skill is improved by more than 40%.Stations with a more than 40% MAE reduction rate are twice as many than with RR.The spatial distributions of the percentage decrease in MAE are similar for different forecast lead times.Also,it is worth mentioning that the preferred machine leaning method for most stations is Gradient Boosting for regression,while the rest adopt the Multi-layer Perceptron regressor.

Table 2 shows the overall wind forecast skill of the multimodel ensemble method at 100 stations.The same as in Table 1,the result is calculated at each station and then averaged.The MAE of the JMA forecast is greater than 1.1 m s-1for DAY1 to DAY4.The ensemble forecast with RR reduces the MAE to 0.85–0.92 m s-1,which is about 27%lower than for JMA.The MAE of the ensemble forecast with MLO is less than or equal to 0.75 m s-1,which is 40.5%–42%lower than that of JMA.In terms of RMSE,the averaged RMSE of the JMA forecast is greater than 1.5 m s-1.The ensemble forecast with RR reduces the RMSE to 1.1–1.22 m s-1,which is about 27% lower than JMA.For the MLO results,the RMSE is less than or equal to 1.01 m s-1,which is 39.1%,39.6%,39.4%,and 39.2% lower than that of JMA for DAY1 to DAY4,respectively.The CC improvement rates of the ensemble forecast with RR compared with JMA are 15.8%–28.3%,while those with MLO are 36.8%,44.4%,49%,and 60.9% for DAY1 to DAY4,respectively.

In general,the multimodel ensemble methods can significantly improve the wind speed forecast performance.In addition,the performance with MLO is better than with RR.

3.2.2.Forecast performance in different months

Fig.3 shows the monthly variation of the forecast performance.The monthly MAE of each station is calculated first,and then the 100 stations are averaged.Comparing the JMA forecast results with observations,the MAE ranges from 1 to 1.5 m s-1,with a maximum of 1.39 m s-1in April and minimum of 1.02 m s-1in October.The RMSE,meanwhile,is 1.2–1.5 m s-1in January and July to October,and 1.5–2.0 m s-1in other months,among which the maximum is 1.78 m s-1in April and the minimum is 1.27 m s-1in October.

The multimodel ensemble methods significantly improve the forecasting skill compared to JMA.For ensemble forecasting with RR,the RMSE and MAE are reduced by more than 25%in all months,but with particularly good reduction rates of 28.2% in September and 28.6% in April.For ensemble forecasting with MLO,the RMSE and MAE are reduced by more than 36% in all months,with the largest reduction rates being 41.6% and 44.2% in November,respectively.

3.2.3.Forecast performance for gale weather processes

To present a direct comparison among observations,the JMA forecast,and the multimodel ensemble forecasts with RR and MLO,we provide an operational forecast case study—namely,a gale weather process that took place on 21 April 2022 at Foyeding,Beijing(Fig.4).On that day,Beijing experienced its strongest winds along with sandstorms since 2022,which was reported by multiple media outlets.Both the JMA and ensemble forecast with RR could not reasonably forecast the wind speed’s temporal variation.However,the multimodel ensemble forecast with MLO significantly improved the forecasting performance,with a very high CC of 0.90.The MAE was 0.63 m s-1,which was 79%and 53%lower than that of JMA and the ensemble forecast with RR,respectively;and the RMSE was 0.88 m s-1,which was 74%and 49%lower than that of JMA and RR.

Fig.4.Observed wind speed and corresponding forecast results during a gale weather process on 21 April 2022 at Foyeding (station number: 54410),Yanqing,Beijing.

4.Discussion and conclusion

This study evaluated the performances of the 10-m wind forecasts of individual numerical models,including two EC models,GFS,JMA,and CMA.On this basis,two multimodel ensemble methods,RR and MLO,were proposed for 10-m wind speed forecasting bias correction.The results can be summarized as follows.

In terms of the MAE and RMSE,JMA was found to be more skillful than the other models in eastern China.Compared to JMA,the ensemble forecast methods of MLO and RR both showed an overall improvement with respect to wind forecast skill,with decreased MAEs at all stations and in each month.However,MLO performed better than RR.With the MLO method,more than 99%of stations had an MAE reduction rate of over 20%for DAY1,and the distributions of the reduction rate for DAY2 to DAY4 were similar to that of DAY1.For an average of 100 stations,the MAE of the ensemble forecast was 40.5%–42%lower than that of JMA for DAY1 to DAY4.During the annual cycle,the MAE was reduced by more than 36% in all months,with the largest reduction rate of 44.2%being in November.

In our study,the RMSE of the results with MLO was 0.92 m s-1for DAY1,which was 39% lower than that of JMA (1.51 m s-1) and 41%lower than that of EC0.1(1.57 m s-1),which is similar to the findings of Wu et al.(2022).In their study,a multimodel ensemble method based on an augmented complex extended Kalman filter reduced EC’s 10-m wind speed forecast RMSE by 35%–45% in most of East China and surrounding areas (Fig.3(i) in Wu et al.,2022).By comparison,Han et al.(2021)showed that the deep learning method could decrease the 10-m wind speed forecast RMSE by around 23% (Fig.8 in Han et al.,2021) compared to EC.It should be noted that only one EC model was used by Han et al.(2021).The above results demonstrate the importance of adopting multiple numerical models.In addition,as mentioned earlier,improving wind speed forecasts at lead times of 96 h is urgently needed by the power industry.Fortunately,the MLO-based multimodel forecasting method proposed in this study is reliable,with a forecast MAE of 0.74 m s-1and a CC of 0.74 for DAY4.

To give more insight into which kind of error is reduced significantly by an MLO-based ensemble,we conducted error decomposition as in Lyu et al.(2022).The RMSE was decomposed into the bias element(BIAS),the distribution element (DIST),and the sequence element(SEQU).Among them,BIAS characterizes the ability of the forecast to reproduce the average characteristics of the observations,DIST characterizes the error due to the difference in data distribution between the forecasts and the observations,and SEQU represents the error due to the forecast being ahead of(or lagging behind)the observations.From Fig.S2,the main source of errors for 10-m wind forecasts is the sequence component(SEQU),which rises rapidly with increasing lead times.The MLO method tends to produce lower SEQU and BIAS than all the single models.However,MLO showed a slight deficiency in reducing the distribution component of forecast errors when compared to EC and JMA,indicating future directions for improvement.

The MLO-based multimodel ensemble method proposed in this study has high skill in forecasting short-term 10-m wind speed,including forecasting at a lead time of four days,which is meaningful for the wind power industry.Thus,the MLO method is promising and deserves more attention in future research.Considering that the machine learning method in this paper uses default parameters,incorporating more features and optimizing parameters in future research is also needed and would be valuable.

Funding

This research was supported by the Beijing Natural Science Foundation [grant number 8234068].

Supplementary materials

Supplementary material associated with this article can be found,in the online version,at doi:10.1016/j.aosl.2023.100402.