Classification of clustered microseismic events in a coal mine using machine learning

Yi Dun, Yirn Shen, Ismet Cnult, Xun Luo, Gungyo Si,**

a CSRIO Mineral Resources, Brisbane, QLD, 4069, Australia

b School of Minerals and Energy Resources Engineering, University of New South Wales, Sydney, NSW, 2052, Australia

c School of Software, Shandong University, Jinan, Shandong, 250100, China

Keywords:Seismic event classification Clustered seismicity Machine learning Cascaded workflow Underground mining

ABSTRACT Discrimination of seismicity distributed in different areas is essential for reliable seismic risk assessment in mines. Although machine learning has been widely applied in seismic data processing, feasibility and reliability of applying this technique to classify spatially clustered seismic events in underground mines are yet to be investigated. In this research, two groups of seismic events with a minimum local magnitude (ML) of -3 were observed in an underground coal mine. They were respectively located around a dyke and the longwall face. Additionally, two types of undesired signals were also recorded.Four machine learning methods, i.e. random forest (RF), support vector machine (SVM), deep convolutional neural network (DCNN), and residual neural network (ResNN), were used for classifying these signals. The results obtained based on a primary dataset showed that these seismic events could be classified with at least 91%accuracy.The DCNN using seismogram images as the inputs reached the best performance with more than 94% accuracy. As mining is a dynamic progress which could change the characteristics of seismic signals,the temporal variance in the prediction performance of DCNN was also investigated to assess the reliability of this classifier during mining. A cascaded workflow consisting of database update, model training, signal prediction, and results review was established. By progressively calibrating the DCNN model, it achieved up to 99% prediction accuracy. The results demonstrated that machine learning is a reliable tool for the automatic discrimination of spatially clustered seismicity in underground mining.

1. Introduction

The process of coal mining changes the stress distributions in the rock mass around the excavations and can pose severe risks to mine personnel and production. Seismic monitoring has been widely used for understanding the stress redistribution and energy release around potential hazard zones for risk assessment and forecasting(Cai et al.,2019;Si et al.,2015,2020).During the seismic monitoring at the Upper Silesian Coal Basin in Poland, two classes of seismicity were recorded, and they were located respectively around the mining faces and regional geological structures(Gibowicz and Kijko, 1994; Lasocki and Idziak, 1998; Stec, 2007).The study of seismicity and associated source mechanisms demonstrated distinct relationships between the local geological structures and the mining conditions created by different mining methods. Due to limited seismic monitoring network coverage,Wiejacz and Lugowski(1997)focused on seismic events located at or near the Arkona fault at Wujek coal mine, Poland. It was found that, even though the fault was distant from the longwall panel, it could still contribute to mining-induced seismicity. Shen et al.(2020) conducted a comprehensive monitoring program around a major dyke at an Australian coal mine.The results showed that the dyke caused a significant change in the stress and seismicity in its proximity,increasing the level of coal burst risk.The studies above showed that seismic events tend to cluster spatially around the mining face and geological structures.These events delineate areas within the rocks that experience failures, representing similar seismic characteristics (Anderson and Spottiswoode, 2001;Hudyma,2008;Potvin,2009).Cluster analysis is a routine task for seismic risk assessment in mines as seismic hazards may be underor over-estimated without classifying seismic event clusters(Le′sniak and Isakow, 2009).

Owing to the recent advances in artificial intelligence,machine learning has become the state-of-the-art technique in seismic signal processing. For earthquake detection, Meier et al. (2019)developed and compared five machine learning methods,including the dense neural network, recurrent neural network(RNN), RNN with attention, deep convolutional neural network(DCNN), and generative adversarial network with random forest(GANRF).The performance of different methods was benchmarked with a linear signal/noise discriminator, which was based on two features extracted from the waveform. They used 3 s-long waveform snippets to train DCNN and GANRF.The results demonstrated that all machine learning methods outperform the linear classifier.The results revealed that DCNN and GANRF,which use raw data as input, perform significantly better than other machine learning classifiers trained with extracted features.A hybrid neural network that combines the convolutional neural network(CNN)and RNN in a residual structure was developed by Mousavi et al. (2019) for earthquake signal recognition. It aimed to discriminate noise from earthquakes by learning the time-frequency characteristics of the three component signals recorded at individual stations. It was found that the model was capable of generalising large datasets and could perform reliably in real-time monitoring of earthquakes.Perol et al.(2018)developed a CNN model for earthquake detection and location.Seven classes of seismic data,including six clusters of earthquakes and noise, were labelled in this study. The data streams recorded by two three-component stations were used as inputs.The results showed that the detection and location accuracy reached more than 94% and 74%, respectively.

In mine seismology, Wilkins et al. (2020) demonstrated that CNN outperformed manual identification of microseismic events and noise signals. It is feasible to implement this method for automatic seismic event detection. In order to classify blasts and seismic events, Vallejos and McKinnon (2013) and Dong et al.(2016) selected characteristic seismic parameters obtained from the full-waveform systems. Logistic regression was applied to evaluating discriminator models based on these parameters. For datasets acquired from different mine sites, it was found that certain common parameters contribute significantly to the classification of seismic events. In addition, Vallejos and McKinnon(2013) suggested that both classifiers require further calibration as the mining condition changes to reliably predict new seismic records.More recently,Pu et al.(2020)compared the classification performance of ten widely used machine learning classifiers to discriminate microseismic events and blasts in underground excavations. They also found that the logistic regression outperformed other methods in distinguishing blasts and seismic events. Peng et al. (2020a) explored the applicability of machine learning to classifying microseismic events, blasting, excavation,mechanical noise, and electronic signals. Features were extracted from seismic data snippets and selected based on their significance as input to the classifiers.They demonstrated that DCNN performed significantly better than other machine learning methods, and it can achieve more than 98%accuracy for classifying complex seismic signals. Furthermore, they proposed a capsule network (CapsNet)(Peng et al., 2020b). Even with limited training samples, CapsNet can still reach 99.2% accuracy for similar tasks.

The studies above demonstrated that machine learning can effectively discriminate seismic events,blasts,and noise.However,there is hardly any discussion on the discrimination of spatially clustered seismic events.Meanwhile,the characteristics of seismic signals can significantly change as mining is a dynamic progress.The temporal changes in the classification accuracy of machine learning need to be investigated. A work-frame needs to be established for routine re-calibration of machine learning model and database to ensure the prediction reliability during mining.This research aims to address the above issues.

In this paper, the feasibility of applying machine learning to classifying seismic events distributed around a major dyke and the longwall face in an underground coal mine was investigated. Four classifiers, i.e. random forest (RF), support vector machine (SVM),DCNN,and residual neural network(ResNN),were tested based on a primary dataset.Their classification performance was compared.All classifiers reached at least 91% accuracy, and DCNN outperformed other methods with more than 94%accuracy.Then,realtime monitoring was simulated, and a cascaded workflow was established to assess the temporal variance in the prediction performance of DCNN. The classified seismic events were located to verify the accuracy of the results. It showed that by progressively calibrating the model, machine learning can reliably discriminate spatially clustered seismic events in mines.

2. Project background

2.1. Terminology

As there is no universally accepted definition of terms used to refer to dynamic rock failure events, the following terms (Galvin,2016; Hebblewhite and Galvin,2017) are adopted in this paper:

(1) Microseismic event: Any intact rock failure or rock movement (e.g. shearing displacement along a major discontinuity)that causes a measurable seismic signal.The minimum local magnitude(ML)of microseismic events recorded in this project is -3.

(2) Pressure bump: A lower level of dynamic energy release(compared to a pressure burst)within the rock(or coal)mass in a coal mine and does not involve intact rock failure at the surface of an excavation and associated expulsion of the failed material.

(3) Coal burst: A release of stored strain energy that causes dynamic failure or displacement of intact coal, resulting in high-velocity expulsion of this broken/failed/displaced material into the mine opening.

2.2. Monitoring site and network configuration

There has been an increasing awareness and research attention to the problem of coal bursts since 2014 in Australia(Hebblewhite and Galvin,2017).A microseismic monitoring project,supported by the Australia Coal Association Research Program (ACARP), was carried out in a longwall coal mine in Australia.The longwall panel selected for monitoring at the mine site was 400 m in width and 3000 m in length. It had a competent conglomerate roof with a thickness of approximately 15-20 m. The mining depth was approximately 280 m below the ground surface.A roadway section between cut-through 10(10ct)and 9ct intersected by a mafic dyke(see Fig.1)was regarded by the mine as a coal burst prone area.The length of the dyke was more than 800 m, and it was across the monitored longwall panel.The thickness of the dyke was 0.5-3 m with an approximate dipping angle of 75°-80°towards the south.The terms “inbye” and “outbye” refer to the two sides of the dyke towards the north and south, respectively. The microseismic monitoring network consisted of four triaxial geophones, which were installed close to this dyke on the maingate side of the longwall panel.Two geophones were installed on inbye of the dyke and the other two were deployed on outbye of the dyke. On both inbye and outbye of the dyke,one vertical geophone was installed approximately 10 m into the roof and one horizontal geophone was installed 5 m into the coal seam. All geophone boreholes were cemented to ensure good coupling with the rock mass. The mechanical properties and velocity of P-wave(Vp)of strata in this area are listed in Table 1.

Fig.1. (a)Plan view of the longwall panel,and(b)Cross-section of the roadway intersected by the dyke.In(a),the dyke-roadway area is indicated by a rectangle in green.The start and end positions of the longwall face with respect to the period for data processing are plotted in blue and orange lines, respectively.

2.3. Seismic dataset

Microseismic monitoring at the coal mine commenced on 26 November 2017 and concluded on 31 July 2018. In this research,seismic data recorded from 26 November 2017 to 23 March 2018 were processed to assess the seismicity within the dyke-roadway area as the longwall mined through the dyke during this period.For preliminary seismic data processing, an empirical amplitude threshold was applied to detecting triggered events.Each triggered signal was segmented to a 512 ms long window consisting of 128 ms pre-trigger and 384 ms after-trigger data. There was no time overlapping between two consecutive triggered segmentations.The following four main types of triggers were identified and defined for classification.

2.3.1. Dyke-roadway events

Major geological discontinuities are key factors which can lead to coal bursts. When intersected by excavations, the geological features usually result in high stress concentration and instability.In this project,the dyke was regarded as a major factor contributing to stress re-distribution in its vicinity (Shen et al., 2020). Seismic events that occurred within the dyke-roadway intersection area were observed even when the longwall face was 400 m ahead of the dyke. These seismic events exhibited similar characteristics:they were associated with shear fractures and the dominant frequency domain of these events ranged from 25 Hz to 150 Hz; the duration of the signal was less than 100 ms and the measured maximum peak particle velocity(PPV)was approximately 10-5m/s. A three-dimensional (3D) velocity model was developed for locating these seismic events. As these events were close to the monitoring network, most of these events can be reliably located.

2.3.2. Longwall events

Seismic events tend to be located close to the longwall face because of failures caused by mining-induced abutment stresses in longwall mining (Luo et al., 2001). Seismicity associated with the rock failures around the longwall face was also registered in this coal mine. When the longwall face was 400 m away from the monitoring network, as the energy of caving events was typically large, the geophones were still able to record the seismic signals.Most of them exhibited clear P-and S-wave signals.As the longwall face was approaching the seismic monitoring network, the duration of seismic signal and time difference between the P- and Swaves reduced as the distances between the sources and the geophones decreased. The locations of these events can be estimated by the time difference between P-and S-wave arrivals and the raypaths derived from the waveforms recorded by the triaxial geophones. As the longwall face position can be obtained from the daily mining reports, seismic events located around the longwall face were categorised as longwall events.

Table 1 Parameters of layers and geological structure in the model.

2.3.3. Low-frequency signal

In addition to the dyke-roadway and longwall events,some lowfrequency signals were also detected during monitoring. These signals were related to two sources. One source was the remote blasts from the nearby open pit mines and further caving of previously mined longwall panels.The other source was the coda wave of some strong seismic events as the duration of segmented signals was not long enough,and the energy of such events took more time to attenuate.The duration of these signals was typically longer than the length of segmented snippets, which could lead to false detection of triggers. As the amplitude of these signals and the dominant frequency domain of these events were relatively low(1-10 Hz), they were labelled as a separate class.

2.3.4. Noise

Underground activities, such as drilling, coal crushing, water pumping,ventilation,and traffic can induce noise that needs to be removed for reliable seismic assessment. The noise level is closely related to the mining practice and machinery vibrations near the microseismic monitoring network. In general, the noise signals have repeated patterns and stronger background amplitude than normal ambient noise. They are generally related to constant machinery vibrations and can last for minutes. They have no clear waveform signatures of P- and S-waves associated with rock fractures.In this study,all triggered signals except for the three classes mentioned above were labelled as noise.

3. Machine learning methods

To investigate the feasibility of classifying spatially clustered seismic events and select an optimum classifier,four classic machine learning methods,including RF,SVM,DCNN,and ResNN,were tested for comparison.RF and SVM used the features extracted from seismic data of six channels as inputs, whereas images of the same six channel seismograms were adopted in DCNN and ResNN.

3.1. RF

RF is an ensemble of decision trees constructed by random and uncorrelated features (Breiman, 2001). The less the correlation between individual trees, the better the model can generalise the data.Each decision tree of RF is split to training and testing sets so that the significance of each feature can be obtained. Due to its randomness and uncorrelated structure,RF is relatively robust and can generally perform well on classification. The number of decision trees is the main parameter to be considered in RF and it contributes to the performance as well as its computational efficiency (Oshiro et al., 2012). By increasing the number of the decision trees,more randomness such as feature subspaces can be introduced (Ho, 1998). It can provide the branch weights which indicate the importance of a feature (Breiman, 2001). These characteristics make RF a widely used method for feature-based classification.In this research,the number of decision trees is tuned for performance optimisation.

Fig. 2. Illustration of a residual block.

3.2. SVM

SVM was originally designed for binary classification (Boser et al.,1992) and has been implemented to solve multi-class problems (Li et al., 2005). It considers the feature points as support vectors and constructs an optimal hyperplane in multi-dimensional space to separate different classes. For a nonlinear problem, SVM uses a kernel technique to transform an input space to a higher dimensional space so that the support vectors can be separable(Chamasemani and Singh,2011).There are four types of commonly used kernels, i.e. linear kernel, polynomial kernel, sigmoid kernel,and Gaussian radial basis function(RBF).In this paper,we selected RBF and the optimisation problem can be expressed as

where xiand xjdenote two feature vectors,xi-x2jis recognised as the squared Euclidean distance between two feature vectors,σ is a free parameter,γ is a hyperparameter which controls the extension of curvature in determining a decision boundary.Meanwhile,there is another parameter C which denotes the punishment degree of the classifier. The optimisation problem is expressed as

The kernel ? is used to transform the input data into the feature space. Parameters γ and C were fine-tuned to optimise the SVM classifier.

3.3. DCNN

DCNN is recognised as a state-of-the-art machine learning method for image recognition (Rawat and Wang, 2017). A typical DCNN consists of multiple convolutional layers and fully connected layers. As its name suggests, the convolutional layer involves iterative two-dimensional (2D) convolution operations. The convolution operation facilitates the trained kernel filters with a relatively small receptive field to traverse through the whole input image(Lecun et al.,1998).It is ideal for extracting local-sensitive features from images,thus it is one of the typical choices when designing a deep neural network for a vision-related task. Each convolution operation is the dot product between the weights of the filter and the input values within the receptive field. The convolution operation can be expressed as

Fig. 4. Image examples of four classes of seismic signals.

where x is the input,W is a trainable kernel,and b is an optimisable bias parameter.

The dense-connected layers learn the optimal representation of features for distinguishing different classes. Different from the convolutional layer, each output of the fully connected layer has connections to all the input neurons. It can be treated as a filter with a receptive field that is the same size as the inputs. Fully connected layers are normally used as the last few layers of a deep learning model.In CNNs,it connects the local features extracted by convolution operations to the output layer, which classifies each input image into its category. Because of its connection structure,the fully connected layer contains a significantly large number of weights and accounts for a large portion of computation.It was first demonstrated in 2012 that DCNN outperformed the handengineered feature learning methods on image classification(Krizhevsky et al., 2012).

3.4. ResNN

ResNN is a DCNN model with residual blocks (ResBlock) which aims to enhance the generalisation capability of neural networks when the layers go deep (He et al., 2016). As shown in Fig. 2, a typical ResBlock consists of two layers of convolution operations followed by activation layers.A skip connection is used to provide a shortcut from the input of the first convolution layer and the output of the second convolution layer.As evaluated in previous work,the ResBlock can avoid the problem of gradient diffusion.In our paper,the structure of ResNN is similar to that of DCNN but with ResBlock instead of convolutional layers.

4. Classification performance evaluation and comparison

4.1. Input preparation

The first test of this research is to comparatively analyse the classification performance of selected machine learning methods.This is essential for investigating the feasibility of applying machine learning in geotechnical engineering (Zhang et al., 2021a). During the monitoring period, a total of 85,614 triggers were detected.Seismic events associated with rock fractures were manually identified and located,and it was found that they mainly clustered around the longwall face and within the dyke-roadway area.In this test, two thousand seismic events located in each area were selected and labelled as dyke-roadway and longwall events,respectively. To avoid the unbalanced samples for training and testing, two thousand low frequency and noise signals were also discriminated and categorised.In total,eight thousand signals(two thousand for each class)were manually labelled as ground truth to establish a primary database. Two formations of inputs were derived from the primary database for this test.

4.1.1. Features

The first type of inputs was signal feature extracted from the time series, and was used for RF and SVM. To avoid secondary parameter tuning process and potential bias in computing the features, 32 statistical features were extracted from the raw signal data of each channel.The seismic data recorded by geophones UG1 and UG4 were selected for feature extraction. As each geophone had three channels, a 32 × 6 dimension of feature tensor was extracted. The details of these features are given in Appendix (see Table A1). The feature relevance was computed using a Python package ‘tsfresh’ based on the Benjamini Hochberg procedure(Haynes,2013;Christ et al.,2018).The lower the p-value,the more significant the feature.

To distinguish the seismic signals, the distributions of feature values were analysed.As shown in Fig.3,the most distinguishable feature of dyke-roadway events is the location of maximum amplitude. As the longwall mined about 600 m and the samples were selected from the entire monitoring period to form the primary dataset, no dominant characteristic was exhibited. The lowfrequency events have the lowest energy and minimum variance in amplitudes,whereas the noise signals exhibit unique features in the longest strikes above/below the mean, average and median amplitudes as they are normally induced by constant machinery vibrations. As summarised in Table A2, multiple classes are involved and each class of signals has its specific features. All features can contribute to the classification and are selected for training the classifiers.

4.1.2. Images

The second type of inputs is seismogram image. All triggers were plotted to 256 × 256 pixels bitmaps in grayscale. The traces were normalised so that each one was plotted in a rectangle of 256 × 42 pixels, and there was no overlap plotting area between adjacent traces. As images were prepared for DCNN and ResNN,which can automate the feature extraction and significance evaluation, no feature evaluation was further conducted on these images. Four image examples of each class are shown in Fig. 4. Each image shows the waveform recorded by the geophones UG1 and UG4.

4.2. Performance evaluation method

For a binary classification task,there are four possible outcomes:(1)true positive(TP),(2)false positive(FP),(3)true negative(TN),and(4)false negative(FN).The metrics,i.e.precision(P),recall(R),F1 score (S) and accuracy (Acc) are used to assess the classifier performance (Fawcett, 2006) by

Table 2 Confusion matrix for evaluating the classifier performance.

Table 3 Level of agreement and associated к statistics.

In this study,there were four types of signals to be classified.The predicted results need to be binarised to use this technique to evaluate the classifier performance. As shown in Table 2, a 4 × 4 confusion matrix was used to demonstrate the results obtained by different classifiers. The formulae to compute the performance metrics are provided below. The precision (P), recall (R), and F1 score(S)are respectively denoted by Pi,Riand Si,where i=1,2,3,or 4, indicating dyke-roadway, longwall, low-frequency, and noise,respectively. The overall accuracy is denoted as Acci.

In addition to the metrics above, Cohen’s kappa coefficient (к)(Cohen, 1960; Artstein and Poesio, 2008) was also evaluated to ensure the reliability of the classification accuracy. This method excludes the probability that a sample is classified by chance and can provide a more robust reliability assessment. It has been used for performance evaluation of machine learning applications in geotechnical engineering (Zhou et al., 2015, 2016). The kappa coefficient (к) can be given by

where N is the number of samples.A qualitative scale was proposed by Artstein and Poesio (2008) to indicate the level of agreement.The details are listed in Table 3.

4.3. Model architecture and hyperparameters tuning

The optimisation of hyperparameters is critical for ensuring classification accuracy.It requires systematic analysis and a range of optimisers which can be applied to the process(Zhang et al.,2021b;Zhou et al., 2021). To select the optimum classifier, the hyperparameters of RF, SVM, DCNN, and ResNN were fine-tuned by applying grid-search in comparative analysis. The primary dataset was split to 80%and 20%, which were used as training and testing data respectively for tuning the hyperparameters of all classifiers.In the tuning process,the performance was evaluated based on the classification accuracy calculated by Eq. (13). The reliability of the optimised model was further verified via ten-fold cross-validation(CV) (Refaeilzadeh et al., 2009). In the CV test, both classification accuracy and Cohen’s kappa were calculated for more reliable evaluation.

As shown in Fig.5a,a relationship between validation accuracy and the number of decision trees was obtained.For this dataset,the number of decision trees was tuned to 200 so that the RF classifier can achieve its optimum accuracy of 94.5%.In this study,the widely used RBF kernel(Chang et al.,2010) was selected to train the SVM classifier. In order to reduce the time to find support vectors, the features were scaled before being fed into the SVM classifier. We then tuned the two parameters for the RBF kernel, i.e. C and γ, to optimise the classifier. The classification accuracy with respect to different configurations of C and γ was compared. The results are shown in Fig.5b,which indicates that the best validation accuracy is 93%.The combination of C=1000 and γ=0.001 was selected as the optimum hyperparameters of SVM.

In comparison with the feature-based machine learning methods, DCNN has been selected in this study to test whether it can classify seismic signals more efficiently and reliably. The underlying architecture for the DCNN model was established first. It had four convolutional layers and two fully connected layers.Each convolutional layer was followed by a max-pooling layer to reduce the number of parameters to alleviate the issue of overfitting while retaining the main features of the image. The kernel size of the convolutional and max-pooling layers was 2 × 2 with the stride being one. Rectified linear unit (ReLu) (Hahnioser et al., 2000;Jarrett et al.,2009;Brown et al.,2017)was selected as the activation function of each convolutional layer. Considering the computational efficiency and memory usage when dealing with large size dataset, Adam optimiser (Kingma and Ba, 2015) was chosen for gradient-based training optimisation and categorical cross-entropy loss function was applied in evaluating the cost.

The number of filters associated with each convolutional layer,number of units and activation function used in the first dense connected layer, and the learning rate were tuned by random search.In total,100 trials were conducted,and each trial configured an individual model to train for ten epochs. Overall, 34 trials achieved more than 95% validation accuracy (Fig. 6). The optimum DCNN model reached the best validation accuracy of 98.6%.In this model,the number of filters of the four convolutional layers was 32,64,64,and 128, respectively. In the first fully connected layer,640 nodes and ‘tanh’ activation function were adopted. The tuned hyperparameters and the architecture of the model are shown in Fig. 7.

The fourth classifier ResNN was developed based on the DCNN model by replacing the last two convolutional layers with a Res-Block.The DCNN model was regarded as a benchmark to test if the ResNN model can improve performance. In order to construct the ResBlock, the number of filters of the ResBlock was configured to 64, which was the same as the number of filters at the second convolutional layer. The kernel size and strides adopted for the ResBlock were 2×2 and 1,respectively.The output of ResBlock was passed to an average pooling layer to reduce the complexity of the features. The kernel size of the pooling layer was 4 × 4. Then the first dense layer was connected to the average pooling layer.Interestingly, ‘tanh’ activation function was firstly adopted for this dense layer as the same with the DCNN model,but the model was unable to converge.As a result,we changed the activation function of this dense layer to ReLU, which performed well. The other hyperparameters of ResNN remained the same as the DCNN model.The structure of the model is shown in Fig. 8. The validation accuracy of ResNN was 94.5%.

4.4. Performance comparison

One of the key elements of training a classifier model is to avoid overfitting or underfitting. Adequate samples are required to train the model to recognise the underlying trends in the data. It also needs a reasonable ratio of test samples to reduce the variance error while testing the effectiveness of the model. To achieve a balance in training and testing,we changed the training and testing dataset distribution using the k-fold CV (Refaeilzadeh et al., 2009)to further examine the classification performance of the optimised classifier models.In this research,the data were randomly split into ten groups.The models were fit using nine groups of the split data as training samples, and one as test dataset. The training and testing were iterated for ten times to ensure the models performed stably while taking full advantage of the existing dataset.

The accuracy and Cohen’s kappa of these four classifiers are shown in Fig.9.All four classifiers achieved excellent performance with more than 91% accuracy. Specifically, DCNN reached more than 94%accuracy with the most stable performance.RF,SVM,and ResNN,respectively reached at least 93%,91%and 91%accuracy.RF and SVM performed relatively stable,whereas ResNN had the least stable performance. The Cohen’s kappa followed the same distribution. The DCNN had the highest value of к of more than 0.91,whilst RF, SVM and ResNN all obtained к of more than 0.89, indicating high levels of classification agreement.

The detailed classification accuracy and Cohen’s kappa of the ten-fold CV are listed in Tables 4 and 5. To further explore the classification reliability with respect to the types of seismic signals,the distributions of accuracy metrics are shown in Fig. 10 and summarised in Table 6. Among the four categories of seismic signals, all classifiers had the lowest accuracy in discriminating longwall events.The reason was that during the monitoring period,the characteristics of the longwall events varied drastically. The longwall events were induced by longwall mining, and they propagated as the longwall retreated. The energy attenuation reduced as the longwall approached the monitoring network.As a result, the seismic amplitude increased, and the time difference between P- and S-waves decreased. This posed difficulties for all classifiers to generalise the seismic signals associated with the longwall events. Meanwhile, the characteristics of low-frequency events remained the same, which was the reason why all classifiers had the best F1-score on identifying this type of signal.Interestingly, the noise was also discriminated with excellent accuracy.Most of those noise signals were in relation to the operation of the machinery and traffic near the geophones.As shown in Fig.3,it shows distinguishable features with symmetric distributions in the length of the longest consecutive subsequence in channel data that is larger or smaller than the mean of the channel data, indicating that the temporal changes in the signal characteristics were insignificant.

Fig. 5. Grid-search for tuning the hyperparameters of RF and SVM: (a) Accuracy with respect to the number of decision trees of RF, and (b) Accuracy relating to the combinations of C and γ of SVM.

5. Temporal prediction performance evaluation

During real-time microseismic monitoring in mines, the capability of any method to correctly predict the types of seismic events is vital. The characteristics of seismic signals vary as the mining proceeds. Therefore, instead of evaluating the classification performance of the classifiers using the complete seismic dataset, it is more important to assess if the machine learning method can reliably predict new seismic data as the mining condition changes over time. The primary test results in Section 4 demonstrated that DCNN performed better than the other three classifiers. Moreover, no extra computation time was required to prepare the images as input. As a result, DCNN was selected for evaluating the prediction accuracy in the near real-time microseismic monitoring simulation.

Fig.6. The process of DCNN hyperparameters tuning for achieving the best classification accuracy.The hyperparameters of the optimum model are indicated by the bold red line.

Fig. 7. Network architecture and hyperparameters of the optimum DCNN model.

Fig. 8. Network architecture and hyperparameters of the ResNN model.

Fig. 9. The box-and-whisker plot of the ten-fold CV performance of all classifiers: (a)Accuracy and (b) Cohen’s kappa.

Real-time data streaming was simulated using existing data,and an empirical amplitude threshold was applied to detecting a trigger.When a trigger was detected,the image and its associated seismic data were saved separately.The length of segmented data and image format were the same as described in Section 4.1.2.The simulation test included eight time periods(see Fig.11).Foreach period,a DCNN model was trained and applied to automatically classifying the seismic signals to be detected in the next period.The classification results were manually reviewed to evaluate the prediction accuracy.The reviewed results were added to the database for calibrating the DCNN model.Then the calibrated model was used for classifying the seismic signals recorded during the next period.The general flowchart is shown in Fig.12.By following this process,we obtained the prediction performance of the DCNN model over the entire monitoring period(see Fig.13).The performance metrics of the model of each period are summarised in Table 7. The cumulative number of seismic events located within the dyke-roadway area and the longwall face as the mining retreated was obtained(see Fig.14).The results show that the DCNN method can successfully generalise the captured data to achieve excellent prediction performance in realtime monitoring as the mining condition changed over time. This cascaded workflow, which includes manual reviewing, model training, prediction of new data, and validation of the predicted results, was proved to be efficient for the near real-time seismic event classification.The following describes the details involved in the simulation for each period:

(1) Period 1 (26 November to 31 December 2017). This period was allocated to accumulate seismic data and investigate if the seismic signals can be categorised for automatic classification. Seismic monitoring commenced on 26 November 2017,when the distance between the longwall face and dyke was 430 m.By December 31,2017,the longwall retreated by 207 m and was 223 m away from the dyke. During this period, few seismic events were captured, and most of the triggers detected were around the longwall face. Interestingly, even when the longwall face was about 400 m away from the dyke,seismic events were still recorded within the dyke-roadway area. Noise and low-frequency events were also detected during this period. All triggers were manually classified. Because the number of dyke-roadway events,noise signals and low-frequency events was much smaller than the number of longwall events.Only 100 seismic signals of each category were randomly selected to train the DCNN model to avoid bias caused by unbalanced samples.

(2) Period 2 (1 to 10 January 2018). The longwall retreated by only 5 m on 8 January,and 342 triggers were detected during this period. The DCNN model obtained from Period 1 was applied to classifying these triggers automatically. Then the results were manually reviewed to validate the prediction accuracy.The overall prediction accuracy and к were 86%and 0.66, respectively. The F1-score in classifying the dykeroadway events was only 74%. This was mainly related to poor precision, which was 60% because some longwall events were classified as dyke-roadway events. The final classified seismic signals were added to the database and the DCNN model was re-trained using the updated database.

(3) Period 3 (11 to 20 January 2018).From 15 to 20 January, the longwall retreated by 60 m, with a rate of 10 m per day. As longwall production accelerated over this period, the number of triggers started to increase significantly,especially that of the longwall events. Meanwhile, the background noise near the monitoring network also intensified.The amplitude threshold was not dynamically adjusted,thus a large number of noise signals were recorded.The seismicity near the dyke also increased, with 807 dyke events being captured. The DCNN model obtained in Period 2 performed well in predicting the triggers detected in this period.The precision and recall in predicting dyke-roadway events both exceeded 90%.The overall accuracy and к were also improved to 90% and 0.8, respectively.

(4) Period 4 (21 to 31 January 2018). During this period, as the longwall approached the dyke,the monitoring network was able to capture weak longwall events. As these events exhibited similar characteristics to that of the noise signals,the precision of longwall events and recall of noise signals both deteriorated. However, in the meantime, the DCNN classifier still performed well in predicting the dyke-roadway events.In this period,there were 3355 dyke-roadway events recorded and 94%of them were correctly discriminated.The overall accuracy and к slightly increased to 91% and 0.83,respectively.

(5) Period 5(1 to 10 February 2018).The DCNN model achieved the best overall prediction performance in this period.As the longwall approached the dyke, the stress around themonitoring location changed drastically. As a result, most of the seismic activities occurred within the dyke-roadway area rather than within the longwall panel. The accuracy and к reached 99% and 0.99, respectively.

Table 4 Classification accuracy obtained from the ten-fold CV tests.

Table 5 Cohen’s kappa obtained from the ten-fold CV tests.

Fig.10. The box-and-whisker plots for metrics of all classifiers. (a) RF, (b) SVM, (c) DCNN, and (d) ResNN.

Table 6 Average metrics of all classifiers obtained in CV (%).

Fig.11. Time periods associated with updating the DCNN model with respect to the mining progress.

Fig.12. Cascaded flowchart for evaluating the DCNN prediction performance.

(6) Period 6 (11 to 20 February 2018). The longwall mined through the dyke at the beginning of this period, when a number of pressure bumps occurred near the dyke.Then the longwall stopped mining for about 10 d.The pressure bumps and other seismic events near the dyke were correctly discriminated. Although the performance in predicting the longwall events was down to 92%, the DCNN classifier was still able to achieve 98% and 0.96 of accuracy and к,respectively.

(7) Period 7 (21 to 28 February 2018). The longwall production was restored on 26 February, causing a sudden increase of seismicity around the dyke. The longwall started to mine outbye of the dyke; the performance of the DCNN classifier remained stable during this period.

(8) Period 8 (1 to 23 March 2018). As the longwall retreated further away from the dyke, the daily number of seismic events that occurred near the dyke steadily decreased. The DCNN model was trained with an adequate dataset,allowing the classifier to be robust enough to accurately classify the rest of the detected triggers without regular calibration.

6. Discussion

Fig.13. The temporal changes of the DCNN model performance on predicting new seismic signals.

Table 7 Performance of DCNN models with respect to different time periods.

Fig.14. The cumulative number of four classes of seismic signals with respect to the longwall mining progress from 26 November 2017 to 23 March 2018.

Fig.15. Comparison of waveforms and spectra associated with four dyke-roadway events located in the (a) floor, (b) rib, (c) roof, and (d) dyke, respectively. These signals were recorded by the east component of UG1.

As the objective of this research is to classify spatially clustered seismic events, the seismic events were categorised based on where they were located. Seismic events can locate in the roof,floor,surrounding rocks,and the geological structures in each area and the seismic waveforms vary accordingly (Fig.15). The reason why machine learning can handle these variations and classify them as one group is because of their generalisation capability(Kawaguchi et al., 2017). Machine learning aims to abstract the training samples in a statistical manner to incorporate the dynamic changes of these statistical features. The key to ensuring that the machine learning model can generalise seismic signals under different conditions is to avoid underfitting and overfitting when training the models. In this research, we built sufficient dataset,fine-tuned the model and applied ten-fold CV to testing the model.The classification results demonstrated that the model can reliably discriminate seismic waveforms associated with rock fractures in the roof, floor, surrounding rocks, and geological structure in the specified areas.

To ensure the reliability of the hand-labelled ground truth and the classified results, the images and signals were manually reviewed and processed. The low-frequency and noise signals can be visually discriminated as their features were different from those of the seismic events.For the latter,as triaxial geophones can reliably register P- and S-waves, the distances from the source to the geophones were estimated based on the time difference between P- and S-arrivals. If the distance was less than 50 m, it was labelled as a dyke-roadway event, and vice versa. A 3D velocity model (see Fig. 1b) incorporating the strata was established to further locate the dyke-roadway event to validate the label accuracy. The location results are shown in Fig. 16, and the location errors of the dyke events are within 20 m.The seismic events were mainly distributed at the inbye side of the dyke and the immediate roof. It was consistent with the stress and displacement measurements (Shen et al., 2020), indicating that the dyke influenced the stress regime in its vicinity, and there was more stress concentration at the inbye side of the dyke. These results demonstrate that the dyke-roadway events were reliably classified.

Fig.16. (a) The contour of dyke-roadway event distributions in the plan view, and (b)The contour of dyke-roadway event distributions in cross-section. The red and cyan lines indicate the dyke and roadway layout, respectively.

On the other hand, P- and S-arrivals and the time difference between them were used for estimating the locations of longwall events(Feng et al.,2017).As shown in Fig.17,as the longwall mined towards the monitoring network, the time difference between Pand S-waves and the distance r between the seismic source and geophone decreased. Furthermore, the ray directions were also derived from the particle motion (Provost et al., 2017; Ding et al.,2018) to reduce the uncertainty of the estimated location.Because the longwall face positions can be obtained from the daily mining report, it can be determined whether these events were around the longwall face. The locations of strong longwall events recorded in each month during the monitoring period and the longwall face positions at the end of each month in the plan view are shown in Fig.18. The location errors in plan view range from 20 m to 100 m. As the longwall face approached the monitoring network, the location errors reduced.

The accuracy of event locations was further evaluated with the pressure bumps reported by the mine. Two pressure bumps were identified during the monitoring period, and their locations are shown in Fig.19.The computed locations of these pressure bumps were also provided. The location errors are subjected to the distances between the seismic events and geophones. The closer the seismic events are to the geophones, the more accurately the events can be located.

One challenge in this classification task was to discriminate the longwall events while the longwall was mining through the dyke.Some of the seismic events located around the longwall face were close to the dyke-roadway area,especially when the longwall was mining near the maingate. There were some events that can be categorised as both dyke-roadway and longwall events. Moreover,as shown in Fig.17b,as the longwall approached the geophones,the characteristics of longwall events changed.The more variances the signals have,the more difficult for the classifiers to generalise.This is why all classifiers have the lowest accuracy and reliability in discriminating longwall events.Hence,because the changes in the characteristics of the signals are closely related to the mining progress, the classifier model needs to be progressively calibrated to ensure its generalisation capability.

The prediction probability of seismic signals can be a quantitative approach to evaluate the confidence of the results and guide the data reviewing. For each seismic signal, the DCNN classifier generates four probabilities that the signal could be classified into four categories of seismic signals. The signal is classified into the category with the maximum probability, i.e. positive probability.The probabilities associated with other classes are defined as negative probabilities.The larger the positive probability,the more distinguishable this seismic signal. A seismic signal with low positive probability exhibits complex characteristics and could belong to another class. In this study, the relationships between the positive probability and the negative probabilities are depicted in Fig.20.As shown in Fig.20a and b,some dyke-roadway events are likely to be misclassified as longwall events and vice versa. This is mainly because when the longwall was mined through the dyke,the seismic events distributed within the longwall panel were also close to the dyke-roadway area.The characteristics of these seismic events were similar and can be difficult to distinguish without detailed analysis.However,based on the probabilities output from DCNN, we can selectively review those seismic signals classified with low positive probability to confirm if the predicted class is correct. This can significantly reduce the time and effort for manually validating the classification results,which will be the key to practice real-time seismic event discrimination.

Fig.17. (a)Locations of three longwall events recorded on different dates with respect to the longwall face positions.The black circle and radius r indicate the area where the source can be located and the distance between the source and the geophone. The triangle in blue indicates the range of the ray direction. (b) Associated seismic signals of the three longwall events recorded by UG1. The waveforms in east, north and vertical directions are indicated by red, green, and blue lines, respectively.

Fig. 18. The locations of longwall events (ML ≥-2) recorded each month and the corresponding longwall face positions in plan view.

Fig.19. Reported and calculated locations of two pressure bumps on mine plan view.

Both features and images of seismic signals were used to train the classifiers in the test.As it takes time to extract and evaluate the combinations of features for classification (Bardainne et al., 2006;K?hler et al., 2010; Mousavi et al., 2016), one objective of this research was to assess whether comparable classification accuracy can be achieved so that the feature extraction and evaluation can be avoided. Images were plotted based on the triggered seismic data.No computation is required, and images are more straightforward to manipulate for input preparation. Furthermore, experienced seismologists can assess whether the signal is correctly classified by simply reviewing the images,hence improving the efficiency in the routine performance evaluation and model calibration process.

Another consideration in microseismic monitoring is that various sources can induce seismic signals, whilst the number of these signals associated with different sources could be unbalanced. The classifier model can be underfit or overfit under these circumstances. Selectively using the labelled samples for training while considering a balanced ratio of all classes can be an effective solution.Despite that,it is possible that the seismic signals that are in question are not adequately recorded, e.g. pressure bumps and coal bursts, in which case, we may simply categorise these signals to the nearest neighbour or consider creating synthetic data to feed to the model (Meier et al., 2019).

7. Conclusions

The performance of four machine learning classifiers,including RF, SVM, DCNN, and ResNN, was investigated. Four categories of seismic signals, i.e. dyke-roadway events, longwall events, lowfrequency events, and noise acquired from an underground longwall coal mine, were classified. All classifiers attained excellent classification performance, suggesting that machine learning methods can be reliable tools for classifying spatially clustered seismic events. DCNN was proved to be the most efficient and reliable classifier for this dataset. Compared with the extracted seismic features, images are much less computationally expensive for trial and error during model calibration. Moreover, they are more straightforward to review.

Further, we evaluated the temporal changes in the prediction accuracy of DCNN in simulating real-time monitoring. A cascaded workflow was established to train the classifier and review the prediction performance. It demonstrated that with proper staged model calibration, the seismic events that occurred within the dyke-roadway area and around the longwall face were reliably discriminated. The capability of machine learning will promote reliable assessment of clustered seismicity for ongoing risk management during mining. As mine conditions and monitoring network configuration would vary at different mine sites, the general workflow presented in this research can be applied in specific scenarios. However, the time interval of applying the workflow was, to some extent, arbitrary in this study. The calibrations of the database and model should be dependent on the mining progress. It is worth developing a numerical model to predict the temporal variations of seismic signal characteristics associated with the mining progress, thus the calibration time interval can be dynamically determined. Moreover, only supervised machine learning is explored in this research.It requires sufficient samples to be accumulated and labelled for model training and testing,especially during an early stage of monitoring at a mine site.Hybrid techniques integrating transfer learning, unsupervised and supervised machine learning methods can be explored to accelerate the development of databases and classifiers.

Fig. 20. Prediction probabilities of four classes of seismic signals. (a) Dyke-roadway; (b) Longwall; (c) Low-frequency; (d) Noise.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors would like to thank the Australia Coal Association Research Program (ACARP) (Grant Nos. C26006 and C26053).Supports from CSIRO are gratefully acknowledged. The authors wish to thank Jane Hodgkinson and Binzhong Zhou for their valuable comments. The authors also thank anonymous reviewers for their insightful comments and suggestions.

List of symbols

VpVelocity of P-wave

γA hyperparameter controls the extend of curvature in determining an SVM decision boundary

CPunishment degree of an SVM classifier

PiPrecision with respect to class i

RiRecall with respect to class i

SiF1 score with respect to class i

AcciClassification accuracy with respect to class i

Kappa (к)Kappa coefficient

NNumber of samples

RowiTotal counts of samples of row i

ColiTotal counts of samples of column i

rDistance between a seismic source and a geophone

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jrmge.2021.09.002.