Automated Hyperparameter Optimization of Convolutional Neural Network (CNN) for First-Break (FB) Arrival Picking

Abstract The Convolutional Neural Network (CNN) has been used successfully to enhance the First-break (FB) automated arrival picking of seismic data. Determining an optimized FB model is challenging as it needs to consider several hyperparameters (HPs) combinations. Tuning the most important HPs manually is infeasible because of a higher number of HP combinations to be tested. Three state-of-the-art automated hyperparameter optimization (HPO) techniques are applied to a CNN model for robust FB arrival picking classification. A CNN model with 4 convolutional (Conv) layers followed by one fully connected (FC) and one output layer is designed to classify the seismic event as FB or non-FB. To control overfitting, dropout (DO), batch normalization are used after every two Conv layers, in addition to only the DO layer after FC. The number and size of kernels, DO rate, Learning rate (Lr), and several neurons in the FC layer are fine-tuned using random search, Bayesian, and Hyper Band HPO techniques. The findings are experimentally evaluated and compared in terms of four performance metrics with respect to classification performance. The five hyperparameters mentioned above are fine-tuned in 13 search spaces for each of the three HPO techniques. From experimental results, applying random search HPO to CNN yields the best accuracy and F1-score of 96.26%, with the best HP combination of 16, 16, 32, and 64 for numbers of kernels in four Conv layers respectively; 2, 2, 2, 5 for the size of kernels in each Conv layer; 0, 0.45, 0.25 for DO rate in each of DO layers; 240 for numbers of neurons in FC layer; and 0.000675 for Lr. In terms of loss on test data, the above combination of HP gives the lowest test loss of 0.1191 among all techniques, making it a robust model. This model outperforms all the other models in terms of precision (96.27%) and recall. Moreover, all HPO models outperformed the baseline in terms of all metrics. The use of DO after Conv layers and FC layers is highly recommended. Moreover, the use of kernel size relatively smaller (i.e. 2) produces the best classification performance. According to the best HP combination results, there is also no harm to use a relatively higher number of neurons in the FC layer than the Conv layer in FB arrival picking classification. The optimal values of Lr range from 0.0001 to 0.000675 depending on the HPO techniques. The model developed in this study improves the accuracy of the auto-picking of FB seismic data and it is anticipated our model to be used more widely in future studies in the processing of seismic data.