Seismically Informed Reference Models Enhance AI-Based Earthquake Prediction Systems

Given the robust nonlinear regression capabilities of Artificial Intelligence (AI) technology, its commendable performance in numerous geophysical tasks is expected. Yet, AI technology suffers from (a) its "black box" nature and (b) the fact that some complicated artificial neural networks (ANNs) claiming superior performance do not surpass some simple geophysical models that clearly describe the underlying physical processes. Numerous reports rely on standard machine learning metrics, often using a spatially uniform Poisson (SUP) distribution as their reference. A good performance just means that the artificial neural network (ANN) outperforms this basic reference, potentially offering little novelty to the scientific community. Worse, this can lead to spurious inference. We demonstrate this by using the monthly average human-made Nighttime Light Map and the cumulative energy of earthquakes in various space-time units as inputs for an Long short-term memory model. The goal is to predict earthquakes with a magnitude of M >= 5.0 across the entire Chinese Mainland. With the SUP reference model, the ANN concludes that human-made Nighttime Light possesses substantial earthquake prediction capability. This is evidently flawed reasoning. We show that this stems from the poor reference model and this spurious inference disappears when using a better benchmark consisting of a spatially varying Poisson (SVP) model informed from statistical seismology. This is implemented by weighting the punishments/rewards of our ANN associated with failed/successful predictions by prior probabilities provided by the stronger SVP model. Scores obtained with the time-space Molchan diagram demonstrate the strong performance improvement obtained by training ANN with a better reference model. Recent impressive advances in Artificial Intelligence promise revolutionary better performance in predicting earthquakes. Many reports are based on some off-the-shelf machine learning evaluation metrics, most of which take spatially uniform Poisson distribution as reference models. However, such encouraging results just inform that the artificial neural network (ANN) is better than this naive reference model, and these ANNs may bring nothing new to the scientific and engineering community. Here, we show how to address this gap and present a general scheme in which the best existing models are used as reference models against which the ANN can compete and learn. We illustrate this method in earthquake predictions. The punishments/rewards of our ANN associated with failed/successful predictions are weighted by prior probabilities provided by a strong statistical seismology model. The scores acquired through the time-space Molchan diagram illustrate the significant performance enhancement achieved by training the ANN with an improved reference model. Positive scores provided by evaluation metrics with poor reference models do not mean that the tested model is improving on the state-of-the-art Introducing specialized knowledge into the reference model of loss functions can improve the power of artificial neural networks Artificial neural networks should be designed to compete with and learn from the most powerful state-of-the-art models