Robust Hyperparameter Estimation in Gaussian Process Regression Model

Abstract A stochastic process mathematically models the random behavior of systems and naturally occurring phenomena. A popular process is a Gaussian process which assumes a multivariate Gaussian distribution for the response variables. The predictive distributions of target variables can be explicitly obtained by using conditioning and inference techniques. The mean function and covariance function of the predictive distributions are parameterized whose estimates can be obtained by maximizing the marginal likelihood of the training responses or their posterior distribution. Unfortunately, these estimates are not robust to outliers that are often present in the observational training data sets. To overcome this problem, we develop a robust data-driven process based on the Schweppe-type Huber-generalized maximum likelihood (SHGM) estimator to bound the influence of all types of outliers on the estimates while exhibiting a high statistical efficiency at the Gaussian and thick-tail distributions. The weights utilized by the SHGM estimator are calculated using projection statistics, which are robust distances of the data points determined by the row vectors of the design matrix. These weights make the estimator robust to both vertical outliers and bad leverage points while not downweighting good leverage points. We show that the proposed robust model is able to handle up to 25 percent of outliers among the data set. The proposed method is illustrated by two real-world examples in which the performance of the proposed method is assessed against the standard Gaussian process regression.