Functional gradient descent for n-tuple regression

n-tuple neural networks have recently been applied to a wide range of learning domains. However, for the particular area of regression, existing systems have displayed two shortcomings: little flexibility in the objective function being optimized and an inability to handle nonstationarity in an online learning setting. A novel n-tuple system is proposed to address these issues. The new architecture leverages the idea of functional gradient descent, drawing inspiration from its use in kernel methods. Furthermore, its capabilities are showcased in reinforcement learning tasks, which involves both nonstationary online learning and task-specific objective functions.