Financial Time Series Trading using DDPG Considering Multi-scale Features.

In recent years, the application of deep reinforcement learning in the field of finance has received a lot of attention from researchers. Due to the non-stationary characteristic and noisy environment in the financial market, single-scale features are difficult to effectively characterize the market environment. In this paper, we extract multi-scale volume-price features and trend features from financial time series by multi-scale processing and propose a deep reinforcement learning model named MSDDPG-R, which is based on the Deep Deterministic Policy Gradient (DDPG) algorithm. Specifically, we consider the trading problem as a Markov Decision Process (MDP), where the state space considering both single-scale and multi-scale features is built and the reward function combining multi-scale trend features is used. We test the MSDDPG-R model on the datasets of SH000001, SH000300, SZ399905 and S&P 500. The results show that MSDDPG-R model performs better in terms of return and risk than other models that excludes the partial components, which illustrates the validity of the multi-scale features and the trend reward function.