OPTIMIZATION OF STOCK TRADING STRATEGY WITH REINFORCEMENT LEARNING

Abstract

This project aims to optimize stock trading strategies by employing reinforcement learning (RL) techniques. The primary goal is to create a trading strategy capable of learning from historical data and executing profitable trades in real-time. The proposed method involves utilizing a deep neural network as a function approximator to learn a trading policy, which is subsequently fine-tuned using RL techniques. The neural network takes a set of technical indicators as input and generates buy, hold, or sell signals for each stock. To assess the performance of the RL-based trading strategy, a dataset comprising historical stock prices from the NIFTY and SENSEX indices is utilized. The performance of this approach is compared to several conventional trading strategies such as buy-and-hold and moving average crossover. The results indicate that the RL-based trading strategy surpasses these traditional strategies in terms of profitability and risk-adjusted returns. Additionally, sensitivity analysis is conducted to evaluate how different hyper-parameters impact the trading strategy's performance. Specifically, variations in the discount factor, learning rate, and exploration rate of the RL algorithm are examined, and their effects on the trading strategy's performance are analysed. The results demonstrate that the RL-based approach is robust against changes in hyper-parameters and consistently outperforms traditional strategies. Furthermore, a back-testing analysis is performed to assess the RL-based trading strategy's performance on out-of-sample data. A rolling window approach is implemented to simulate real-time trading, and the trading strategy's performance is evaluated over time. The results consistently show that the RL-based trading strategy outperforms traditional strategies and achieves higher risk-adjusted returns. Overall, the findings suggest that RL-based approaches have significant potential for enhancing the performance of stock trading strategies. The proposed method can be applied to various financial assets and can facilitate the development of automated trading systems capable of executing profitable trades in real-time.