Development and investigation of an agent based on Deep Q-Network algorithm for control tasks in dynamic environments

Abstract

Deep reinforcement learning (Deep RL) is one of the most promising areas of machine learning. Deep reinforcement learning is an important area of machine learning that finds application in autonomous control, robotics, and gaming systems. This paper presents the implementation of the Deep Q-Network (DQN) algorithm for training an agent to control the Snake game. The methodology includes the formation of a compact vector representation of the environment state from 11 binary features, the use of a fully connected neural network to approximate the Q-function, the use of an experience replay mechanism with a buffer of 100,000 records, and an ε-greedy strategy to ensure a balance between exploring the environment and exploiting the knowledge gained. During the study, the agent was trained over 500 game episodes with different environment configurations. The results showed a steady increase in the average score from 0 to 23.7 during training and an average score of 25.5 during testing, which is 7.6% higher than the final training score. The maximum score achieved was 74 points. The results confirm the applicability of Deep RL for solving control problems in stochastic environments and demonstrate the ability of the DQN agent to generalize the learned strategy.