Model-free reinforcement learning (RL) methods have gained significant attention in the field of artificial intelligence due to their unique advantages over model-based methods. The primary advantage of model-free methods lies in their ability to learn optimal policies and value functions without requiring an explicit model of the environment. This characteristic provides several benefits, including reduced