The Monte Carlo (MC) method is a fundamental approach in the field of reinforcement learning (RL) for estimating the value of states or state-action pairs. This method is particularly useful in model-free prediction and control, where the underlying dynamics of the environment are not known. The Monte Carlo method leverages the power of repeated random