High-fidelity fluid simulations are essential for accurate aerodynamic analysis but remain computationally prohibitive when coupled with iterative design exploration loops. To address this bottleneck, this work presents a multi-fidelity geometry optimization framework that bridges the gap between rapid Reynolds-Averaged Navier-Stokes (RANS) estimations and computational demanding Large Eddy Simulation (LES), driven by Deep Reinforcement Learning (DRL). The core decision-making engine relies on the Soft Actor-Critic (SAC) algorithm. SAC utilizes an entropy-regularized formulation that compels agents to maximize both the expected reward and the entropy of the policy. This feature is critical for complex design landscapes, ensuring robust exploration and preventing premature convergence to local optima. The proposed workflow adopts a hierarchical two-stage strategy. First, the DRL agents interact with a lower-fidelity RANS environment to perform a rapid, broad exploration of the parametric space. This phase efficiently filters the design space to identify promising regions with minimal computational cost. Once a coarse optimum is located, the system transitions to a high-fidelity refinement stage using the spectral element solver SOD2D. In this second phase, LES is employed to capture unsteady flow phenomena and fine-tune the geometry near the optimal point. To make the high-fidelity phase computationally feasible, we integrate a solution mapping strategy. Instead of initializing each LES case from a uniform field, the framework interpolates initial conditions from previously converged fields. This drastically reduces the physical time required to wash out initial transients during the expensive LES simulations, significantly decreasing time-to-solution. This study demonstrates that coupling a multi-fidelity framework with a batch-based SAC approach and solution mapping provides a robust pipeline that balances the speed of RANS with the physics-resolving accuracy of LES for autonomous high-fidelity design exploration.