Mesoscopic-Informed Residual Reinforcement Learning for Adaptive CAV Headway Control in Mixed-Autonomy Traffic

Classical constant-time-headway control limits each connected automated vehicle to predecessor-relative information, so disturbances developing farther upstream are not incorporated into the headway command until they reach the local interaction. This paper introduces a mesoscopic-informed residual reinforcement learning framework that embeds a learned correction within the coupling between microscopic car-following states and aggregate upstream traffic information. Each connected automated vehicle forms a bounded and interpretable headway prior from a forward-looking velocity descriptor through a rule-based mesoscopic fusion layer. A decentralized soft actorcritic policy, shared by all connected automated vehicles, then applies a state-dependent residual correction to this prior while preserving the underlying cooperative adaptive cruise control structure. The resulting controller updates the desired spacing together with the associated feedforward and free-flow contributions online, replacing a fixed macroscopic-to-microscopic mapping with an adaptive closed-loop coupling. The framework is evaluated on a mixed-autonomy ring road containing humandriven vehicles governed by the intelligent driver model and connected automated vehicles governed by cooperative adaptive cruise control, under multiple penetration rates and both singleblock and distributed-block arrangements. Compared with fixed cooperative adaptive cruise control, rule-only mesoscopic control, and proximal policy optimization based residual control, the proposed method reduces velocity and time-headway fluctuations and produces stronger front-to-rear disturbance attenuation, improving stop-and-go wave suppression while retaining the physical structure and constraints of the baseline controller.