CMRL: Collision-Aware and Memory-Enhanced Reinforcement Learning for UAV Navigation in Multi-Scale Obstacle Environments

Collision-Aware and Memory-Enhanced Reinforcement Learning
for UAV Navigation in Multi-Scale Obstacle Environments

Hong Hong^*, Feiyu Liao^*, Yongheng Liang, Boning Zhang, Haitao Wang^†, Hejun Wu^†

^*Equal contribution ^†Corresponding authors

Sun Yat-sen University

Abstract

In obstacle avoidance navigation of unmanned aerial vehicles (UAVs), variations in obstacle scale have received less attention than obstacle number or density. Existing methods typically extract purely geometric features from single-frame depth observations. Such representations tend to neglect small obstacles and lose spatial context under occlusions caused by large obstacles, leading to noticeable degradation in environments with multi-scale obstacles.

To address this issue, we propose CMRL, a Collision-aware and Memory-enhanced Reinforcement Learning framework for UAV navigation. The collision-aware latent representation encodes risk-sensitive depth cues to preserve fine-grained obstacle structures, thereby improving sensitivity to small obstacles. The temporal memory module integrates observations across frames, mitigating partial observability caused by large-obstacle occlusions.

We evaluate CMRL with multi-scale obstacles, including ultra-small and extra-large obstacle settings. Results show that CMRL outperforms state-of-the-art baselines across all scales, with success rate gains of 0.47 and 0.29 in the ultra-small and extra-large settings, respectively. More importantly, CMRL achieves reliable navigation in cluttered outdoor environments.

Method

CMRL combines two complementary representation modules. A VAE is trained to reconstruct collision-aware depth images, where obstacle boundaries are inflated by the UAV body size, embedding safety-relevant geometry into the latent space without runtime preprocessing. An LSTM then aggregates latent sequences and is supervised by jointly reconstructing both the current and a temporally offset collision-aware observation, encouraging the hidden state to maintain scene context under occlusion. The navigation policy is subsequently trained via RL with the representation modules frozen.

Multi-scale Obstacle Avoidance in Simulation

The policy is trained in a nominal-scale environment and evaluated zero-shot across six obstacle scales ranging from ultra-small (1–5 cm) to extra-large (400–500 cm). CMRL achieves consistently high success rates across all configurations, demonstrating robust generalization to obstacle scale variation without retraining.

Real-world Deployment

CMRL is deployed on a 250 mm quadrotor with an Intel RealSense D435, NVIDIA Jetson Orin NX, and Pixhawk 6C mini. The representation modules are fine-tuned on real-world depth data, while the policy transfers directly from simulation. Outdoor flight experiments confirm the feasibility of fully onboard deployment.

Citation

If you find our work useful in your research, please consider citing:

@article{hong2026cmrl,
  title   = {CMRL: Collision-Aware and Memory-Enhanced Reinforcement Learning for UAV Navigation in Multi-Scale Obstacle Environments},
  author  = {Hong, Hong and Liao, Feiyu and Liang, Yongheng and Zhang, Boning and Wang, Haitao and Wu, Hejun},
  journal = {arXiv preprint arXiv:2605.14810},
  year    = {2026},
}

CMRL