基于扩散强化学习的UAV能量导向轨迹规划与物联网服务质量优化

张子天; 葛天豪; 诸葛斌; 郑运强; 董黎刚; 蒋献

doi:10.11959/j.issn.2096-3750.WLW25132

您当前的位置：

首页 >

文章列表页 >

基于扩散强化学习的UAV能量导向轨迹规划与物联网服务质量优化

理论与技术 | 更新时间：2026-07-24

- 基于扩散强化学习的UAV能量导向轨迹规划与物联网服务质量优化
- UAV energy-oriented trajectory planning and IoT service quality optimization based on diffusion reinforcement learning
- 物联网学报 2026年10卷第2期页码：165-176
- 作者机构：
  
  浙江工商大学信息与电子工程学院，浙江杭州 310018
- 作者简介：
  
  张子天（1988‒ ），男，博士，浙江工商大学信息与电子工程学院副研究员，主要研究方向为基于机器学习的网络流量预测与资源管理。
  葛天豪（2000‒ ），男，浙江工商大学信息与电子工程学院硕士生，主要研究方向为空天地网络。
  诸葛斌（1976‒ ），男，博士，浙江工商大学信息与电子工程学院教授，主要研究方向为网络和通信技术、互联网技术和网络安全。
  郑运强（1998‒ ），男，浙江工商大学信息与电子工程学院硕士生，主要研究方向为基于强化学习的网络资源管理。
  董黎刚（1972‒ ），男，博士，浙江工商大学信息与电子工程学院教授，主要研究方向为智能网络、在线教育。
  蒋献（1988‒ ），男，浙江工商大学信息与电子工程学院讲师、实验员，主要研究方向为在线教育。
- 基金信息：
  
  国家自然科学基金资助项目(62301488;W2421086)
- DOI：10.11959/j.issn.2096-3750.WLW25132
  中图分类号：
- 收稿：2025-08-26，
  
  修回：2025-09-28，
  
  录用：2025-10-20，
  
  网络首发：2026-07-24，
  
  纸质出版：2026-06-20
- 稿件说明：
移动端阅览
张子天,葛天豪,诸葛斌等.基于扩散强化学习的UAV能量导向轨迹规划与物联网服务质量优化[J].物联网学报,2026,10(02):165-176.

Zhang Zitian,Ge Tianhao,Zhuge Bin,et al.UAV energy-oriented trajectory planning and IoT service quality optimization based on diffusion reinforcement learning[J].Chinese Journal on Internet of Things,2026,10(02):165-176.
张子天,葛天豪,诸葛斌等.基于扩散强化学习的UAV能量导向轨迹规划与物联网服务质量优化[J].物联网学报,2026,10(02):165-176. DOI： 10.11959/j.issn.2096-3750.WLW25132.

Zhang Zitian,Ge Tianhao,Zhuge Bin,et al.UAV energy-oriented trajectory planning and IoT service quality optimization based on diffusion reinforcement learning[J].Chinese Journal on Internet of Things,2026,10(02):165-176. DOI： 10.11959/j.issn.2096-3750.WLW25132.

摘要

为解决物联网（IoT

Internet of things）设备产生的异构计算任务需要高效调度的问题，提出基于扩散强化学习（DiffRL

diffusion reinforcement learning）的无人机（UAV

unmanned aerial vehicle）通信网络与任务卸载系统。主要创新点包括：（1）扩散强化学习卸载决策框架，通过去噪扩散隐式模型（DDIM

denoising diffusion implicit model）采样技术将采样步数从50步减少到15步，加速70%的采样过程，同时保持98%的决策质量；（2）能量导向无人机轨迹规划算法，降低系统总能耗15.3%；（3）实现DiffRL决策与轨迹规划的紧耦合，解决动态环境下多目标优化问题。实验表明，新系统在能耗和任务时延方面较传统方法均有下降，在任务卸载决策中比传统深度Q网络（DQN

deep Q-network）和深度确定性策略梯度（DDPG

deep deterministic policy gradient）算法任务时延分别降低30.2%和9.2%。

Abstract

To address the high-frequency heterogeneous computing task requirements generated by IoT devices

an unmanned aerial vehicle (UAV) communication network and task offloading system based on diffusion reinforcement learning (DiffRL) was proposed. There were three main innovations. (1) A diffusion reinforcement learning offloading decision framework that reduced sampling steps from 50 to 15 steps through denoising diffusion implicit model (DDIM) sampling technology

accelerating the sampling process by 70% while maintaining 98% of decision quality. (2) An energy-oriented UAV trajectory planning algorithm that reduced total system energy consumption by 15.3%. (3)Achieving tight coupling between DiffRL decision-making and trajectory planning to solve multi-objective optimization problems in dynamic environments. Experiments show that this system achieves reductions in both energy consumption and task delay compared to traditional methods

reducing task delay by 30.2% and 9.2% respectively when compared to traditional deep Q-network (DQN) and deep deterministic policy gradient algorithms (DDPG) in task offloading decisions.

关键词

Keywords

references

Zeng Y , Zhang R . Energy-efficient UAV communication with trajectory optimization [J ] . IEEE Transactions on Wireless Communications , 2017 , 16 ( 6 ): 3747 - 3760 .

Zhou F , Wu Y , Hu R Q , et al . Computation rate maximization in UAV-enabled wireless-powered mobile-edge computing systems [J ] . IEEE Journal on Selected Areas in Communications , 2018 , 36 ( 9 ): 1927 - 1941 .

Mnih V , Kavukcuoglu K , Silver D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

Schulman J , Wolski F , Dhariwal P , et al . Proximal policy optimization algorithms [PP ] . V2. arXiv ( 2017-08-28 )[ 2025-07-20 ] . arXiv： 1707.06347 .

Janner M , Du Y , Tenenbaum J T , et al . Planning with diffusion for flexible behavior synthesis [C ] // Proceedings of the 37th International Conference on Machine Learning (ICML) . New York : PMLR Press , 2022 : 9782 - 9793 .

Liang H , Yang Z Q , Zhang G B , et al . Resource allocation for space-air-ground integrated networks: a comprehensive review [J ] . Journal of Communications and Information Networks , 2024 , 9 ( 1 ): 1 - 23 .

Hansen-Estruch P , Zhang A K , Vuong Q , et al . Diffusion policies as an expressive policy class for offline reinforcement learning [C ] // Proceedings of the International Conference on Learning Representations (ICLR) . Washington DC : ICLR Press , 2023 .

Du H Y , Li Z H , Niyato D , et al . Diffusion-based reinforcement learning for edge-enabled AI-generated content services [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 9 ): 8902 - 8918 .

Song J , Meng C , Ermon S . Denoising diffusion implicit models [C ] // Proceedings of the 9th International Conference on Learning Representations (ICLR) . Washington DC : ICLR Press , 2021 .

Wu Q Q , Zeng Y , Zhang R . Joint trajectory and communication design for multi-UAV enabled wireless networks [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 3 ): 2109 - 2121 .

Liu L , Zhang S W , Zhang R . Multi-beam UAV communication in cellular uplink: cooperative interference cancellation and sum-rate maximization [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 10 ): 4679 - 4691 .

Zhang S W , Zhang R . Radio map-based 3D path planning for cellular-connected UAV [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 3 ): 1975 - 1989 .

Liu X , Liu Y W , Chen Y . Reinforcement learning in multiple-UAV networks: deployment and movement design [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 8 ): 8036 - 8049 .

Khan N , Ahmad A , Wakeel A , et al . Efficient UAVs deployment and resource allocation in UAV-relay assisted public safety networks for video transmission [J ] . IEEE Access , 2024 , 12 : 4561 - 4574 .

Zheng Y S , Chen J T . Geography-aware optimal UAV 3D placement for LOS relaying: a geometry approach [J ] . IEEE Transactions on Wireless Communications , 2024 , 23 ( 8 ): 9301 - 9314 .

Chen M Z , Mozaffari M , Saad W , et al . Caching in the sky: proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience [J ] . IEEE Journal on Selected Areas in Communications , 2017 , 35 ( 5 ): 1046 - 1061 .

Liu C H , Chen Z Y , Tang J , et al . Energy-efficient UAV control for effective and fair communication coverage: a deep reinforcement learning approach [J ] . IEEE Journal on Selected Areas in Communications , 2018 , 36 ( 9 ): 2059 - 2070 .

Haarnoja T , Zhou A , Abbeel P . Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor [C ] // Proceedings of the 35th International Conference on Machine Learning (ICML) . New York : PMLR Press , 2018 : 1861 - 1870 .

Sefati S S , Haq A U , Nidhi , et al . A comprehensive survey on resource management in 6G network based on Internet of things [J ] . IEEE Access , 2024 , 12 : 113741 - 113784 .

Sohl-Dickstein J , Weiss E , Maheswaranathan N , et al . Deep unsupervised learning using nonequilibrium thermodynamics [C ] // Proceedings of the 32nd International Conference on Machine Learning (ICML) . New York : PMLR Press , 2015 : 2256 - 2265 .

Ho J , Jain A , Abbeel P . Denoising diffusion probabilistic models [C ] // Proceedings of the 34th Conference on Neural Information Processing Systems (NeurlPS) . New Orleans : NeurlPS , 2020 , 33 : 6840 - 6851 .

Wang P , Zhou Y , Xu K . Diffusion policies as an expressive policy class for offline reinforcement learning [C ] // Proceedings of the 10th International Conference on Learning Representations (ICLR) . Washington DC : ICLR Press , 2022 .

Du H , Zhang R , Liu Y , et al . Enhancing deep reinforcement learning: a tutorial on generative diffusion models in network optimization [J ] . IEEE Communications Surveys & Tutorials , 2024 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

无人机辅助车联网边缘计算卸载和资源分配方案

空天地一体化网络中的联合轨迹优化与计算卸载策略

面向高效巡检任务推理的边缘辅助无人机机载视频压缩与传输

面向无人机集群的感知与控制闭环调度策略

面向无人机航拍图像传输的信道弹性编解码方法