基于深度强化学习的无人机自主部署及能效优化策略

周毅; 马晓勇; 郜富晓; 李伟; 承楠; 路宁

doi:10.11959/j.issn.2096-3750.2019.00106

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的无人机自主部署及能效优化策略

理论与技术 | 更新时间：2024-06-05

- 基于深度强化学习的无人机自主部署及能效优化策略
- Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning
- 物联网学报 2019年3卷第2期页码：47-55
- 作者机构：
  
  1. 河南大学计算机与信息工程学院，河南开封 475004
  2. 河南省车联网协同技术国际联合实验室，河南开封 475004
  3. 滑铁卢大学电气与计算机工程学院，安大略滑铁卢 N2L 3G1
  4. 汤普森河大学计算机科学系，不列颠哥伦比亚坎路普斯 V2C 0C8
- 作者简介：
  
  [ "周毅（1981- ），男，河南信阳人，河南大学副教授、博士生导师，主要研究方向为车联网、空地协同组网、平行增强学习、协作机器人等。" ]
  [ "马晓勇（1993- ），男，河南洛阳人，河南大学硕士生，主要研究方向为无人机组网、边缘计算等。" ]
  [ "郜富晓（1992- ），女，河南洛阳人，河南大学硕士生，主要研究方向为空地协同组网、深度学习等。" ]
  [ "李伟（1979- ），女，河南济源人，河南大学副教授，主要研究方向为车联网优化控制、协作通信等。" ]
  [ "承楠（1987- ），男，辽宁锦州人，滑铁卢大学在站博士后，主要研究方向为车联网、人工智能、空地协同等。" ]
  [ "路宁（1984- ），男，山西长治人，汤普森河大学助理教授，主要研究方向为车联网、移动边缘计算等。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61701170);河南省国际科技合作项目;Henan International Science ＆ Technology Cooperation Program(182102410050);河南省青年骨干教师计划;Henan Young Scholar Promotion Program(2016GGJS-018)
- DOI：10.11959/j.issn.2096-3750.2019.00106
  中图分类号： TP393
- 纸质出版日期：2019-06-30，
  
  网络出版日期：2019-06，
- 稿件说明：
移动端阅览
周毅, 马晓勇, 郜富晓, 等. 基于深度强化学习的无人机自主部署及能效优化策略[J]. 物联网学报, 2019,3(2):47-55.

YI ZHOU, XIAOYONG MA, FUXIAO GAO, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2019, 3(2): 47-55.
周毅, 马晓勇, 郜富晓, 等. 基于深度强化学习的无人机自主部署及能效优化策略[J]. 物联网学报, 2019,3(2):47-55. DOI： 10.11959/j.issn.2096-3750.2019.00106.

YI ZHOU, XIAOYONG MA, FUXIAO GAO, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2019, 3(2): 47-55. DOI： 10.11959/j.issn.2096-3750.2019.00106.

摘要

利用无人机组建空中移动基站，可为地面终端用户提供更灵活、高效的接入服务。受无人机覆盖范围和有限能量的约束，研究如何建立快速、高效、节能的空地协同网络至关重要，无人机需要根据复杂动态场景进行最优覆盖部署，同时要减少部署过程中的路径损耗和能量消耗。基于深度强化学习提出了无人机自主部署和能效优化策略，建立无人机覆盖状态集合，以能效作为奖励函数，利用深度神经网络和Q-learning引导无人机自主决策，部署最佳位置。仿真结果表明，该方法的部署时间能够有效减少60%，能耗可降低10%~20%。

Abstract

Utilizing a UAV to build aerial mobile small cell can provide more flexible and efficient access services for ground terminal users.Constrained by the coverage and limited energy of the UAV

it is necessary to study how to build a fast

efficient and energy-saving air-ground collaborative network.To deal with complex dynamic scenarios

the UAV needs to deploy an optimal coverage position

and meanwhile reduce both path loss and energy consumption in the deployment process.Based on the deep reinforcement learning

a strategy of autonomous UAV deployment and efficiency optimization was proposed.The coverage state set of UAV was established

and the energy efficiency was used as a reward function.Depth neural network and Q-learning were used to guide UAV to make autonomous decision and deploy the optimal position.The simulation results show that the deployment time of the proposed method can be effectively reduced by 60%

while the energy consumption can be reduced by 10%~20%.

关键词

空地协作组网无人机自主部署能效优化深度强化学习

Keywords

aerial-ground cooperative networkingunmanned aerial vehicleautonomous deploymentefficiency optimizationdeep reinforcement learning

references

LU N, ZHOU Y, SHI C ,et al. Planning while flying:a measurement-aided dynamic planning of drone small cells[J]. IEEE Internet of Things Journal, 2019,6(2): 2693-2705.

BOR-YALINIZ R, EL-KEYI A, YANIKOMEROGLU H ,et al. Efficient 3D placement of an aerial base station in next generation cellular networks[C]// 2016 IEEE International Conference on Communications(ICC). IEEE, 2016: 1-5.

ZHOU Y, CHENG N, LU N ,et al. Multi-UAV aided networks:aerial-ground cooperative vehicular networking architecture[J]. IEEE Vehicular Technology Magazine, 2015,10(4): 36-44.

CHENG N, XU W, SHI W ,et al. Air-ground integrated mobile edge networks:architecture,challenges and opportunities[J]. IEEE Communications Magazine, 2018,56(8): 26-32.

MOZAFFARI M, SAAD W, BENNIS M ,et al. Unmanned aerial vehicle with underlaid device-to-device communications:performance and tradeoffs[J]. IEEE Transactions on Wireless Communications, 2016,15(6): 3949-3963.

ZENG Y, ZHANG R . Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017,16(6): 3747-3760.

AL-HOURANI A, KANDEEPAN S, JAMALIPOUR A . Modeling air-to-ground path loss for low altitude platforms in urban environments[C]// 2014 IEEE Global Communications Conference. IEEE, 2014: 2898-2904.

LYU J, ZENG Y, ZHANG R ,et al. Placement optimization of UAV-mounted mobile base stations[C]// IEEE Communications Letters. 2017,21(3): 604-607.

AL-HOURANI A, KANDEEPAN S, LARDNER S . Optimal LAP altitude for maximum coverage[J]. IEEE Wireless Communications, 2014,3: 569-572.

KALANTARI E, YANIKOMEROGLU H, YONGACOGLU A . On the number and 3D placement of drone base stations in wireless cellular networks[C]// 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall). IEEE, 2016: 1-6.

ALZENAD M, EL-KEYI A, LAGUM F ,et al. 3D placement of an unmanned aerial vehicle base station (UAV-BS) for energy-efficient maximal coverage[C]// IEEE Wireless Communications Letters, 2017,6(4): 434-437.

RUAN L, WANG J, CHEN J ,et al. Energy-efficient multi-UAV coverage deployment in UAV networks:a game-theoretic framework[J]. China Communications, 2018,15(10): 194-209.

CHEN Y, LI N, WANG C ,et al. A 3D placement of unmanned aerial vehicle base station based on multi-population genetic algorithm for maximizing users with different QoS requirements[C]// 2018 IEEE 18th International Conference on Communication Technology (ICCT). IEEE, 2018: 967-972.

LAI C, CHEN C, WANG L . On-demand density-aware UAV base station 3D placement for arbitrarily distributed users with guaranteed data rates[C]// IEEE Wireless Communications Letters. IEEE, 2019:1.

WANG L, HU B, CHEN S . Energy efficient placement of a drone base station for minimum required transmit power[C]// IEEE Wireless Communications Letters. IEEE, 2018.

SHAKHATREH H, KHREISHAH A, ALSARHAN A ,et al. Efficient 3D placement of a UAV using particle swarm optimization[C]// 2017 8th International Conference on Information and Communication Systems (ICICS). IEEE, 2017: 258-263.

MNIH V, KAVUKCUOGLU K, SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.

XU Y, LIU Z, WANG X . Monocular vision based autonomous landing of quadrotor through deep reinforcement learning[C]// 2018 37th Chinese Control Conference (CCC). IEEE, 2018: 10014-10019.

ZHANG Y, SUN P, YIN Y ,et al. Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning[C]// 2018 IEEE Intelligent Vehicles Symposium(IV). IEEE, 2018: 1251-1256.

GOODFELLOW I, BENGIO Y, COURVILLE A . Deep learning[M]. Cambridge: MIT PressPress, 2016.

浏览量

994

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

空地协同通信定位一体化无人机应急部署优化

基于WPT的去蜂窝mMIMO系统中无人机轨迹与充放电联合优化方法

面向低空物联网的云-边协同演进模型与通信范式

面向3GPP ISAC标准化的低空无人机场景路径损耗测量与建模

基于能量收集技术的协作卸载计算方案