Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning

Yi ZHOU; Xiaoyong MA; Fuxiao GAO; Wei LI; Nan CHENG; Ning LU

doi:10.11959/j.issn.2096-3750.2019.00106

您当前的位置：

首页 >

文章列表页 >

Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning

Theory and Technology | 更新时间：2024-06-05

- Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning
- Chinese Journal on Internet of Things Vol. 3, Issue 2, Pages: 47-55(2019)
- 作者机构：
  
  1. 河南大学计算机与信息工程学院，河南开封 475004
  2. 河南省车联网协同技术国际联合实验室，河南开封 475004
  3. 滑铁卢大学电气与计算机工程学院，安大略滑铁卢 N2L 3G1
  4. 汤普森河大学计算机科学系，不列颠哥伦比亚坎路普斯 V2C 0C8
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2019.00106
  CLC： TP393
- Published：30 June 2019，
  
  Published Online：2019-06，
- 稿件说明：
移动端阅览
YI ZHOU, XIAOYONG MA, FUXIAO GAO, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2019, 3(2): 47-55.
DOI：

YI ZHOU, XIAOYONG MA, FUXIAO GAO, et al. Autonomous deployment and energy efficiency optimization strategy of UAV based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2019, 3(2): 47-55. DOI： 10.11959/j.issn.2096-3750.2019.00106.

摘要

利用无人机组建空中移动基站，可为地面终端用户提供更灵活、高效的接入服务。受无人机覆盖范围和有限能量的约束，研究如何建立快速、高效、节能的空地协同网络至关重要，无人机需要根据复杂动态场景进行最优覆盖部署，同时要减少部署过程中的路径损耗和能量消耗。基于深度强化学习提出了无人机自主部署和能效优化策略，建立无人机覆盖状态集合，以能效作为奖励函数，利用深度神经网络和Q-learning引导无人机自主决策，部署最佳位置。仿真结果表明，该方法的部署时间能够有效减少60%，能耗可降低10%~20%。

Abstract

Utilizing a UAV to build aerial mobile small cell can provide more flexible and efficient access services for ground terminal users.Constrained by the coverage and limited energy of the UAV

it is necessary to study how to build a fast

efficient and energy-saving air-ground collaborative network.To deal with complex dynamic scenarios

the UAV needs to deploy an optimal coverage position

and meanwhile reduce both path loss and energy consumption in the deployment process.Based on the deep reinforcement learning

a strategy of autonomous UAV deployment and efficiency optimization was proposed.The coverage state set of UAV was established

and the energy efficiency was used as a reward function.Depth neural network and Q-learning were used to guide UAV to make autonomous decision and deploy the optimal position.The simulation results show that the deployment time of the proposed method can be effectively reduced by 60%

while the energy consumption can be reduced by 10%~20%.

关键词

空地协作组网无人机自主部署能效优化深度强化学习

Keywords

aerial-ground cooperative networkingunmanned aerial vehicleautonomous deploymentefficiency optimizationdeep reinforcement learning

references

LU N, ZHOU Y, SHI C ,et al. Planning while flying:a measurement-aided dynamic planning of drone small cells[J]. IEEE Internet of Things Journal, 2019,6(2): 2693-2705.

BOR-YALINIZ R, EL-KEYI A, YANIKOMEROGLU H ,et al. Efficient 3D placement of an aerial base station in next generation cellular networks[C]// 2016 IEEE International Conference on Communications(ICC). IEEE, 2016: 1-5.

ZHOU Y, CHENG N, LU N ,et al. Multi-UAV aided networks:aerial-ground cooperative vehicular networking architecture[J]. IEEE Vehicular Technology Magazine, 2015,10(4): 36-44.

CHENG N, XU W, SHI W ,et al. Air-ground integrated mobile edge networks:architecture,challenges and opportunities[J]. IEEE Communications Magazine, 2018,56(8): 26-32.

MOZAFFARI M, SAAD W, BENNIS M ,et al. Unmanned aerial vehicle with underlaid device-to-device communications:performance and tradeoffs[J]. IEEE Transactions on Wireless Communications, 2016,15(6): 3949-3963.

ZENG Y, ZHANG R . Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017,16(6): 3747-3760.

AL-HOURANI A, KANDEEPAN S, JAMALIPOUR A . Modeling air-to-ground path loss for low altitude platforms in urban environments[C]// 2014 IEEE Global Communications Conference. IEEE, 2014: 2898-2904.

LYU J, ZENG Y, ZHANG R ,et al. Placement optimization of UAV-mounted mobile base stations[C]// IEEE Communications Letters. 2017,21(3): 604-607.

AL-HOURANI A, KANDEEPAN S, LARDNER S . Optimal LAP altitude for maximum coverage[J]. IEEE Wireless Communications, 2014,3: 569-572.

KALANTARI E, YANIKOMEROGLU H, YONGACOGLU A . On the number and 3D placement of drone base stations in wireless cellular networks[C]// 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall). IEEE, 2016: 1-6.

ALZENAD M, EL-KEYI A, LAGUM F ,et al. 3D placement of an unmanned aerial vehicle base station (UAV-BS) for energy-efficient maximal coverage[C]// IEEE Wireless Communications Letters, 2017,6(4): 434-437.

RUAN L, WANG J, CHEN J ,et al. Energy-efficient multi-UAV coverage deployment in UAV networks:a game-theoretic framework[J]. China Communications, 2018,15(10): 194-209.

CHEN Y, LI N, WANG C ,et al. A 3D placement of unmanned aerial vehicle base station based on multi-population genetic algorithm for maximizing users with different QoS requirements[C]// 2018 IEEE 18th International Conference on Communication Technology (ICCT). IEEE, 2018: 967-972.

LAI C, CHEN C, WANG L . On-demand density-aware UAV base station 3D placement for arbitrarily distributed users with guaranteed data rates[C]// IEEE Wireless Communications Letters. IEEE, 2019:1.

WANG L, HU B, CHEN S . Energy efficient placement of a drone base station for minimum required transmit power[C]// IEEE Wireless Communications Letters. IEEE, 2018.

SHAKHATREH H, KHREISHAH A, ALSARHAN A ,et al. Efficient 3D placement of a UAV using particle swarm optimization[C]// 2017 8th International Conference on Information and Communication Systems (ICICS). IEEE, 2017: 258-263.

MNIH V, KAVUKCUOGLU K, SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.

XU Y, LIU Z, WANG X . Monocular vision based autonomous landing of quadrotor through deep reinforcement learning[C]// 2018 37th Chinese Control Conference (CCC). IEEE, 2018: 10014-10019.

ZHANG Y, SUN P, YIN Y ,et al. Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning[C]// 2018 IEEE Intelligent Vehicles Symposium(IV). IEEE, 2018: 1251-1256.

GOODFELLOW I, BENGIO Y, COURVILLE A . Deep learning[M]. Cambridge: MIT PressPress, 2016.

Views

994

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Emergency UAV deployment optimization in an integrated air-ground network with joint communication and localization service

Joint optimization method for UAV trajectory and charging/discharging in cell-free mMIMO system on WPT

An edge-cloud collaborative model evolution and communication paradigm in Internet of low-altitude UAV

Path loss measurement and modeling for low-altitude UAV scenarios towards 3GPP ISAC standadization

Collaborative offloading computing scheme based on energy harvesting technology

Related Author

BI Suzhi

ZHAO Haitao

CAI Yan

LIU Ying

ZHANG Yuxuan

NING Luohan

WANG Qin

YU Xinbo

Related Institution

College of Electronics and Information Engineering, Shenzhen University

College of Internet of Things, Nanjing University of Posts and Telecommunications

College of Communication and Information Engineering, Nanjing University of Posts and Telecommunications

School of Electronics, Peking University

PengCheng Laboratory

⁰