Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN

Haibo MEI; Kun YANG; Xinyu FAN

doi:10.11959/j.issn.2096-3750.2021.00234

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN

Topic: Edge Intelligence and Fog Computing in IoT | 更新时间：2024-06-05

- Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN
- Chinese Journal on Internet of Things Vol. 5, Issue 2, Pages: 48-59(2021)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2021.00234
  CLC： TP393
- Published：30 June 2021，
  
  Published Online：2021-06，
- 稿件说明：
移动端阅览
HAIBO MEI, KUN YANG, XINYU FAN. Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN. [J]. Chinese journal on internet of things, 2021, 5(2): 48-59.
DOI：

HAIBO MEI, KUN YANG, XINYU FAN. Deep reinforcement learning to enhance the energy-efficient performance of UAV-enabled F-RAN. [J]. Chinese journal on internet of things, 2021, 5(2): 48-59. DOI： 10.11959/j.issn.2096-3750.2021.00234.

摘要

雾无线电接入网络适合用于广域范围内的诸如管线管网监测等国家重要行业的物联网应用场景。然而基于地面雾接入节点的网络将受到环境、地形等影响，无法及时有效地提供雾接入服务。利用低空无人机作为雾接入点实现空地的边缘通信和雾计算方面引起了普遍的关注。本文探讨怎样利用深度增强学习来提高无人机雾接入点的能效，延长无人机的任务时间。深度增强学习可以保障无人机雾接入点及时地调整空地通信和计算的配置策略，包括资源优化、动态任务卸载以及缓存，也可以优化无人机在三维空间中的飞行航迹，提高无人机赋能的雾无线电接入网络的总体性能。研究的创新性在于综合论述了深度增强学习用于无人机赋能的雾无线电接入网络要解决的主要优化问题，并且总结了解决相关优化问题的技术细节，最后对深度增强学习应用于无人机赋能的雾无线电接入网络的技术挑战和未来研究方向展开讨论。

Abstract

Fog radio access network (F-RAN) is suitable for Internet of things applications of national important industries

such as pipeline network monitoring in wide area.However

the performance of the F-RAN based on the territorial fog access point will be affected greatly by the complicated territorial environment.This causes F-RAN not able to provide fog access service in a timely and effectively manner.To this problem

the research was proposed to utilize low altitude UAV as the fog access point to realize air ground edge communication and fog computing

which has attracted enormous research interests.How to use deep reinforcement learning (DRL) to improve the energy efficiency of UAV fog access point and extend the mission time of UAV were discussed.Deep reinforcement learning can ensure the UAV fog access point to adjust the configuration strategy timely of air ground communication and computing

including resource optimization

dynamic task offloading and caching.DRL can also optimize the UAV trajectory in 3-D space

and improve the overall performance of UAV enabled fog access network.The innovation of the research lies in the comprehensive discussion of the main optimization problems to be solved in the UAV-enabled F-RAN using DRL.The technical details were also summarized to solve the related optimization problems.Finally

the technical challenges and future research directions of the application of DRL in the UAV-enabled F-RAN were discussed.

关键词

无人机雾无线电接入网络深度增强学习航迹规划网络配置

Keywords

unmanned aerial vehiclefog radio access networkdeep reinforcement learningtrajectory designnetwork configuration

references

CHIANG M, ZHANG T . Fog and IoT:an overview of research opportunities[J]. IEEE Internet of Things Journal, 2016,3(6): 854-864.

LOKE S W . The Internet of flying-things:opportunities and challenges with airborne fog computing and mobile cloud in the clouds[J]. arXiv preprint arXiv:1507.04492, 2015.

ZENG Y, ZHANG R, LIM T J . Wireless communications with unmanned aerial vehicles:opportunities and challenges[J]. IEEE Communications Magazine, 2016,54(5): 36-42.

WU Q Q, LIU L, ZHANG R . Fundamental trade-offs in communication and trajectory design for UAV-enabled wireless network[J]. IEEE Wireless Communications, 2019,26(1): 36-44.

KU Y J, LIN D Y, LEE C F ,et al. 5G radio access network design with the fog paradigm:confluence of communications and computing[J]. IEEE Communications Magazine, 2017,55(4): 46-52.

ZENG Y, XU J, ZHANG R . Energy minimization for wireless communication with rotary-wing UAV[J]. IEEE Transactions on Wireless Communications, 2019,18(4): 2329-2345.

ZENG Y, ZHANG R . Energy-efficient UAV communication with trajectory optimization[J]. IEEE Transactions on Wireless Communications, 2017,16(6): 3747-3760.

MEI H B, WANG K Z, ZHOU D D ,et al. Joint trajectory-task-cache optimization in UAV-enabled mobile edge networks for cyber-physical system[J]. IEEE Access, 2019(7): 156476-156488.

MNIH V, KAVUKCUOGLU K, SILVER D ,et al. Human-level control through deep reinforcement learning[J]. Nature, 2015,518(7540): 529-533.

LI R P, ZHAO Z F, ZHOU X ,et al. Intelligent 5G:when cellular networks meet artificial intelligence[J]. IEEE Wireless Communications, 2017,24(5): 175-183.

CHEN M Z, MOZAFFARI M, SAAD W ,et al. Caching in the sky:proactive deployment of cache-enabled unmanned aerial vehicles for optimized quality-of-experience[J]. IEEE Journal on Selected Areas in Communications, 2017,35(5): 1046-1061.

CHEN M Z, SAAD W, YIN C C . Liquid state machine learning for resource and cache management in LTE-U unmanned aerial vehicle (UAV) networks[J]. IEEE Transactions on Wireless Communications, 2019,18(3): 1504-1517.

BOYD S, VANDENBERGHE L . Convex optimization[M]. Cambridge: Cambridge University Press, 2004.

MEI H B, YANG K, LIU Q ,et al. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone[J]. IEEE Internet of Things Journal, 2020,7(7): 5906-5921.

HE Y, YU F R, ZHAO N ,et al. Software-defined networks with mobile edge computing and caching for smart cities:a big data deep reinforcement learning approach[J]. IEEE Communications Magazine, 2017,55(12): 31-37.

CHALLITA U, SAAD W, BETTSTETTER C . Deep reinforcement learning for interference-aware path planning of cellular-connected UAVs[C]// 2018 IEEE International Conference on Communications (ICC). Piscataway:IEEE Press, 2018: 1-7.

LIU C H, CHEN Z Y, TANG J ,et al. Energy-efficient UAV control for effective and fair communication coverage:a deep reinforcement learning approach[J]. IEEE Journal on Selected Areas in Communications, 2018,36(9): 2059-2070.

MEI H B, YANG K, SHEN J ,et al. Joint trajectory-task-cache optimization with phase-shift design of RIS-assisted UAV for MEC[EB]. IEEE Wireless Communications Letters, 2021.

PENG H X, SHEN X S . DDPG-based resource management for MEC/UAV-assisted vehicular networks[C]// 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall). Piscataway:IEEE Press, 2020: 1-6.

Views

250

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Emergency UAV deployment optimization in an integrated air-ground network with joint communication and localization service

Joint optimization method for UAV trajectory and charging/discharging in cell-free mMIMO system on WPT

An edge-cloud collaborative model evolution and communication paradigm in Internet of low-altitude UAV

Path loss measurement and modeling for low-altitude UAV scenarios towards 3GPP ISAC standadization

A distributed strategy for the multi-target rescue using a UAV swarm under communication constraints

Related Author

BI Suzhi

ZHAO Haitao

CAI Yan

LIU Ying

ZHANG Yuxuan

NING Luohan

WANG Qin

YU Xinbo

Related Institution

College of Electronics and Information Engineering, Shenzhen University

College of Internet of Things, Nanjing University of Posts and Telecommunications

College of Communication and Information Engineering, Nanjing University of Posts and Telecommunications

School of Electronics, Peking University

PengCheng Laboratory

⁰