基于深度强化学习的多租户算网资源分配算法

胡宇翔; 冯旭; 董永吉; 和孟佯; 庄雷; 宋艳蕊

doi:10.11959/j.issn.2096-3750.2024.00446

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的多租户算网资源分配算法

专题：算力物联网 | 更新时间：2025-01-20

- 基于深度强化学习的多租户算网资源分配算法
- Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning
- 物联网学报 2024年8卷第4期页码：34-44
- 作者机构：
  
  1.信息工程大学信息技术研究所，河南郑州 450002
  2.先进通信网全国重点实验室，河南郑州 450002
  3.郑州大学网络空间安全学院，河南郑州 450002
  4.嵩山实验室，河南郑州 450002
  5.郑州大学计算机与人工智能学院，河南郑州 450001
- 作者简介：
  
  [ "胡宇翔（1982‒ ），男，信息工程大学信息技术研究所、先进通信网全国重点实验室教授、博士生导师，主要研究方向为新型网络架构、网络空间安全、智能路由与可编程转发等。" ]
  [ "冯旭（1997‒ ），男，信息工程大学信息技术研究所博士生，主要研究方向为零信任网络、下一代互联网等。" ]
  [ "董永吉（1983‒ ），男，信息工程大学信息技术研究所、先进通信网全国重点实验室副教授，主要研究方向为网络可编程数据平面和网络安全。" ]
  [ "和孟佯（1994‒ ），女，郑州大学网络空间安全学院、嵩山实验室助理研究员，主要研究方向为下一代互联网、互联网应用程序等。" ]
  [ "庄雷（1963‒ ），女，郑州大学计算机与人工智能学院教授、博士生导师，主要研究方向为模型检查、未来网络架构和网络虚拟化等。" ]
  [ "宋艳蕊（2000‒ ），女，郑州大学计算机与人工智能学院硕士生，主要研究方向为下一代互联网和服务功能链的安全部署。" ]
- 基金信息：
  
  国家重点研发计划(2023YFB2903902);中原科技创新领军人才项目(244200510038);河南省科技攻关-嵩山实验室资助项目(232102210154)
- DOI：10.11959/j.issn.2096-3750.2024.00446
  中图分类号：
- 纸质出版日期：2024-12-10，
  
  收稿日期：2024-10-21，
  
  修回日期：2024-12-11，
- 稿件说明：
移动端阅览
胡宇翔, 冯旭, 董永吉, 等. 基于深度强化学习的多租户算网资源分配算法[J]. 物联网学报, 2024,8(4):34-44.

HU YUXIANG, FENG XU, DONG YONGJI, et al. Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2024, 8(4): 34-44.
胡宇翔, 冯旭, 董永吉, 等. 基于深度强化学习的多租户算网资源分配算法[J]. 物联网学报, 2024,8(4):34-44. DOI： 10.11959/j.issn.2096-3750.2024.00446.

HU YUXIANG, FENG XU, DONG YONGJI, et al. Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning. [J]. Chinese journal on internet of things, 2024, 8(4): 34-44. DOI： 10.11959/j.issn.2096-3750.2024.00446.

摘要

随着智能化业务的迅猛发展，传统网络架构与计算能力之间的既有关系已难以满足当前需求，算网融合的实施势在必行。在算网融合所催生的新型算力网络框架下，高效且智能的资源调度策略成为提升用户体验的关键环节，但现有的资源调度算法优化目标单一，无法满足多租户差异化的业务需求。为此，提出了一种基于深度强化学习的多目标资源调度（MODRLRS

Multi objective deep reinforcement learning resource scheduling）算法来调用网络中的计算资源和网络资源，该算法通过构建帕累托最优解集的方法对算网资源进行多目标调度优化以满足不同租户的个性化业务需求。仿真对比实验表明，相比其他多目标资源调度算法，新算法提升了4.9%的请求接受率和4.78%的符合时延请求率，能够灵活适应各种计算业务的独特需求。

Abstract

With the rapid advancement of intelligent businesses

the pre-existing relationship between traditional network architectures and computing capabilities has made it difficult to meet the current demands

making the implementation of computing-network convergence inevitable. Under the new computing power network framework brought about by the convergence of computing networks

efficient and intelligent resource scheduling strategy has become a key link to improve user experience. However

the existing resource scheduling algorithms have a single optimization objective and cannot meet the differentiated business needs of multi-tenants. To this end

a Multi objective deep reinforcement learning resource scheduling (MODRLRS) was proposed to call the computing resources and network resources in the computing power network. The algorithm performs multi-objective scheduling optimization of computing network resources by constructing a Pareto optimal solution set to meet the personalized business needs of different tenants. Simulation experimental results show that compared with other multi-objective resource scheduling algorithms

the proposed algorithm improves the request acceptance rate by 4.9% and the compliant delay request rate by 4.78%

which can flexibly adapt to the unique requirements of various computing services.

关键词

算网融合算力网络资源调度多目标优化深度强化学习

Keywords

integration of computing and networkingcomputing power networkresource schedulingmulti objective optimizationdeep reinforcement learning

references

雷波, 刘增义, 王旭亮, 等. 基于云、网、边融合的边缘计算新方案: 算力网络[J]. 电信科学, 2019, 35(9): 44-51.

LEI B, LIU Z Y, WANG X L, et al. Computing network: a new multi-access edge computing[J]. Telecommunications Science, 2019, 35(9): 44-51.

郭凤仙, 孙耀华, 彭木根. 6G算力网络: 体系架构与关键技术[J]. 无线电通信技术, 2023, 49(1): 21-30.

GUO F X, SUN Y H, PENG M G. Computing force networks in 6G: architecture and key technologies[J]. Radio Communications Technology, 2023, 49(1): 21-30.

刘宇航, 张菲. 计算概念谱系: 算势、算力、算术、算法、算礼[J]. 中国科学院院刊, 2022, 37(10): 1500-1510.

LIU Y H, ZHANG F. Computing concept genealogy: potential, power, arithmetic, algorithm and ritual of computation[J]. Bulletin of Chinese Academy of Sciences, 2022, 37(10): 1500-1510.

张宏科, 权伟, 刘康. 算力网络研究与探索[J]. 中兴通讯技术, 2023, 29(1): 1-5.

ZHANG H K, QUAN W, LIU K. Research and exploration of computing power network[J]. ZTE Technology Journal, 2023, 29(1): 1-5.

陈晓红, 许冠英, 徐雪松, 等. 我国算力服务体系构建及路径研究[J]. 中国工程科学, 2023, 25(6): 49-60.

CHEN X H, XU G Y, XU X S, et al. Computing power service system of China and its development path[J]. Strategic Study of CAE, 2023, 25(6): 49-60.

梁芳, 佟恬, 马贺荣, 等. 东数西算下算力网络发展分析[J]. 信息通信技术与政策, 2022(11): 79-83.

LIANG F, TONG T, MA H R, et al. Analysis of the development of the computing power network under east-data-west-computing project[J]. Information and Communications Technology and Policy, 2022(11): 79-8.

ITU-T. Computing power network-framework and architecture: Y.2501[S]. 2021.

杨帆, 宋闻萱, 许方敏, 等. 工业互联网算网一体技术研究[J]. 无线电通信技术, 2023, 49(1): 63-71.

YANG F, SONG W X, XU F M, et al. Research on the application of computing force network technology in industrial Internet of things[J]. Radio Communications Technology, 2023, 49(1): 63-71.

王少鹏, 邱奔. 算网协同对算力产业发展的影响分析[J]. 信息通信技术与政策, 2022(3): 29-33.

WANG S P, QIU B. Analysis on the impact of computing network collaboration on the development of computing power industry[J]. Information and Communications Technology and Policy, 2022(3): 29-33.

王淑玲, 孙杰, 王鹏, 等. 云边协同中的资源调度优化[J]. 电信科学, 2023, 39(2): 163-170.

WANG S L, SUN J, WANG P, et al. Resource scheduling optimization in cloud-edge collaboration[J]. Telecommunications Science, 2023, 39(2): 163-170.

HAZRA A, RANA P, ADHIKARI M, et al. Fog computing for next-generation Internet of things: fundamental, state-of-the-art and research challenges[J]. Computer Science Review, 2023, 48: 100549.

WU Y L, DAI H N, WANG H Z, et al. A survey of intelligent network slicing management for industrial IoT: integrated approaches for smart transportation, smart energy, and smart factory[J]. IEEE Communications Surveys & Tutorials, 2022, 24(2): 1175-1211.

XIONG Z H, ZHANG Y, LUONG N C, et al. The best of both worlds: a general architecture for data management in blockchain-enabled Internet-of-things[J]. IEEE Network, 2020, 34(1): 166-173.

ITU-T. Signalling requirements for service deployment in computing power network: Q.4140[S]. 2023.

CUI Y Y, ZHANG D G, ZHANG T, et al. A novel offloading scheduling method for mobile application in mobile edge computing[J]. Wireless Networks, 2022, 28(6): 2345-2363.

JAMIL B, IJAZ H, SHOJAFAR M, et al. Resource allocation and task scheduling in fog computing and Internet of everything environments: a taxonomy, review, and future directions[J]. ACM Computing Surveys, 2022, 54(11s): 1-38.

ADDYA S K, SATPATHY A, GHOSH B C, et al. CoMCLOUD: virtual machine coalition for multi-tier applications over multi-cloud environments[J]. IEEE Transactions on Cloud Computing, 2023, 11(1): 956-970.

FARHADI V, MEHMETI F, HE T, et al. Service placement and request scheduling for data-intensive applications in edge clouds[J]. IEEE/ACM Transactions on Networking, 2021, 29(2): 779-792.

GAO Y F, YAN Z B, ZHAO K L, et al. Joint optimization of server and service selection in satellite-terrestrial integrated edge computing networks[J]. IEEE Transactions on Vehicular Technology, 2024, 73(2): 2740-2754.

BAHRAMI B, KHAYYAMBASHI M R, MIRJALILI S. Multiobjective placement of edge servers in MEC environment using a hybrid algorithm based on NSGA-II and MOPSO[J]. IEEE Internet of Things Journal, 2024, 11(18): 29819-29837.

ABDELMONEEM R M, BENSLIMANE A, SHAABAN E. Mobility-aware task scheduling in cloud-Fog IoT-based healthcare architectures[J]. Computer Networks, 2020, 179: 107348.

NI L N, ZHANG J Q, JIANG C J, et al. Resource allocation strategy in fog computing based on priced timed petri nets[J]. IEEE Internet of Things Journal, 2017, 4(5): 1216-1228.

张维庭, 孙呈蕙, 王洪超, 等. 算网资源智能适配与融合调度方法[J]. 电信科学, 2023, 39(9): 12-20.

ZHANG W T, SUN C H, WANG H C, et al. Intelligent adaptation and integrated scheduling method for computing and networking resources[J]. Telecommunications Science, 2023, 39(9): 12-20.

ZHAO X Y, ZONG Q, TIAN B L, et al. Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning[J]. Aerospace Science and Technology, 2019, 92: 588-594.

WU Q, WANG W H, FAN P Y, et al. Cooperative edge caching based on elastic federated and multi-agent deep reinforcement learning in next-generation networks[J]. IEEE Transactions on Network and Service Management, 2024, 21(4): 4179-4196.

WANG X B, WU Q, FAN P Y, et al. Vehicle selection for C-V2X mode 4-based federated edge learning systems[J]. IEEE Systems Journal, 2024, 18(4): 1927-1938.

ZHU H B. Pareto improvement: a GRA perspective[J]. IEEE Transactions on Computational Social Systems, 2023, 10(3): 1241-1251.

FAN Y Q, LIU J H, YE H, et al. TA-LSTM: a time and attribute aware LSTM for deep flight track clustering[J]. IEEE Transactions on Aerospace and Electronic Systems, 2023, 59(5): 7047-7060.

CAO K, WENG J. REPFS: reliability-ensured personalized function scheduling in sustainable serverless edge computing[J]. IEEE Transactions on Sustainable Computing, 2024, 9(3): 494-511.

PENG K, HUANG H L, ZHAO B H, et al. Intelligent computation offloading and resource allocation in IIoT with end-edge-cloud computing using NSGA-III[J]. IEEE Transactions on Network Science and Engineering, 2023, 10(5): 3032-3046.

WANG H Z, TANG L X, XIAO M, et al. Multi-objective optimization for joint communication and computing resource allocation in NOMA-based MEC system[C]//Proceedings of the 2024 IEEE Congress on Evolutionary Computation (CEC). Piscataway: IEEE Press, 2024: 1-8.

WANG S M, SONG X Q, XU H, et al. Joint offloading decision and resource allocation in vehicular edge computing networks[J]. Digital Communications and Networks, 2023.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于能量收集技术的协作卸载计算方案

基于连续动作空间深度强化学习的多数据融合室内定位方法

基于深度强化学习的SDN服务质量智能优化算法

基于深度强化学习的智能车间调度方法研究

基于深度强化学习的物联网智能路由策略