Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning

HU Yuxiang; FENG Xu; DONG Yongji; HE Mengyang; ZHUANG Lei; SONG Yanrui

doi:10.11959/j.issn.2096-3750.2024.00446

您当前的位置：

首页 >

文章列表页 >

Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning

Topic: Computing Power Internet of Things | 更新时间：2025-01-20

- Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning
- Chinese Journal on Internet of Things Vol. 8, Issue 4, Pages: 34-44(2024)
- 作者机构：
  
  1.信息工程大学信息技术研究所，河南郑州 450002
  2.先进通信网全国重点实验室，河南郑州 450002
  3.郑州大学网络空间安全学院，河南郑州 450002
  4.嵩山实验室，河南郑州 450002
  5.郑州大学计算机与人工智能学院，河南郑州 450001
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2023YFB2903902);The Science and Technology Innovation Leading Talents Subsidy Project of Central Plains(244200510038);The Scientific and Technological Project in Henan Province-Pre-Research Project of Songshan Laboratory(232102210154)
- DOI：10.11959/j.issn.2096-3750.2024.00446
  CLC：
- Received：21 October 2024，
  
  Revised：2024-12-11，
  
  Published：10 December 2024
- 稿件说明：
移动端阅览
胡宇翔,冯旭,董永吉等.基于深度强化学习的多租户算网资源分配算法[J].物联网学报,2024,08(04):34-44.

HU Yuxiang,FENG Xu,DONG Yongji,et al.Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning[J].Chinese Journal on Internet of Things,2024,08(04):34-44.
胡宇翔,冯旭,董永吉等.基于深度强化学习的多租户算网资源分配算法[J].物联网学报,2024,08(04):34-44. DOI： 10.11959/j.issn.2096-3750.2024.00446.

HU Yuxiang,FENG Xu,DONG Yongji,et al.Multi-tenant computing network resource allocation algorithm based on deep reinforcement learning[J].Chinese Journal on Internet of Things,2024,08(04):34-44. DOI： 10.11959/j.issn.2096-3750.2024.00446.

摘要

随着智能化业务的迅猛发展，传统网络架构与计算能力之间的既有关系已难以满足当前需求，算网融合的实施势在必行。在算网融合所催生的新型算力网络框架下，高效且智能的资源调度策略成为提升用户体验的关键环节，但现有的资源调度算法优化目标单一，无法满足多租户差异化的业务需求。为此，提出了一种基于深度强化学习的多目标资源调度（MODRLRS

Multi objective deep reinforcement learning resource scheduling）算法来调用网络中的计算资源和网络资源，该算法通过构建帕累托最优解集的方法对算网资源进行多目标调度优化以满足不同租户的个性化业务需求。仿真对比实验表明，相比其他多目标资源调度算法，新算法提升了4.9%的请求接受率和4.78%的符合时延请求率，能够灵活适应各种计算业务的独特需求。

Abstract

With the rapid advancement of intelligent businesses

the pre-existing relationship between traditional network architectures and computing capabilities has made it difficult to meet the current demands

making the implementation of computing-network convergence inevitable. Under the new computing power network framework brought about by the convergence of computing networks

efficient and intelligent resource scheduling strategy has become a key link to improve user experience. However

the existing resource scheduling algorithms have a single optimization objective and cannot meet the differentiated business needs of multi-tenants. To this end

a Multi objective deep reinforcement learning resource scheduling (MODRLRS) was proposed to call the computing resources and network resources in the computing power network. The algorithm performs multi-objective scheduling optimization of computing network resources by constructing a Pareto optimal solution set to meet the personalized business needs of different tenants. Simulation experimental results show that compared with other multi-objective resource scheduling algorithms

the proposed algorithm improves the request acceptance rate by 4.9% and the compliant delay request rate by 4.78%

which can flexibly adapt to the unique requirements of various computing services.

关键词

Keywords

references

雷波 , 刘增义 , 王旭亮 , 等 . 基于云、网、边融合的边缘计算新方案: 算力网络 [J ] . 电信科学 , 2019 , 35 ( 9 ): 44 - 51 .

LEI B , LIU Z Y , WANG X L , et al . Computing network: a new multi-access edge computing [J ] . Telecommunications Science , 2019 , 35 ( 9 ): 44 - 51 .

郭凤仙 , 孙耀华 , 彭木根 . 6G算力网络: 体系架构与关键技术 [J ] . 无线电通信技术 , 2023 , 49 ( 1 ): 21 - 30 .

GUO F X , SUN Y H , PENG M G . Computing force networks in 6G: architecture and key technologies [J ] . Radio Communications Technology , 2023 , 49 ( 1 ): 21 - 30 .

刘宇航 , 张菲 . 计算概念谱系: 算势、算力、算术、算法、算礼 [J ] . 中国科学院院刊 , 2022 , 37 ( 10 ): 1500 - 1510 .

LIU Y H , ZHANG F . Computing concept genealogy: potential, power, arithmetic, algorithm and ritual of computation [J ] . Bulletin of Chinese Academy of Sciences , 2022 , 37 ( 10 ): 1500 - 1510 .

张宏科 , 权伟 , 刘康 . 算力网络研究与探索 [J ] . 中兴通讯技术 , 2023 , 29 ( 1 ): 1 - 5 .

ZHANG H K , QUAN W , LIU K . Research and exploration of computing power network [J ] . ZTE Technology Journal , 2023 , 29 ( 1 ): 1 - 5 .

陈晓红 , 许冠英 , 徐雪松 , 等 . 我国算力服务体系构建及路径研究 [J ] . 中国工程科学 , 2023 , 25 ( 6 ): 49 - 60 .

CHEN X H , XU G Y , XU X S , et al . Computing power service system of China and its development path [J ] . Strategic Study of CAE , 2023 , 25 ( 6 ): 49 - 60 .

梁芳 , 佟恬 , 马贺荣 , 等 . 东数西算下算力网络发展分析 [J ] . 信息通信技术与政策 , 2022 ( 11 ): 79 - 83 .

LIANG F , TONG T , MA H R , et al . Analysis of the development of the computing power network under east-data-west-computing project [J ] . Information and Communications Technology and Policy , 2022 ( 11 ): 79 - 8 .

ITU-T . Computing power network-framework and architecture: Y.2501 [S ] . 2021 .

杨帆 , 宋闻萱 , 许方敏 , 等 . 工业互联网算网一体技术研究 [J ] . 无线电通信技术 , 2023 , 49 ( 1 ): 63 - 71 .

YANG F , SONG W X , XU F M , et al . Research on the application of computing force network technology in industrial Internet of things [J ] . Radio Communications Technology , 2023 , 49 ( 1 ): 63 - 71 .

王少鹏 , 邱奔 . 算网协同对算力产业发展的影响分析 [J ] . 信息通信技术与政策 , 2022 ( 3 ): 29 - 33 .

WANG S P , QIU B . Analysis on the impact of computing network collaboration on the development of computing power industry [J ] . Information and Communications Technology and Policy , 2022 ( 3 ): 29 - 33 .

王淑玲 , 孙杰 , 王鹏 , 等 . 云边协同中的资源调度优化 [J ] . 电信科学 , 2023 , 39 ( 2 ): 163 - 170 .

WANG S L , SUN J , WANG P , et al . Resource scheduling optimization in cloud-edge collaboration [J ] . Telecommunications Science , 2023 , 39 ( 2 ): 163 - 170 .

HAZRA A , RANA P , ADHIKARI M , et al . Fog computing for next-generation Internet of things: fundamental, state-of-the-art and research challenges [J ] . Computer Science Review , 2023 , 48 : 100549 .

WU Y L , DAI H N , WANG H Z , et al . A survey of intelligent network slicing management for industrial IoT: integrated approaches for smart transportation, smart energy, and smart factory [J ] . IEEE Communications Surveys & Tutorials , 2022 , 24 ( 2 ): 1175 - 1211 .

XIONG Z H , ZHANG Y , LUONG N C , et al . The best of both worlds: a general architecture for data management in blockchain-enabled Internet-of-things [J ] . IEEE Network , 2020 , 34 ( 1 ): 166 - 173 .

ITU-T . Signalling requirements for service deployment in computing power network : Q.4140 [S ] . 2023 .

CUI Y Y , ZHANG D G , ZHANG T , et al . A novel offloading scheduling method for mobile application in mobile edge computing [J ] . Wireless Networks , 2022 , 28 ( 6 ): 2345 - 2363 .

JAMIL B , IJAZ H , SHOJAFAR M , et al . Resource allocation and task scheduling in fog computing and Internet of everything environments: a taxonomy, review, and future directions [J ] . ACM Computing Surveys , 2022 , 54 ( 11 s): 1 - 38 .

ADDYA S K , SATPATHY A , GHOSH B C , et al . CoMCLOUD: virtual machine coalition for multi-tier applications over multi-cloud environments [J ] . IEEE Transactions on Cloud Computing , 2023 , 11 ( 1 ): 956 - 970 .

FARHADI V , MEHMETI F , HE T , et al . Service placement and request scheduling for data-intensive applications in edge clouds [J ] . IEEE/ACM Transactions on Networking , 2021 , 29 ( 2 ): 779 - 792 .

GAO Y F , YAN Z B , ZHAO K L , et al . Joint optimization of server and service selection in satellite-terrestrial integrated edge computing networks [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 2 ): 2740 - 2754 .

BAHRAMI B , KHAYYAMBASHI M R , MIRJALILI S . Multiobjective placement of edge servers in MEC environment using a hybrid algorithm based on NSGA-II and MOPSO [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 18 ): 29819 - 29837 .

ABDELMONEEM R M , BENSLIMANE A , SHAABAN E . Mobility-aware task scheduling in cloud-Fog IoT-based healthcare architectures [J ] . Computer Networks , 2020 , 179 : 107348 .

NI L N , ZHANG J Q , JIANG C J , et al . Resource allocation strategy in fog computing based on priced timed petri nets [J ] . IEEE Internet of Things Journal , 2017 , 4 ( 5 ): 1216 - 1228 .

张维庭 , 孙呈蕙 , 王洪超 , 等 . 算网资源智能适配与融合调度方法 [J ] . 电信科学 , 2023 , 39 ( 9 ): 12 - 20 .

ZHANG W T , SUN C H , WANG H C , et al . Intelligent adaptation and integrated scheduling method for computing and networking resources [J ] . Telecommunications Science , 2023 , 39 ( 9 ): 12 - 20 .

ZHAO X Y , ZONG Q , TIAN B L , et al . Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning [J ] . Aerospace Science and Technology , 2019 , 92 : 588 - 594 .

WU Q , WANG W H , FAN P Y , et al . Cooperative edge caching based on elastic federated and multi-agent deep reinforcement learning in next-generation networks [J ] . IEEE Transactions on Network and Service Management , 2024 , 21 ( 4 ): 4179 - 4196 .

WANG X B , WU Q , FAN P Y , et al . Vehicle selection for C-V2X mode 4-based federated edge learning systems [J ] . IEEE Systems Journal , 2024 , 18 ( 4 ): 1927 - 1938 .

ZHU H B . Pareto improvement: a GRA perspective [J ] . IEEE Transactions on Computational Social Systems , 2023 , 10 ( 3 ): 1241 - 1251 .

FAN Y Q , LIU J H , YE H , et al . TA-LSTM: a time and attribute aware LSTM for deep flight track clustering [J ] . IEEE Transactions on Aerospace and Electronic Systems , 2023 , 59 ( 5 ): 7047 - 7060 .

CAO K , WENG J . REPFS: reliability-ensured personalized function scheduling in sustainable serverless edge computing [J ] . IEEE Transactions on Sustainable Computing , 2024 , 9 ( 3 ): 494 - 511 .

PENG K , HUANG H L , ZHAO B H , et al . Intelligent computation offloading and resource allocation in IIoT with end-edge-cloud computing using NSGA-III [J ] . IEEE Transactions on Network Science and Engineering , 2023 , 10 ( 5 ): 3032 - 3046 .

WANG H Z , TANG L X , XIAO M , et al . Multi-objective optimization for joint communication and computing resource allocation in NOMA-based MEC system [C ] // Proceedings of the 2024 IEEE Congress on Evolutionary Computation (CEC) . Piscataway : IEEE Press , 2024 : 1 - 8 .

WANG S M , SONG X Q , XU H , et al . Joint offloading decision and resource allocation in vehicular edge computing networks [J ] . Digital Communications and Networks , 2023 .

Views

326

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Graph reducing method based on deep reinforcement learning

Intent-driven dual-time-scale resource allocation approach for Internet of things

Deep diffusion deterministic policy gradient based performance optimization for Wi-Fi network

Related Author

Sun Yanfei

Dong Zhenjiang

Gao Yu

Liu Yali

Qi Jin

Chen Genxin

Wang Jiansu

Song Hang

Related Institution

School of Computer Science, Nanjing University of Posts and Telecommunications

Nanjing Big Data Security Technology Co., Ltd.

School of Internet of Things, Nanjing University of Posts and Telecommunications

School of Automation, Nanjing University of Posts and Telecommunications

State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications

⁰