Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things

Jun SUN; Shangweikang ZHAO

doi:10.11959/j.issn.2096-3750.2022.00280

您当前的位置：

首页 >

文章列表页 >

Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things

Theory and Technology | 更新时间：2024-06-05

- Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things
- Chinese Journal on Internet of Things Vol. 6, Issue 3, Pages: 82-90(2022)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2022.00280
  CLC： TN929.5
- Published：30 September 2022，
  
  Published Online：2022-09，
- 稿件说明：
移动端阅览
JUN SUN, SHANGWEIKANG ZHAO. Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things. [J]. Chinese journal on internet of things, 2022, 6(3): 82-90.
DOI：

JUN SUN, SHANGWEIKANG ZHAO. Energy-saving computation offloading scheme based on Sarsa algorithm in industrial internet of things. [J]. Chinese journal on internet of things, 2022, 6(3): 82-90. DOI： 10.11959/j.issn.2096-3750.2022.00280.

摘要

针对任务有完成截止时间要求的工业物联网系统，为了降低该类系统的总能源损耗，提出一种基于Sarsa算法的节能计算卸载方案。基于Sarsa算法的特点对能耗优化问题进行了解耦，由Sarsa算法进行外部状态值函数的迭代，选择合适的边缘计算服务器进行计算任务卸载，再由粒子群优化算法解决其中的资源分配问题，最后更新环境信息进行下一轮算法迭代，直至算法满足结束条件。仿真结果表明，本方案具有更快的收敛速度并且有效降低了系统能耗。

Abstract

In order to reduce the total energy consumption of industrial internet of things systems with deadline requirements

a computing offloading scheme based on Sarsa algorithm was proposed.Based on the characteristics of Sarsa algorithm

the energy consumption optimization problem was coupled.The Sarsa algorithm iterated the external state value function

selected the appropriate edge computing server to offload the computing task

and then the particle swarm optimization algorithm solved the resource allocation problem.Finally

the environmental information was updated for the next round of algorithm iteration until the algorithm met the end conditions.The simulation results show that this scheme has faster convergence speed and effectively reduces the system energy consumption.

关键词

工业物联网边缘计算Sarsa节能计算卸载资源分配

Keywords

industrial internet of thingsedge computingSarsaenergy-saving computation offloadingresource allocation

references

MAO W L, ZHAO Z W, CHANG Z ,et al. Energy-efficient industrial Internet of things,overview and open issues[J]. IEEE Transactions on Industrial Informatics, 2021,17(11): 7225-7237.

姚鑫 . 软件定义工业物联网计算卸载技术研究[D]. 成都,电子科技大学, 2019.

YAO X . A research on software-defined industrial Internet of things computing offloading technology[D]. Chengdu,University of Electronic Science and Technology of China, 2019.

CHEN B T, WAN J F, SHU L ,et al. Smart factory of industry 4.0,key technologies,application case,and challenges[J]. IEEE Access, 2018,6: 6505-6519.

YOUNAN M N, HOUSSEIN E H, ELHOSENY M ,et al. Challenges and recommended technologies for the industrial Internet of things,a comprehensive review[J]. Measurement, 2020,151,107198.

刘斐, 曹钰杰, 章国安 . 车联网场景下移动边缘计算协作式资源分配策略[J]. 电讯技术, 2021,61(7): 858-864.

LIU F, CAO Y J, ZHANG G A . Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021,61(7): 858-864.

代振楠 . 工业互联网中基于边缘计算的任务卸载策略研究[D]. 长沙:湖南大学, 2019.

DAI Z N . Research on task offloading strategy based on edge computing for industrial Internet[D]. Changsha,Hunan University, 2019.

DUAN X T, XU F,SUNYY . Research on offloading strategy in edge computing of Internet of things[C]// Proceedings of 2020 International Conference on Computer Network,Electronic and Automation (ICCNEA). Piscataway,IEEE Press, 2020: 206-210.

HOU X W, REN Z Y, YANG K ,et al. IIoT-MEC,a novel mobile edge computing framework for 5G-enabled IIoT[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.

MENG J Y, TAN H S, XU C ,et al. Dedas,online task dispatching and scheduling with bandwidth constraint in edge computing[C]// Proceedings of IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway,IEEE Press, 2019: 2287-2295.

LI M, YU F R, SI P B ,et al. Energy-efficient machine-to-machine (M2M) communications in virtualized cellular networks with mobile edge computing (MEC)[J]. IEEE Transactions on Mobile Computing, 2019,18(7): 1541-1555.

JIANG C F, CHENG X L, GAO H H ,et al. Toward computation offloading in edge computing,asurvey[J]. IEEE Access, 7: 131543-131558.

HE X, DOU W C . Offloading deadline-aware task in edge computing[C]// Proceedings of 2020 IEEE 13th International Conference on Cloud Computing. Piscataway,IEEE Press, 2020: 28-30.

黄永明, 郑冲, 张征明 ,等. 大规模无线通信网络移动边缘计算和缓存研究[J]. 通信学报, 2021,42(4): 44-61.

HUANG Y M, ZHENG C, ZHANG Z M ,et al. Research on mobile edge computing and caching in massive wireless communication network[J]. Journal on Communications, 2021,42(4): 44-61.

LI Z, ZHOU X, QIN Y F . A survey of mobile edge computing in the industrial Internet[C]// Proceedings of 2019 7th International Conference on Information,Communication and Networks (ICICN). Piscataway,IEEE Press, 2019: 94-98.

安宜豪 . 移动边缘计算中面向低功耗的任务迁移问题研究[D]. 哈尔滨,哈尔滨工业大学, 2019.

AN Y H . Research on task offloading for low-power in MEC[D]. Harbin,Harbin Institute of Technology, 2019.

LYU X C, TIAN H, SENGUL C ,et al. Multiuser joint task offloading and resource optimization in proximate clouds[J]. IEEE Transactions on Vehicular Technology, 2017,66(4): 3435-3447.

WANG C M, LIANG CC, YU F R ,et al. Computation offloading and resource allocation in wireless cellular networks with mobile edge computing[J]. IEEE Transactions on Wireless Communications, 2017,16(8): 4924-4938.

BOYD S . Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends® in Machine Learning, 2010,3(1): 1-122.

YOU C S, HUANG K B, CHAE H ,et al. Energy-efficient resource allocation for mobile-edge computation offloading[J]. IEEE Transactions on Wireless Communications, 2017,16(3): 1397-1411.

EL HABER E, NGUYEN T M, ASSI C ,et al. An energy-efficient task offloading solution for MEC-based IoT in ultra-dense networks[C]// Proceedings of 2019 IEEE Wireless Communications and Networking Conference. Piscataway,IEEE Press, 2019: 1-7.

LI Q P, ZHAO J H, GONG Y ,et al. Energy-efficient computation offloading and resource allocation in fog computing for Internet of Everything[J]. China Communications, 2019,16(3): 32-41.

季子豪, 江凌云 . 一种基于遗传算法的多站点协同计算卸载算法[J]. 计算机工程与科学, 2021,43(3): 426-434.

JI Z H, JIANG L Y . A genetic-based multi-site collaborative computation offloading algorithm[J]. Computer Engineering ＆Science, 2021,43(3): 426-434.

KE H C, WANG J, WANG H ,et al. Joint optimization of data offloading and resource allocation with renewable energy aware for IoT devices,a deep reinforcement learning approach[J]. IEEE Access, 7: 179349-179363.

张延华, 杨乐, 李萌 ,等. 基于 Q-learning 的工业互联网资源优化调度[J]. 北京工业大学学报, 2020,46(11): 1213-1221.

ZHANG Y H, YANG L, LI M ,et al. Optimization of resource allocation for industrial Internet based on Q-learning[J]. Journal of Beijing University of Technology, 2020,46(11): 1213-1221.

石雪琴 . MEC 网络中联合优化计算卸载算法研究[D]. 重庆,重庆邮电大学, 2020.

SHI X Q . Research on joint optimization of computation offloading in MEC network[D]. Chongqing,Chongqing University of Posts and Telecommunications, 2020.

程百川 . 基于深度强化学习的 MEC 计算卸载和资源分配研究[D]. 北京,北京邮电大学, 2019.

CHENG B C . Research on MEC computing offloading and resource allocation based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.

李季 . 基于深度强化学习的移动边缘计算中的计算卸载与资源分配算法研究与实现[D]. 北京,北京邮电大学, 2019.

LI J . Research and implementation of computation offloading and resource allocation algorithm in mobile edge computing based on deep reinforcement learning[D]. Beijing,Beijing University of Posts and Telecommunications, 2019.

李超 . 基于深度强化学习的移动边缘计算卸载策略及其物理层安全研究[D]. 广州,广州大学, 2020.

LI C . Research on mobile edge unloading strategy based on deep reinforcement learning and its physical layer security[D]. Guangzhou:Guangzhou University, 2020.

KHALIL R A, SAEED N, MASOOD M ,et al. Deep learning in the industrial Internet of Things,potentials,challenges,and emerging applications[J]. IEEE Internet of Things Journal, 2021,8(14): 11016-11040.

黄冬晴 . 移动边缘计算中联合计算卸载和资源分配策略研究[D]. 上海,华东师范大学, 2021.

HUANG D Q . Research on joint computation offloading and resource allocation strategy in mobile edge computing[D]. Shanghai,East China Normal University, 2021.

ALFAKIH T, HASSAN M M, GUMAEI A ,et al. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA[J]. IEEE Access, 8: 54074-54084.

LI Y Y,FADDAE, MANERBA D, et al . Reinforcement learning algorithms for online single-machine scheduling[C]// Proceedings of the 2020 Federated Conference on Computer Science and Information Systems","Annals of Computer Science and Information Systems. Piscataway,IEEE Press, 2020: 277-283.

AHSAN W, YI W Q, QIN Z J ,et al. Resource allocation in uplink NOMA-IoT networks,a reinforcement-learning approach[J]. IEEE Transactions on Wireless Communications, 2021,20(8): 5083-5098.

JIANG H, GUI R J, CHEN Z ,et al. An improved sarsa(-) reinforcement learning algorithm for wireless communication systems[J]. IEEE Access, 7: 115418-115427.

KUCHIBHOTLA V, HARSHITHA P, GOYAL S . An N-step look ahead algorithm using mixed (on and off) policy reinforcement learning[C]// Proceedings of 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). Piscataway,IEEE Press, 2020: 677-681.

KENNEDY J, EBERHART R . Particle swarm optimization[C]// Proc eedings of ICNN'95 - International Conference on Neural Networks. Piscataway,IEEE Press, 1995: 1942-1948.

Views

415

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Resource allocation based on optimal transport theory in IoT edge computing

Cross-domain task offloading and computing resource allocation for edge computation in industrial Internet of things

A sensing and control closed-loop scheduling strategy for UAV swarms

Research on access control strategy for a CR-RSMA network

Construction and optimization of a smart classroom system for emotional computing

Related Author

Qi ZHANG

Yuna JIANG

Xiaohu GE

Yonghui LI

Peng ZHOU

Jincheng XU

Bo YANG

PING Yuqi

Related Institution

The University of Sydney

Huazhong University of Science and Technology

Key Laboratory of System Control and Information Processing,Ministry of Education

Department of Automation,Shanghai Jiao Tong University

School of Information Science and Technology, Harbin Institute of Technology

⁰