PD-TD3：高速公路场景下边路协同计算卸载策略

刘毅; 杨琪; 李国燕; 何军; 张明辉

doi:10.11959/j.issn.2096-3750.2025.00490

您当前的位置：

首页 >

文章列表页 >

PD-TD3：高速公路场景下边路协同计算卸载策略

智慧交通物联网前沿与进展 | 更新时间：2025-07-29

- PD-TD3：高速公路场景下边路协同计算卸载策略
- PD-TD3: edge-cloud collaborative computation offloading strategy in highway scenarios
- 物联网学报 2025年9卷第2期页码：39-50
- 作者机构：
  
  天津城建大学计算机与信息工程学院，天津 300384
- 作者简介：
  
  [ "刘毅（1972‒ ），男，博士，天津城建大学计算机与信息工程学院教授，主要研究方向为微机检测与控制、智能控制理论。" ]
  [ "杨琪（2000‒ ），女，天津城建大学计算机与信息工程学院硕士生，主要研究方向为车联网移动边缘计算卸载。" ]
  [ "李国燕（1984‒ ），女，博士，天津城建大学计算机与信息工程学院副教授、副院长，主要研究方向为下一代网络技术。" ]
  [ "何军（1975‒ ），男，天津城建大学计算机与信息工程学院讲师，主要研究方向为软件工程、大数据分析与应用。" ]
  [ "张明辉（1994‒ ），女，博士，天津城建大学计算机与信息工程学院讲师，主要研究方向为机器学习、智能信息处理技术。" ]
- 基金信息：
  
  国家自然科学基金资助项目(61876131);天津市研究生科研创新资助项目(2022SKYZ391)
- DOI：10.11959/j.issn.2096-3750.2025.00490
  中图分类号： U495;
- 收稿日期：2025-02-28，
  
  修回日期：2025-03-27，
  
  纸质出版日期：2025-06-10
- 稿件说明：
移动端阅览
刘毅,杨琪,李国燕等.PD-TD3：高速公路场景下边路协同计算卸载策略[J].物联网学报,2025,09(02):39-50.

LIU Yi,YANG Qi,LI Guoyan,et al.PD-TD3: edge-cloud collaborative computation offloading strategy in highway scenarios[J].Chinese Journal on Internet of Things,2025,09(02):39-50.
刘毅,杨琪,李国燕等.PD-TD3：高速公路场景下边路协同计算卸载策略[J].物联网学报,2025,09(02):39-50. DOI： 10.11959/j.issn.2096-3750.2025.00490.

LIU Yi,YANG Qi,LI Guoyan,et al.PD-TD3: edge-cloud collaborative computation offloading strategy in highway scenarios[J].Chinese Journal on Internet of Things,2025,09(02):39-50. DOI： 10.11959/j.issn.2096-3750.2025.00490.

摘要

针对高速公路场景下，现有卸载模型忽视车辆高速移动导致的网络动态变化引起的高时延和能耗，且算法在降低时延和能耗方面效力不足的问题，提出PD-TD3（twin delayed deep deterministic policy gradient with prioritized double buffer pool experience replay）卸载策略方案。首先，搭建高速公路3层分布式卸载模型；然后，将计算卸载问题转化为马尔可夫最优策略问题求解，以均衡优化时延与能耗建立奖励函数，以最大化奖励函数作为优化目标；最后，改进TD3（twin delayed deep deterministic policy gradient）算法中收敛较慢且不稳定、Q值低估偏差、抽取经验效率低的问题，提出PD-TD3算法求解最优化问题。仿真实验结果表明，与TD3算法相比，PD-TD3算法有效提升了早期算法探索效率，并且有效降低了计算卸载的时延约50%、能耗约70%。

Abstract

In the highway scenarios

existing offloading models often overlook the network dynamics caused by the high-speed movement of vehicles

leading to increased latency and energy consumption

and exhibit insufficient effectiveness in reducing latency and energy consumption. To address these challenges

an offloading strategy utilizing the prioritized double-buffer pool experience replay twin delayed deep deterministic policy gradient (PD-TD3) algorithm was proposed. Initially

a three-layer distributed offloading model tailored for highway environments was developed. Subsequently

the computation offloading problem was formulated as a Markov decision process (MDP)

with the reward function designed to optimize the trade-off between latency and energy consumption

aiming to maximize the reward. To address the limitations of the traditional TD3 algorithm

including slow convergence

Q-value underestimation bias

and inefficient experience sampling

the PD-TD3 algorithm was introduced to solve the optimization problem. Simulation results indicate that

compared with the TD3 algorithm

the PD-TD3 algorithm can effectively improve the efficiency of early algorithm exploration and effectively reduces computation offloading latency by approximately 50% and energy consumption by about 70%.

关键词

Keywords

references

WANG X J , ZHU H L , NING Z L , et al . Blockchain intelligence for Internet of vehicles: challenges and solutions [J ] . IEEE Communications Surveys & Tutorials , 2023 , 25 ( 4 ): 2325 - 2355 .

HEMMATI A , ZAREI M , RAHMANI A M . Big data challenges and opportunities in Internet of vehicles: a systematic review [J ] . International Journal of Pervasive Computing and Communications , 2024 , 20 ( 2 ): 308 - 342 .

李智勇 , 王琦 , 陈一凡 , 等 . 车辆边缘计算环境下任务卸载研究综述 [J ] . 计算机学报 , 2021 , 44 ( 5 ): 963 - 982 .

LI Z Y , WANG Q , CHEN Y F , et al . A survey on task offloading research in vehicular edge computing [J ] . Chinese Journal of Computers , 2021 , 44 ( 5 ): 963 - 982 .

LI Z P , YU H Q , FAN G S , et al . Energy-efficient offloading for DNN-based applications in edge-cloud computing: a hybrid chaotic evolutionary approach [J ] . Journal of Parallel and Distributed Computing , 2024 , 187 : 104850 .

SEN P , PANDIT R , SARDDAR D . A survey on methods and apparatus of offloading in mobile cloud computing [J ] . International Journal of System of Systems Engineering , 2024 , 14 ( 2 ): 212 - 225 .

ABBAS N , ZHANG Y , TAHERKORDI A , et al . Mobile edge computing: a survey [J ] . IEEE Internet of Things Journal , 2018 , 5 ( 1 ): 450 - 465 .

SABELLA D , VAILLANT A , KUURE P , et al . Mobile-edge computing architecture: the role of MEC in the Internet of things [J ] . IEEE Consumer Electronics Magazine , 2016 , 5 ( 4 ): 84 - 91 .

ZHENG D S , CHEN Y Y , WEI L , et al . Dynamic NOMA-based computation offloading in vehicular platoons [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 10 ): 13000 - 13010 .

XUANWEN , SUN H M . Parking cooperation-based mobile edge computing using task offloading strategy [J ] . Journal of Grid Computing , 2024 , 22 : 8 .

MAO Q C , CHENG J J , ZHOU M C , et al . An edge computing-based autonomous vehicle group cooperation model in a highway scene [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 7 ): 9682 - 9695 .

李国燕 , 薛翔 , 刘毅 , 等 . 改进TD3的SDN车联网边缘计算卸载策略 [J ] . 计算机集成制造系统 , 2023 , 29 ( 5 ): 1627 - 1634 .

LI G Y , XUE X , LIU Y , et al . Improved TD3 edge computing offloading strategy for software defined networking Internet of vehicles [J ] . Computer Integrated Manufacturing Systems , 2023 , 29 ( 5 ): 1627 - 1634 .

刘婷 , 罗喜良 . 移动边缘计算中的在线任务卸载方法 [J ] . 中国科学院大学学报 , 2022 , 39 ( 2 ): 267 - 274 .

LIU T , LUO X L . Online task offloading in mobile edge computing [J ] . Journal of University of Chinese Academy of Sciences , 2022 , 39 ( 2 ): 267 - 274 .

WEI Z L , ZHAO B K , SU J S . Event-driven computation offloading in IoT with edge computing [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 9 ): 6847 - 6860 .

WU Q , WANG S , GE H , et al . Delay-sensitive task offloading in vehicular fog computing-assisted platoons [J ] . IEEE Transactions on Control Systems Technology , 2023 .

LIAO Z F , PENG J S , XIONG B , et al . Adaptive offloading in mobile-edge computing for ultra-dense cellular networks based on genetic algorithm [J ] . Journal of Cloud Computing , 2021 , 10 : 15 .

杨文杰 , 巨涛 , 杨阳 , 等 . 面向边缘计算的人工鱼群搜索任务调度 [J ] . 电子测量与仪器学报 , 2022 , 36 ( 11 ): 149 - 159 .

YANG W J , JU T , YANG Y , et al . Artificial fish swarm search task scheduling for edge computing [J ] . Journal of Electronic Measurement and Instrumentation , 2022 , 36 ( 11 ): 149 - 159 .

DONG S , XIA Y J , KAMRUZZAMAN J . Quantum particle swarm optimization for task offloading in mobile edge computing [J ] . IEEE Transactions on Industrial Informatics , 2023 , 19 ( 8 ): 9113 - 9122 .

ALQARNI M A , MOUSA M H , HUSSEIN M K . Task offloading using GPU-based particle swarm optimization for high-performance vehicular edge computing [J ] . Journal of King Saud University-Computer and Information Sciences , 2022 , 34 ( 10 ): 10356 - 10364 .

CHEN Z Y , ZHENG H Q , ZHANG J S , et al . Joint computation offloading and deployment optimization in multi-UAV-enabled MEC systems [J ] . Peer-to-Peer Networking and Applications , 2022 , 15 ( 1 ): 194 - 205 .

ZENG F , CHEN Q , MENG L , et al . Volunteer assisted collaborative offloading and resource allocation in vehicular edge computing [J ] . IEEE Transactions on Intelligent Transportation Systems , 2021 , 22 ( 6 ): 3247 - 3257 .

ZHAN Y F , GUO S , LI P , et al . A deep reinforcement learning based offloading game in edge computing [J ] . IEEE Transactions on Computers , 2020 , 69 ( 6 ): 883 - 893 .

DAI F , LIU G Z , MO Q , et al . Task offloading for vehicular edge computing with edge-cloud cooperation [J ] . World Wide Web , 2022 , 25 ( 5 ): 1999 - 2017 .

ZHOU H , JIANG K , LIU X X , et al . Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 2 ): 1517 - 1530 .

WANG T , LUO X , ZHAO W B . Improving the performance of tasks offloading for Internet of vehicles via deep reinforcement learning methods [J ] . IET Communications , 2022 , 16 ( 10 ): 1230 - 1240 .

GAO H H , WANG X J , WEI W , et al . Com-DDPG: task offloading based on multiagent reinforcement learning for information-communication-enhanced mobile edge computing in the Internet of vehicles [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 1 ): 348 - 361 .

YAO L , XU X L , BILAL M , et al . Dynamic edge computation offloading for Internet of vehicles with deep reinforcement learning [J ] . IEEE Transactions on Intelligent Transportation Systems , 2023 , 24 ( 11 ): 12991 - 12999 .

LI C X , WANG H , SONG R F . Mobility-aware offloading and resource allocation in NOMA-MEC systems via DC [J ] . IEEE Communications Letters , 2022 , 26 ( 5 ): 1091 - 1095 .

ZHOU T Q , XU M , QIN D , et al . Computing offloading based on TD3 algorithm in cache-assisted vehicular NOMA–MEC networks [J ] . Sensors , 2023 , 23 ( 22 ): 9064 .

WAKGRA F G , YAHYA W , KAR B , et al . Ratio-based offloading optimization for edge and vehicular-fog federated systems: a multi-agent TD3 approach [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 11 ): 17684 - 17696 .

DONG J D , PAN K , ZHENG C X , et al . A dual-agent approach for coordinated task offloading and resource allocation in MEC [J ] . Journal of Electrical and Computer Engineering , 2023 , 2023 ( 1 ): 6134837 .

SAITO Y , KISHIYAMA Y , BENJEBBOUR A , et al . Non-orthogonal multiple access (NOMA) for cellular future radio access [C ] // Proceedings of the 2013 IEEE 77th Vehicular Technology Conference (VTC Spring) . Piscataway : IEEE Press , 2013 : 1 - 5 .

CHEN Z , WANG X . Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach [J ] . EURASIP Journal on Wireless Communications and Networking , 2020 , 2020( 1 ): 1 - 21 .

LUO L P , LI Q Z , CHENG J L . Performance analysis of overlay cognitive NOMA systems with imperfect successive interference cancellation [J ] . IEEE Transactions on Communications , 2020 , 68 ( 8 ): 4709 - 4722 .

CHOU Y L , LIU S S , CHUNG E Y , et al . An energy and performance efficient DVFS scheme for irregular parallel divide-and-conquer algorithms on the intel SCC [J ] . IEEE Computer Architecture Letters , 2014 , 13 ( 1 ): 13 - 16 .

HE Q , HOU X . Wd3: taming the estimation bias in deep reinforcement learning [C ] // Proceedings of the 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) . Piscataway : IEEE Press , 2020 : 391 - 398 .

QIU Y , ZHOU S Y , XIA D , et al . Local integrated energy system operational optimization considering multi-type uncertainties: a reinforcement learning approach based on improved TD3 algorithm [J ] . IET Renewable Power Generation , 2023 , 17 ( 9 ): 2236 - 2256 .

HE S T , SHEN L Y , WU Q , et al . A certified cubic B-spline interpolation method with tangential direction constraints [J ] . Journal of Systems Science and Complexity , 2024 , 37 ( 3 ): 1271 - 1294 .

ALAM M G R , HASSAN M M , UDDIN M Z , et al . Autonomic computation offloading in mobile edge for IoT applications [J ] . Future Generation Computer Systems , 2019 , 90 : 149 - 157 .

WANG J Y , WANG Y F , CHENG P , et al . DDPG-based joint resource management for latency minimization in NOMA-MEC networks [J ] . IEEE Communications Letters , 2023 , 27 ( 7 ): 1814 - 1818 .

ZHAN M , CHEN J , DU C , et al . Twin delayed multi-agent deep deterministic policy gradient [C ] // Proceedings of the 2021 IEEE International Conference on Progress in Informatics and Computing (PIC) . Piscataway : IEEE Press , 2021 : 48 - 52 .

LI S Y , HU X H , DU Y W . Deep reinforcement learning for computation offloading and resource allocation in unmanned-aerial-vehicle assisted edge computing [J ] . Sensors , 2021 , 21 ( 19 ): 6499 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的多租户算网资源分配算法

基于能量收集技术的协作卸载计算方案

基于连续动作空间深度强化学习的多数据融合室内定位方法

基于深度强化学习的SDN服务质量智能优化算法

基于深度强化学习的智能车间调度方法研究