车联网中基于迁移强化学习的跨域充电站推荐算法

林海; 赵家仪; 曹越; 苏航宇; 王丽园

doi:10.11959/j.issn.2096-3750.2025.00462

您当前的位置：

首页 >

文章列表页 >

车联网中基于迁移强化学习的跨域充电站推荐算法

理论与技术 | 更新时间：2026-01-26

- 车联网中基于迁移强化学习的跨域充电站推荐算法
- A transfer reinforcement learning-based approach for cross-domain charging station recommendation in the Internet of vehicles
- 物联网学报 2025年9卷第3期页码：37-47
- 作者机构：
  
  1.武汉大学国家网络安全学院，湖北武汉 430072
  2.中交第二公路勘察设计研究院有限公司，湖北武汉 430056
- 作者简介：
  
  [ "林海（1976‒ ），男，博士，武汉大学国家网络安全学院副教授，主要研究方向为计算机网络。" ]
  [ "赵家仪（2000‒ ），女，武汉大学国家网络安全学院硕士生，主要研究方向为车联网、强化学习、数据融合。" ]
  [ "曹越（1984‒ ），男，博士，武汉大学国家网络安全学院教授、系主任，主要研究方向为网络安全。" ]
  [ "苏航宇（2002‒ ），男，武汉大学国家网络安全学院硕士生，主要研究方向为车联网、迁移学习、人工智能。" ]
  [ "王丽园（1980‒ ），女，中交第二公路勘察设计研究院有限公司正高级工程师、首席研究员，主要研究方向为公路智慧交通技术。" ]
- 基金信息：
  
  国家重点研发计划项目(2023YFB3907105);湖北省重点研发计划项目(2023BAB022)
- DOI：10.11959/j.issn.2096-3750.2025.00462
  中图分类号： TN915.08
- 收稿：2024-10-13，
  
  修回：2025-05-30，
  
  纸质出版：2025-09-10
- 稿件说明：
移动端阅览
林海,赵家仪,曹越等.车联网中基于迁移强化学习的跨域充电站推荐算法[J].物联网学报,2025,09(03):37-47.

LIN Hai,ZHAO Jiayi,CAO Yue,et al.A transfer reinforcement learning-based approach for cross-domain charging station recommendation in the Internet of vehicles[J].Chinese Journal on Internet of Things,2025,09(03):37-47.
林海,赵家仪,曹越等.车联网中基于迁移强化学习的跨域充电站推荐算法[J].物联网学报,2025,09(03):37-47. DOI： 10.11959/j.issn.2096-3750.2025.00462.

LIN Hai,ZHAO Jiayi,CAO Yue,et al.A transfer reinforcement learning-based approach for cross-domain charging station recommendation in the Internet of vehicles[J].Chinese Journal on Internet of Things,2025,09(03):37-47. DOI： 10.11959/j.issn.2096-3750.2025.00462.

摘要

深度强化学习已广泛应用于车联网充电站推荐，但传统方法通常需要为每个区域单独训练神经网络，增加了计算负担和数据需求。迁移学习通过利用先前任务的知识加速新任务学习，减少重复训练。为此，提出了基于迁移强化学习的跨域充电站推荐算法。该算法引入嵌入编码器对齐源域和目标域中系统状态空间和动作空间的维度，有效解决了维度差异问题。同时，该算法基于互信息构造变分分布，最大化对齐前后目标域状态相似度，确保迁移有效性。相较3个典型的充电站推荐算法，在低维向高维迁移中，该算法平均总充电时间分别减少57.6%、59.3%和7.1%；在高维向低维迁移中，分别减少12.3%、40.8%和4.7%。仿真结果证明该算法具备较强的迁移性，显著提升了跨域充电站推荐系统的性能。

Abstract

Deep reinforcement learning has been widely applied in charging station recommendations in the internet of vehicles

but training separate neural networks for each region are often required by traditional methods

leading to increased computational load and data demands. Transfer learning accelerates the learning process for new tasks by leveraging knowledge from previous tasks

thus reducing redundant training. Therefore

a transfer reinforcement learning-based cross-domain charging station recommendation algorithm was proposed. An embedding encoder was introduced by this algorithm to align the system state and action space dimensions between the source and target domains

effectively solving the dimensionality discrepancy problem. Additionally

variational distributions were constructed based on mutual information to maximize the similarity between pre-aligned and post-aligned target domain states to ensure effective transfer. Compared to three typical charging station recommendation algorithms

in the low-dimensional to high-dimensional transfer

the average total charging time of the proposed algorithm was reduced by 57.6%

59.3%

and 7.1%. In the high-dimensional to low-dimensional transfer

the reductions were 12.3%

40.8%

and 4.7%

respectively. Simulation results demonstrate that the proposed algorithm exhibits strong transferability and significantly enhances the performance of cross-domain charging station recommendation systems.

关键词

Keywords

references

熊凯 , 冷甦鹏 , 张可 , 等 . 车联雾计算中的异构接入与资源分配算法研究 [J ] . 物联网学报 , 2019 , 3 ( 2 ): 20 - 27 .

XIONG K , LENG S P , ZHANG K , et al . Research on heterogeneous radio access and resource allocation algorithm in vehicular fog computing [J ] . Chinese Journal on Internet of Things , 2019 , 3 ( 2 ): 20 - 27 .

罗梓珲 , 江呈羚 , 刘亮 , 等 . 基于深度强化学习的智能车间调度方法研究 [J ] . 物联网学报 , 2022 , 6 ( 1 ): 53 - 64 .

LUO Z H , JIANG C L , LIU L , et al . Research on deep reinforcement learning based intelligent shop scheduling method [J ] . Chinese Journal on Internet of Things , 2022 , 6 ( 1 ): 53 - 64 .

WANG X , WANG S , LIANG X X , et al . Deep reinforcement learning: a survey [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2024 , 35 ( 4 ): 5064 - 5078 .

张启阳 , 陈希亮 , 曹雷 , 等 . 深度强化学习中的知识迁移方法研究综述 [J ] . 计算机科学 , 2023 , 50 ( 5 ): 201 - 216 .

ZHANG Q Y , CHEN X L , CAO L , et al . Survey on knowledge transfer method in deep reinforcement learning [J ] . Computer Science , 2023 , 50 ( 5 ): 201 - 216 .

HARIZAJ M , BISHA I , BASHOLLI F . IOT integration of electric vehicle charging infrastructure [J ] . Advanced Engineering Days (AED) , 2023 , 6 : 152 - 155 .

HESTER T , VECERIK M , PIETQUIN O , et al . Deep Q-learning from demonstrations [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Menlo Park : AAAI Press , 2018 : 3223 - 3230 .

ALKAABNEH F , DIABAT A , GAO H O . Benders decomposition for the inventory vehicle routing problem with perishable products and environmental costs [J ] . Computers & Operations Research , 2020 , 113 : 104751 .

WANG H , TAO C , QI J , et al . Avoiding negative transfer for semantic segmentation of remote sensing images [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 4413215 .

CHEN Z H , CHEN C , CHENG Z W , et al . Selective transfer with reinforced transfer network for partial domain adaptation [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2020 : 12703 - 12711 .

WU Z H , PAN S R , CHEN F W , et al . A comprehensive survey on graph neural networks [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 1 ): 4 - 24 .

贺晓 , 王文学 . 基于图嵌入编码形态信息的非均匀多任务强化学习方法 [J ] . 计算机应用研究 , 2024 , 41 ( 4 ): 1022 - 1028 .

HE X , WANG W X . Method for inhomogeneous multi-task reinforcement learning based on morphological information encoding by graph embedding [J ] . Application Research of Computers , 2024 , 41 ( 4 ): 1022 - 1028 .

GUPTA A , DEVIN C , LIU Y X , et al . Learning invariant feature spaces to transfer skills with reinforcement learning [C ] // Proceedings of the International Conference on Learning Representations . Toulon : ICLR Press , 2017 : 1456 - 1469 .

GUO Q L , XIN S J , SUN H B , et al . Rapid-charging navigation of electric vehicles based on real-time power systems and traffic data [J ] . IEEE Transactions on Smart Grid , 2014 , 5 ( 4 ): 1969 - 1979 .

WU H , PANG G K H , CHOY K L , et al . A scheduling and control system for electric vehicle charging at parking lot [C ] // Proceedings of the 2017 11th Asian Control Conference (ASCC) . Piscataway : IEEE Press , 2017 : 13 - 18 .

LIU S H , XIA X , CAO Y , et al . Reservation-based EV charging recommendation concerning charging urgency policy [J ] . Sustainable Cities and Society , 2021 , 74 : 103150 .

TIAN Z Y , JUNG T , WANG Y , et al . Real-time charging station recommendation system for electric-vehicle taxis [J ] . IEEE Transactions on Intelligent Transportation Systems , 2016 , 17 ( 11 ): 3098 - 3109 .

PAREEK S , SUJIL A , RATRA S , et al . Electric vehicle charging station challenges and opportunities: a future perspective [C ] // Proceedings of the 2020 International Conference on Emerging Trends in Communication, Control and Computing (ICONC3) . Piscataway : IEEE Press , 2020 : 1 - 6 .

BI X W , WANG R H , JIA Q . On the speed-varying range of electric vehicles in time-windowed routing problems with en-route partial re-charging [J ] . IEEE Transactions on Consumer Electronics , 2024 , 70 ( 1 ): 3650 - 3657 .

WANG S Y , BI S Z , ZHANG Y A . Reinforcement learning for real-time pricing and scheduling control in EV charging stations [J ] . IEEE Transactions on Industrial Informatics , 2021 , 17 ( 2 ): 849 - 859 .

XING Q , XU Y , CHEN Z , et al . A graph reinforcement learning-based decision-making platform for real-time charging navigation of urban electric vehicles [J ] . IEEE Transactions on Industrial Informatics , 2023 , 19 ( 3 ): 3284 - 3295 .

ZHANG W J , LIU H , WANG F , et al . Intelligent electric vehicle charging recommendation based on multi-agent reinforcement learning [C ] // Proceedings of the Web Conference 2021 . New York : ACM Press , 2021 : 1856 - 1867 .

CUI F F , LIN X X , ZHANG R N , et al . Multi-objective optimal scheduling of charging stations based on deep reinforcement learning [J ] . Frontiers in Energy Research , 2023 , 10 : 1042882 .

丁瑞金 , 高飞飞 , 邢玲 . 基于深度强化学习的物联网智能路由策略 [J ] . 物联网学报 , 2019 , 3 ( 2 ): 56 - 63 .

DING R J , GAO F F , XING L . Intelligent routing strategy in the Internet of things based on deep reinforcement learning [J ] . Chinese Journal on Internet of Things , 2019 , 3 ( 2 ): 56 - 63 .

WANG K , WANG H X , YANG Z H , et al . A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning [J ] . Applied Energy , 2023 , 343 : 121186 .

FOROOTANI A , RASTEGAR M , ZAREIPOUR H . Transfer learning-based framework enhanced by deep generative model for cold-start forecasting of residential EV charging behavior [J ] . IEEE Transactions on Intelligent Vehicles , 2024 , 9 ( 1 ): 190 - 198 .

CAI Y H , LI M X , CHENG Y F . Small sample load forecasting of electric vehicle charging stations based on transfer learning [C ] // Proceedings of the 2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) . Piscataway : IEEE Press , 2024 : 2041 - 2045 .

FERNÁNDEZ F , VELOSO M . Probabilistic policy reuse in a reinforcement learning agent [C ] // Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems . New York : ACM Press , 2006 : 720 - 727 .

ZHUANG F Z , QI Z Y , DUAN K Y , et al . A comprehensive survey on transfer learning [EB ] . 2020 .

TAYLOR M E , STONE P , LIU Y X . Transfer learning via inter-task mappings for temporal difference learning [J ] . Journal of Machine Learning Research , 2007 , 8 : 2125 - 2167 .

TAYLOR M E , STONE P . Cross-domain transfer for reinforcement learning [C ] // Proceedings of the 24th International Conference on Machine Learning . New York : ACM Press , 2007 : 879 - 886 .

LIAN R Z , TAN H C , PENG J K , et al . Cross-type transfer for deep reinforcement learning based hybrid electric vehicle energy management [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 8 ): 8367 - 8380 .

NIPU A S , LIU S M , HARRIS A . Enabling multi-agent transfer reinforcement learning via scenario independent representation [C ] // Proceedings of the 2023 IEEE Conference on Games (CoG) . Piscataway : IEEE Press , 2023 : 1 - 8 .

NOGUCHI H , ISODA T , ARAI S . Shared trained models selection and management for transfer reinforcement learning in open IoT [C ] // Proceedings of the 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC) . Piscataway : IEEE Press , 2021 : 2170 - 2176 .

KHALID M , WANG L , WANG K Z , et al . Deep reinforcement learning-based long-range autonomous valet parking for smart cities [J ] . Sustainable Cities and Society , 2023 , 89 : 104311 .

FANG X , GONG G C , LI G N , et al . Cross temporal-spatial transferability investigation of deep reinforcement learning control strategy in the building HVAC system level [J ] . Energy , 2023 , 263 : 125679 .

ZHU Z D , LIN K X , JAIN A K , et al . Transfer learning in deep reinforcement learning: a survey [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 11 ): 13344 - 13362 .

浏览量

143

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的图约简方法

意图驱动的物联网双时间尺度资源分配方法

基于深度扩散确定性策略梯度的Wi-Fi网络性能优化