基于D3QN的Wi-Fi网络智能调制方法

吴婷婷; 方旭明

doi:10.11959/j.issn.2096-3750.2025.00460

您当前的位置：

首页 >

文章列表页 >

基于D3QN的Wi-Fi网络智能调制方法

理论与技术 | 更新时间：2026-01-26

- 基于D3QN的Wi-Fi网络智能调制方法
- Intelligent modulation method for Wi-Fi networks based on D3QN
- 物联网学报 2025年9卷第3期页码：122-131
- 作者机构：
  
  西南交通大学信息科学与技术学院，四川成都 611756
- 作者简介：
  
  [ "吴婷婷（2000‒ ），女，西南交通大学信息科学与技术学院硕士生，主要研究方向为Wi-Fi网络。" ]
  [ "方旭明（1962‒ ），男，博士，西南交通大学信息科学与技术学院教授，主要研究方向为通信感知计算一体化网络、Wi-Fi网络、智能交通移动通信系统等。" ]
- 基金信息：
  
  国家自然科学基金资助项目(62071393);四川省重点研发计划项目(2024YFHZ0093)
- DOI：10.11959/j.issn.2096-3750.2025.00460
  中图分类号： TN929.5
- 收稿：2024-10-22，
  
  修回：2024-12-19，
  
  纸质出版：2025-09-10
- 稿件说明：
移动端阅览
吴婷婷,方旭明.基于D3QN的Wi-Fi网络智能调制方法[J].物联网学报,2025,09(03):122-131.

WU Tingting,FANG Xuming.Intelligent modulation method for Wi-Fi networks based on D3QN[J].Chinese Journal on Internet of Things,2025,09(03):122-131.
吴婷婷,方旭明.基于D3QN的Wi-Fi网络智能调制方法[J].物联网学报,2025,09(03):122-131. DOI： 10.11959/j.issn.2096-3750.2025.00460.

WU Tingting,FANG Xuming.Intelligent modulation method for Wi-Fi networks based on D3QN[J].Chinese Journal on Internet of Things,2025,09(03):122-131. DOI： 10.11959/j.issn.2096-3750.2025.00460.

摘要

速率自适应（RA

rate adaptation）技术是Wi-Fi网络中的关键功能，能够根据实时观测的信道状态选择最优的数据传输速率。然而，现有的大多数速率自适应算法主要存在两个问题：一是依赖跨层信息反馈的方法在实际应用中较难实现；二是所采用的方法在速率选择策略上过于保守，当环境变化信噪比（SNR

signal-to-noise ratio）处于两个可选调制阶数之间时，均选择较低的速率阶数。为了解决这些问题，提出了一种基于深度强化学习中双决斗深度Q网络（D3QN

dueling double deep Q-network）的速率自适应算法，该算法无须进行跨层反馈，通过观测物理层信息来动态调整数据速率，并在奖励函数设计和模型加载阶段参考了现有的查表速率调节方法。仿真结果表明，所提算法相比其他4种基线方法，在不同场景中都能迅速适应环境变化，实现了更高的吞吐量性能。

Abstract

Rate adaptation (RA) technology is a key feature in Wi-Fi networks

capable of selecting the optimal data transmission rate based on real-time observed channel conditions. However

most existing rate adaptation algorithms exhibit two issues. Firstly

methods relying on cross-layer information feedback are often challenging to implement in practical applications. Secondly

the strategies employed are over-conservative in rate selection

opting for lower rates when the signal-to-noise ratio (SNR) varies between two selectable modulation levels. A rate adaptation algorithm based on the dueling double deep Q-network (D3QN) in deep reinforcement learning was proposed to address these issues. This algorithm eliminated the need for cross-layer feedback and dynamically adjusted the data rate through the observation of physical layer information. Additionally

it referenced existing table-based rate adjustment methods during the design of the reward function and model loading phase. Simulation results show that the proposed algorithm can rapidly adapt to environmental changes and achieve higher throughput performance compared with four baseline methods across various scenarios.

关键词

Keywords

references

PESERICO G , FEDULLO T , MORATO A , et al . SNR-based reinforcement learning rate adaptation for time critical Wi-Fi networks: assessment through a calibrated simulator [C ] // Proceedings of the 2021 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) . Piscataway : IEEE Press , 2021 : 1 - 6 .

LIN W H , GUO Z Y , LIU P , et al . Deep reinforcement learning based rate adaptation for Wi-Fi networks [C ] // Proceedings of the 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall) . Piscataway : IEEE Press , 2022 : 1 - 5 .

SAMMOUR I , CHALHOUB G . Application-level data rate adaptation in Wi-Fi networks using deep reinforcement learning [C ] // Proceedings of the 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall) . Piscataway : IEEE Press , 2022 : 1 - 7 .

LI Z R , WANG X , PAN L , et al . Network topology optimization via deep reinforcement learning [J ] . IEEE Transactions on Communications , 2023 , 71 ( 5 ): 2847 - 2859 .

CALLEGARO D , LEVORATO M , RESTUCCIA F . SmartDet: context-aware dynamic control of edge task offloading for mobile object detection [C ] // Proceedings of the 2022 IEEE 23rd International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM) . Piscataway : IEEE Press , 2022 : 357 - 366 .

QIN P , FU Y , ZHANG J , et al . DRL-based resource allocation and trajectory planning for NOMA-enabled multi-UAV collaborative caching 6G network [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 6 ): 8750 - 8764 .

AL-ERYANI Y , HOSSAIN E . Self-organizing mmWave MIMO cell-free networks with hybrid beamforming: a hierarchical DRL-based design [J ] . IEEE Transactions on Communications , 2022 , 70 ( 5 ): 3169 - 3185 .

HUANG R , WONG V W S . Joint user scheduling, phase shift control, and beamforming optimization in intelligent reflecting surface-aided systems [J ] . IEEE Transactions on Wireless Communications , 2022 , 21 ( 9 ): 7521 - 7535 .

KURMA S , KATWE M , SINGH K , et al . Spectral-energy efficient resource allocation in RIS-aided FD-MIMO systems [J ] . IEEE Transactions on Wireless Communications , 2024 , 23 ( 5 ): 5125 - 5141 .

ZHONG Y , CHEN H , LIU W , et al . Deep reinforcement learning based channel allocation for channel bonding Wi-Fi networks [C ] // Proceedings of the 2023 19th International Conference on Mobility, Sensing and Networking (MSN) . Piscataway : IEEE Press , 2023 : 113 - 119 .

CHEN H , LIU P , YOU L Z , et al . Deep reinforcement learning based dynamic channel bonding for Wi-Fi networks [C ] // Proceedings of the 2023 IEEE Intl Conf on Parallel & Distributed Processing with Applications , Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom) . Piscataway : IEEE Press , 2023 : 153 - 160 .

HUANG Y W , CHIN K W . A hierarchical deep learning approach for optimizing CCA threshold and transmit power in Wi-Fi networks [J ] . IEEE Transactions on Cognitive Communications and Networking , 2023 , 9 ( 5 ): 1296 - 1307 .

EL JAMOUS Z , DAVASLIOGLU K , SAGDUYU Y E . Deep reinforcement learning for power control in next-generation WiFi network systems [C ] // Proceedings of the MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM) . Piscataway : IEEE Press , 2022 : 547 - 552 .

LUO Y Z , CHIN K W . An energy efficient channel bonding and transmit power control approach for WiFi networks [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 8 ): 8251 - 8263 .

WYDMANSKI W , SZOTT S . Contention window optimization in IEEE 802.11ax networks with deep reinforcement learning [C ] // Proceedings of the 2021 IEEE Wireless Communications and Networking Conference (WCNC) . Piscataway : IEEE Press , 2021 : 1 - 6 .

GRASSO C , RAFTOPOULOS R , SCHEMBRA G . OSCAR: a contention window optimization approach using deep reinforcement learning [C ] // Proceedings of the ICC 2023 - IEEE International Conference on Communications . Piscataway : IEEE Press , 2023 : 459 - 465 .

PURWITA A A , YESILKAYA A , HAAS H . Intelligent subflow steering in MPTCP-based hybrid Wi-Fi and LiFi networks using model-augmented DRL [C ] // Proceedings of the GLOBECOM 2022-2022 IEEE Global Communications Conference . Piscataway : IEEE Press , 2022 : 425 - 430 .

KHASTOO S , BRECHT T , ABEDI A . NeuRA: using neural networks to improve WiFi rate adaptation [C ] // Proceedings of the 23rd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems . New York : ACM Press , 2020 : 161 - 170 .

LI C Y , CHEN S C , KUO C T , et al . Practical machine learning-based rate adaptation solution for Wi-Fi NICs: IEEE 802.11ac as a case study [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 9 ): 10264 - 10277 .

JOSHI T , AHUJA D , SINGH D , et al . SARA: stochastic automata rate adaptation for IEEE 802.11 networks [J ] . IEEE Transactions on Parallel and Distributed Systems , 2008 , 19 ( 11 ): 1579 - 1590 .

WANG C . Dynamic ARF for throughput improvement in 802.11 WLAN via a machine-learning approach [J ] . Journal of Network and Computer Applications , 2013 , 36 ( 2 ): 667 - 676 .

CIEZOBKA W , WOJNAR M , KOSEK-SZOTT K , et al . FTMRate: collision-immune distance-based data rate selection for IEEE 802.11 networks [C ] // Proceedings of the 2023 IEEE 24th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM) . Piscataway : IEEE Press , 2023 : 242 - 251 .

KARMAKAR R , CHATTOPADHYAY S , CHAKRABORTY S . Dynamic link adaptation in IEEE 802.11ac: a distributed learning based approach [C ] // Proceedings of the 2016 IEEE 41st Conference on Local Computer Networks (LCN) . Piscataway : IEEE Press , 2016 : 87 - 94 .

QUEIRÓS R , ALMEIDA E N , FONTES H , et al . Wi-Fi rate adaptation using a simple deep reinforcement learning approach [C ] // Proceedings of the 2022 IEEE Symposium on Computers and Communications (ISCC) . Piscataway : IEEE Press , 2022 : 1 - 3 .

CHEN S C , LI C Y , CHIU C H . An experience driven design for IEEE 802.11ac rate adaptation based on reinforcement learning [C ] // Proceedings of the IEEE INFOCOM 2021-IEEE Conference on Computer Communications . Piscataway : IEEE Press , 2021 : 1 - 10 .

YE X W , FU L Q , CIOFFI J M . Joint codebook selection and MCS adaptation for mmWave eMBB services based on deep reinforcement learning [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 19 ): 31545 - 31560 .

GAO W , ZHENG P , HU Y L , et al . A novel link adaptation approach for URLLC: a DRL-based method with OLLA [C ] // Proceedings of the 2024 IEEE Wireless Communications and Networking Conference (WCNC) . Piscataway : IEEE Press , 2024 : 1 - 6 .

IEEE . Draft standard for information technology-telecommunications and information exchange between systems local and metropolitan area networks-specific requirements-part 11: wireless LAN medium access control (MAC) and physical layer (PHY) specifications amendment 1: enhancements for high efficiency WLAN: IEEE P802.11ax/D5.0 [S ] . 2019 .

PORAT R . 11ax evaluation methodology [S ] . Doc : IEEE 802 - 11-14/0571r12 , 2016.

VAN HASSELT H , GUEZ A , SILVER D . Deep reinforcement learning with double Q-learning [C ] // Proceedings of 30th Association-for-the-Advancement-of-Artificial-Intelligence (AAAI) Conference on Artificial Intelligence . Menlo Park : AAAI Press , 2016 : 2094 - 2100 .

WANG Z Y , SCHAUL T , HESSEL M , et al . Dueling network architectures for deep reinforcement learning [C ] // Proceedings of the 33rd International Conference on Machine Learning . International Machine Learning Society , 2016 : 1995 - 2003 .

WATKINS C J C H . Learning from delayed rewards [J ] . Robotics & Autonomous Systems , 1989 .

YIN H , LIU P , LIU K , et al . Ns3-Ai: fostering artificial intelligence algorithms for networking research [C ] // Proceedings of the 2020 Workshop on ns-3 . ns-3 Consortium , 2020 : 57 - 64 .

ALBAR R , ARIF T Y , MUNADI R . Modified rate control for collision-aware in Minstrel-HT rate adaptation algorithm [C ] // Proceedings of the 2018 International Conference on Electrical Engineering and Informatics (ICELTICs) . Piscataway : IEEE Press , 2018 : 7 - 12 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度扩散确定性策略梯度的Wi-Fi网络性能优化

车联网中基于迁移强化学习的跨域充电站推荐算法

PD-TD3：高速公路场景下边路协同计算卸载策略

基于深度强化学习的多租户算网资源分配算法

基于能量收集技术的协作卸载计算方案