Reinforcement learning-based channel access mechanism for multi-base station slotted Aloha with cooperative reception

HUANG Yuankang; ZHAN Wen; SUN Xinghua

doi:10.11959/j.issn.2096-3750.2024.00388

您当前的位置：

首页 >

文章列表页 >

Reinforcement learning-based channel access mechanism for multi-base station slotted Aloha with cooperative reception

Theory and Technology | 更新时间：2024-08-16

- Reinforcement learning-based channel access mechanism for multi-base station slotted Aloha with cooperative reception
- Chinese Journal on Internet of Things Vol. 8, Issue 2, Pages: 26-35(2024)
- 作者机构：
  
  1.中山大学（深圳），广东深圳 518107
  2.中山大学，广东广州 510275
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2023YFB2904100)
- DOI：10.11959/j.issn.2096-3750.2024.00388
  CLC：
- Received：21 November 2023，
  
  Revised：2024-05-16，
  
  Published：10 June 2024
- 稿件说明：
移动端阅览
黄元康,詹文,孙兴华.基于强化学习的多基站协作接收时隙Aloha网络信道接入机制[J].物联网学报,2024,08(02):26-35.

HUANG Yuankang,ZHAN Wen,SUN Xinghua.Reinforcement learning-based channel access mechanism for multi-base station slotted Aloha with cooperative reception[J].Chinese Journal on Internet of Things,2024,08(02):26-35.
黄元康,詹文,孙兴华.基于强化学习的多基站协作接收时隙Aloha网络信道接入机制[J].物联网学报,2024,08(02):26-35. DOI： 10.11959/j.issn.2096-3750.2024.00388.

HUANG Yuankang,ZHAN Wen,SUN Xinghua.Reinforcement learning-based channel access mechanism for multi-base station slotted Aloha with cooperative reception[J].Chinese Journal on Internet of Things,2024,08(02):26-35. DOI： 10.11959/j.issn.2096-3750.2024.00388.

摘要

随着物联网（IoT

internet of things）基站的部署愈发密集，网络干扰管控的重要性愈发凸显。物联网中，设备常采用随机接入，以分布式的方式接入信道。在海量设备的物联网场景中，节点之间可能会出现严重的干扰，导致网络的吞吐量性能严重下降。为了解决随机接入网络中的干扰管控问题，考虑基于协作接收的多基站时隙Aloha网络，利用强化学习工具，设计自适应传输算法，实现干扰管控，优化网络的吞吐量性能，并提高网络的公平性。首先，设计了基于Q-学习的自适应传输算法，通过仿真验证了该算法面对不同网络流量时均能保障较高的网络吞吐量性能。其次，为了提高网络的公平性，采用惩罚函数法改进自适应传输算法，并通过仿真验证了面向公平性优化后的算法能够大幅提高网络的公平性，并保障网络的吞吐性能。

Abstract

With the increasingly dense deployment of base stations in the internet of things (IoT)

the importance of interference management becomes ever more pronounced. In IoT environments

devices often employ random access

connecting to channels in a distributed manner. In scenarios involving massive numbers of devices

severe interference may arise between nodes

leading to significant degradation in the throughput performance of the network. To address interference control issues in networks with random access

a multi-base station slotted Aloha network based on cooperative reception was considered

the reinforcement learning techniques was leveraged to design adaptive transmission algorithms that effectively managed interference

optimized network throughput performance

and enhanced network fairness. Firstly

an adaptive transmission algorithm were devised based on Q-learning

which was verified to maintain high network throughput performance under varying traffic conditions through simulation. Secondly

to improve network fairness

the penalty function method was employed to refine the adaptive transmission algorithm. Simulations confirm that the fairness-optimized algorithm significantly enhances network fairness while preserving satisfactory network throughput performance.

关键词

Keywords

references

BOCCARDI F , HEATH R W , LOZANO A , et al . Five disruptive technology directions for 5G [J ] . IEEE Communication Magazine , 2014 , 52 ( 2 ): 74 - 80 .

CHETTRI L , BERA R . A comprehensive survey on Internet of things (IoT) toward 5G wireless systems [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 1 ): 16 - 32 .

AL-GARADI M A , MOHAMED A , AL-ALI A K , et al . A survey of machine and deep learning methods for internet of things (IoT) security [J ] . IEEE Communications Surveys and Tutorials , 2020 , 22 ( 3 ): 1646 - 1685 .

GUPTA A , JHA R K . A survey of 5G network: architecture and emerging technologies [J ] . IEEE Access , 2015 , 3 : 1206 - 1232 .

VAEZI M , AZARI A , KHOSRAVIRAD S R , et al . Cellular, wide-area, and non-terrestrial IoT: a survey on 5G advances and the road toward 6G [J ] . IEEE Communications Surveys and Tutorials , 2022 , 24 ( 2 ): 1117 - 1174 .

AL-TURJMAN F , EVER E , ZAHMATKESH H . Small cells in the forthcoming 5G/IoT: traffic modelling and deployment overview [J ] . IEEE Communications Surveys and Tutorials , 2019 , 21 ( 1 ), 28 - 65 .

ZHANG Z , LI Y , HUANG C , et al . User activity detection and channel estimation for grant-free random access in LEO satellite-enabled internet of things [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 9 ): 8811 - 8825 .

SHANMUGA SUNDARAM J P , DU W , ZHAO Z . A survey on LoRa networking: research problems, current solutions, and open issues [J ] . IEEE Communications Surveys and Tutorials , 2020 , 22 ( 1 ): 371 - 388 .

BELTRAMELLI L , MAHMOOD A , OSTERBETG P , et al . LoRa beyond ALOHA: an investigation of alternative random access protocols [J ] . IEEE Transactions on Industrial Informatics , 2021 , 17 ( 5 ): 3544 - 3554 .

ABRAMSON N . The throughput of packet broadcasting channels [J ] . IEEE Transactions on Communications , 1977 , 25 ( 1 ): 117 - 128 .

SAYED A H , TARIGHAT A , KHAJEHNOURI N . Network-based wireless location: challenges faced in developing techniques for accurate wireless location information [J ] . IEEE Signal Processing Magazine , 2005 , 22 ( 4 ): 24 - 40 .

ZHAN W , SUN X , WANG X , et al . Performance optimization for massive random access of mMTC in cellular networks with preamble retransmission limit [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 9 ): 8854 - 8867 .

ZHAN W , DAI L . Massive random access of machine-to-machine communications in LTE networks: throughput optimization with a finite data transmission rate [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 12 ): 5749 - 5763 .

YANG Y , DAI L . Stability region and transmission control of multi-cell Aloha networks [J ] . IEEE Transactions on Communications , 2023 , 71 ( 9 ): 5348 - 5364 .

AHMED F , CHO H-S . A time-slotted data gathering medium access control protocol using Q-learning for underwater acoustic sensor networks [J ] . IEEE Access , 2021 ( 9 ): 48742 - 48752 .

JADOON M A , PASTORE A , NAVARRO M , et al . Deep reinforcement learning for random access in machine-type communication [C ] // Proceedings of the 2022 IEEE Wireless Communications and Networking Conference (WCNC) , 2022 : 2553 - 2558 .

CUI Q , ZHANG Z , SHI Y , et al . Dynamic multichannel access based on deep reinforcement learning in distributed wireless networks [J ] . IEEE Systems Journal , 2022 , 16 ( 4 ): 5831 - 5834 .

YU Y , WANG T , LIEW S C . Deep-reinforcement learning multiple access for heterogeneous wireless networks [C ] // Proceedings of the 2018 IEEE International Conference on Communications (ICC) , 2018 : 1 - 7 .

SHIN K-S , CHOI H-H , LEE H . Knowledge transfer-based multiagent Q-learning for medium access in dense cellular networks [J ] . IEEE Wireless Communications Letters , 2022 , 11 ( 12 ): 2542 - 2545 .

WANG S , WANG X , ZHANG Y , et al . Resource allocation in multi-cell NOMA systems with multi-agent deep reinforcement learning [C ] // Proceedings of the 2021 IEEE Wireless Communications and Networking Conference (WCNC) , 2021 : 1 - 6 .

KHAIRY S , BALAPRAKASH P , CAI L X , et al . Constrained deep reinforcement learning for energy sustainable multi-UAV based random access IoT networks with NOMA [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 4 ): 1101 - 1115 .

YOUN J , PARK J , KIM S , et al . MARL-based random access scheme for delay-constrained umMTC in 6G [C ] // Proceedings of the 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring) , 2023 : 1 - 6 .

OGATA S , ISHIBASHI K , FREITAS DE ABREU G T . Optimized frameless aloha for cooperative base stations with overlapped coverage areas [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 11 ): 7486 - 7499 .

JAKOVETIC D , BAJOVIC D , VUKOBRATOVIC D , et al . Cooperative slotted aloha for multi-base station systems [J ] . IEEE Transactions on Communications , 2015 , 63 ( 4 ): 1443 - 1456 .

YU C-H , HUANG L , CHANG C-S , et al . Poisson receivers: a probabilistic framework for analyzing coded random access [J ] . IEEE/ACM Transactions on Networking , 2021 , 29 ( 2 ): 862 - 875 .

WANG R , LI P , CUI G , et al . Cooperative slotted aloha with reservation for multi-receiver satellite IoT networks [C ] // Proceedings of the 2018 IEEE/CIC International Conference on Communications in China (ICCC) , 2018 : 593 - 597 .

MASTILOVIC A , VUKOBRATOVIC D , JAKOVETIC D , et al . Cooperative slotted aloha for massive M2M random access using directional antennas [C ] // Proceedings of the 2017 IEEE International Conference on Communications Workshops (ICC Workshops) , 2017 : 731 - 736 .

GEORGIOU O , PSOMAS C , KRIKIDIS I . Coverage scalability analysis of multi-cell LoRa networks [C ] // Proceedings of the 2020 IEEE International Conference on Communications (ICC) , 2020 : 1 - 7 .

BOUAZIZI Y , BENKHELIFA F , MCCANN J . Spatiotemporal modelling of multi-gateway LoRa networks with imperfect SF orthogonality [C ] // Proceedings of the 2020 IEEE Global Communications Conference (GLOBECOM) , 2020 : 1 - 7 .

TU L T , BRADAI A , POUSSET Y . Coverage probability and spectral efficiency analysis of multi-gateway downlink LoRa networks [C ] // Proceedings of the 2022 IEEE International Conference on Communications (ICC) , 2022 : 1 - 6 .

OCHOA M N , MAMAN M , DUDA A . Spreading factor allocation for LoRa nodes progressively joining a multi-gateway adaptive network [C ] // Proceedings of the IEEE Global Communications Conference (GLOBECOM) , 2020 : 1 - 6 .

HEUSSE M , CAILOUET C , DUDA A . Performance of unslotted ALOHA with capture and multiple collisions in LoRaWAN [J ] . IEEE Internet of Things Journal , 2023 , 10 ( 20 ): 17824 - 17838 .

JAIN R , CHIU D , HAWE W . A quantitative measure of fairness and discrimination for resource allocation in shared computer systems [J ] . arXiv preprint , 1998 , arXiv: cs-NI/9809099.

Views

183

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on low-latency transmission migration method for federated learning models in the Internet of vehicles

Evolution analysis of power wireless private networks for control services

Research on the new paradigm for optimizing 5G base station energy-saving control strategies utilizing the Decision Transformer model

Review of research on smart railway Internet of things

Related Author

Wang Shuai

Yin Hongbo

Jiang Chi

Zhang Ke

Zhang Yin

WEI Lei

MIU Weiwei

WANG Dayang

Related Institution

Shenzhen Institute for Advanced Study, UESTC

School of Information and Communication Engineering, University of Electronic Science and Technology of China

Guangdong Intelligent Robot Research Institute

State Grid Jiangsu Electric Power Co., Ltd.

State Grid Electric Power Research Institute Co., Ltd.

⁰