A Wi-Fi sensing method for complex continuous human activities based on CNN-BiGRU

Yang LIU; Anming DONG; Jiguo YU; Kai ZHAO; You ZHOU

doi:10.11959/j.issn.2096-3750.2023.00360

您当前的位置：

首页 >

文章列表页 >

A Wi-Fi sensing method for complex continuous human activities based on CNN-BiGRU

Theory and Technology | 更新时间：2024-08-16

- A Wi-Fi sensing method for complex continuous human activities based on CNN-BiGRU
- Chinese Journal on Internet of Things Vol. 7, Issue 4, Pages: 153-167(2023)
- 作者机构：
  
  1. 齐鲁工业大学（山东省科学院）山东省计算中心（国家超级计算济南中心）算力互联网与信息安全教育部重点实验室，山东济南 250353
  2. 齐鲁工业大学（山东省科学院）计算机科学与技术学院，山东济南 250353
  3. 齐鲁工业大学（山东省科学院）大数据研究院，山东济南 250353
  4. 山东省基础科学研究中心（计算机科学）山东省计算机网络重点实验室，山东济南 250353
  5. 中国科学院自动化研究所，北京 100190
  6. 山东海看新媒体研究院有限公司，山东济南 250013
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2023.00360
  CLC： TN92
- Online First：2023-12，
  
  Published：20 December 2023
- 稿件说明：
移动端阅览
Yang LIU, Anming DONG, Jiguo YU, et al. A Wi-Fi sensing method for complex continuous human activities based on CNN-BiGRU[J]. Chinese Journal on Internet of Things, 2023, 7(4): 153-167.
DOI：

Yang LIU, Anming DONG, Jiguo YU, et al. A Wi-Fi sensing method for complex continuous human activities based on CNN-BiGRU[J]. Chinese Journal on Internet of Things, 2023, 7(4): 153-167. DOI： 10.11959/j.issn.2096-3750.2023.00360.

摘要

基于Wi-Fi信道状态信息（CSI

channel state information）的人体活动感知在虚拟现实、智能游戏、元宇宙等未来智能交互场景具有重要的应用前景，复杂连续人体活动的精准感知是Wi-Fi感知的重要挑战。卷积神经网络（CNN

convolutional neural network）具备空间特征提取能力，但对数据的时序特征建模能力差。而适用于时间序列数据建模的长短期记忆（LSTM

long short-term memory）网络或门控循环单元（GRU

gated recurrent unit）网络忽视了对数据空间特征的学习。针对此问题，提出了一种融合双向门控循环单元（BiGRU

bidirectional gated recurrent unit）网络的改进型 CNN。所提网络利用 BiGRU的双向特征提取能力捕捉时序数据前后信息的关联和依赖性，实现时序CSI数据的时空特征提取，进而呈现动作与CSI数据的映射关系，从而提高对复杂连续动作的识别精度。以篮球动作为场景对所提网络结构进行了实验，结果表明，该方法在多种条件下识别准确率均高于95%，与传统多层感知机（MLP

multi-layer perceptron）、CNN、LSTM、GRU、具有注意力机制的双向长短期记忆（ABLSTM

attention based bidirectional long short-term memory）网络等基线方法相比，识别准确率提升了1％～20％。

Abstract

Human activity sensing based on Wi-Fi channel state information (CSI) has an important application prospect in future intelligent interaction scenarios such as virtual reality

intelligent games

and the metaverse.Accurate sensing of complex and continuous human activities is an important challenge for Wi-Fi sensing.Convolutional neural network (CNN) has the ability of spatial feature extraction but is poor at modeling the temporal features of the data.While long short-term memory (LSTM) network or gated recurrent unit (GRU) network

which are suitable for modeling time-series data

neglect learning spatial features of data.In order to solve this problem

an improved CNN that integrates bidirectional gated recurrent unit (BiGRU) network was proposed.The bi-directional feature extraction ability of BiGRU was used to capture the correlation and dependence of the front and back information in the time series data.The extraction of the spatiotemporal features of the time series CSI data was realized

and then the mapping relationship between the action and the CSI data was present.Thus the recognition accuracy of the complex continuous action was improved.The proposed network structure was tested with basketball actions.The results show that the recognition accuracy of this method is above 95% under various conditions.Compared with the traditional multi-layer perceptron (MLP)

CNN

LSTM

GRU

and attention based bidirectional long short-term memory (ABLSTM) baseline methods

the recognition accuracy has been improved by 1%～20%.

关键词

Keywords

references

张宇翔 , 任爽 . 定位技术在虚拟现实中的应用综述 [J ] . 计算机科学 , 2021 , 48 ( 1 ): 308 - 318 .

ZHANG Y X , REN S . Overview of application of positioning technology in virtual reality [J ] . Computer Science , 2021 , 48 ( 1 ): 308 - 318 .

CHEN C , SHU Y , SHU K I , et al . WiTT:modeling and the evaluation of table tennis actions based on Wi-Fi signals [C ] // Proceedings of 2018 24th International Conference on Pattern Recognition (ICPR) . Piscataway:IEEE Press , 2018 : 3100 - 3107 .

王文喜 , 周芳 , 万月亮 , 等 . 元宇宙技术综述 [J ] . 工程科学学报 , 2022 , 44 ( 4 ): 744 - 756 .

WANG W X , ZHOU F , WAN Y L , et al . A survey of metaverse technology [J ] . Chinese Journal of Engineering , 2022 , 44 ( 4 ): 744 - 756 .

WANG Y , LIU J , CHEN Y Y , et al . E-eyes:device-free location-oriented activity identification using fine-grained Wi-Fi signatures [C ] // Proceedings of the 20th annual international conference on Mobile computing and networking . New York:ACM Press , 2014 : 617 - 628 .

LI F , VALERO M , SHAHRIAR H , et al . Wi-COVID:a COVID-19 symptom detection and patient monitoring framework using Wi-Fi [J ] . Smart Health , 2021 ( 19 ): 100147 .

CHEN M D , MA J , ZENG X M , et al . MD-alarm:a novel manpower detection method for ship bridge watchkeeping using Wi-Fi signals [J ] . IEEE Transactions on Instrumentation and Measurement , 2022 ( 71 ): 1 - 13 .

GU Y , WANG Y T , LIU T , et al . EmoSense:computational intelligence driven emotion sensing via wireless channel data [J ] . IEEE Transactions on Emerging Topics in Computational Intelligence , 2020 , 4 ( 3 ): 216 - 226 .

LARA O D , LABRADOR M A . A survey on human activity recognition using wearable sensors [J ] . IEEE Communications Surveys ＆ Tutorials , 2013 , 15 ( 3 ): 1192 - 1209 .

JALAL A , KIM Y H , KIM Y J . Robust human activity recognition from depth video using spatiotemporal multi-fused features [J ] . Pattern Recognition , 2017 ( 61 ): 295 - 308 .

YOUSEFI S , NARUI H , DAYAL S , et al . A survey on behavior recognition using Wi-Fi channel state information [J ] . IEEE Communications Magazine , 2017 , 55 ( 10 ): 98 - 104 .

YAN H , ZHANG Y , WANG Y J , et al . WiAct:a passive Wi-Fi-based human activity recognition system [J ] . IEEE Sensors Journal , 2020 , 20 ( 1 ): 296 - 305 .

AKHTAR Z U A , WANG H . Wi-Fi-based driver’s activity recognition using multi-layer classification [J ] . Neurocomputing , 2020 ( 405 ): 12 - 25 .

ZOU H , ZHOU Y X , YANG J F , et al . DeepSense:device-free human activity recognition via auto encoder long-term recurrent convolutional network [C ] // Proceedings of 2018 IEEE International Conference on Communications (ICC) . Piscataway:IEEE Press , 2018 : 1 - 6 .

ZHOU Z Y , LIU C , YU X D , et al . Deep-WiID:Wi-Fi-based contactless human identification via deep learning [C ] // Proceedings of 2019 IEEE SmartWorld,Ubiquitous Intelligence ＆ Computing,Advanced ＆Trusted Computing,Scalable Computing ＆ Communications,Cloud ＆Big Data Computing,Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) . Piscataway:IEEE Press , 2020 : 877 - 884 .

SHEN X , NI Z , LIU L , et al . WiPass:1D-CNN-based smartphone keystroke recognition Using Wi-Fi signals [J ] . Pervasive and Mobile Computing , 2021 ( 73 ): 101393 .

JIA L Y , GU Y , CHENG K , et al . BeAware:convolutional neural network(CNN) based user behavior understanding through Wi-Fi channel state information [J ] . Neurocomputing , 2020 , 397 : 457 - 463 .

ZENG Y W , WU D , XIONG J , et al . Boosting Wi-Fi sensing performance via CSI ratio [J ] . IEEE Pervasive Computing , 2021 , 20 ( 1 ): 62 - 70 .

郝占军 , 段渝 , 党小超 , 等 . 基于信道状态信息的人体复杂动作识别方法 [J ] . 计算机工程 , 2020 , 46 ( 1 ): 286 - 293 .

HAO Z J , DUAN Y , DANG X C , et al . Human complex motion recognition method based on channel state information [J ] . Computer Engineering , 2020 , 46 ( 1 ): 286 - 293 .

郑增威 , 杜俊杰 , 霍梅梅 , 等 . 基于可穿戴传感器的人体活动识别研究综述 [J ] . 计算机应用 , 2018 , 38 ( 5 ): 1223 - 1229 , 1238 .

ZHENG Z W , DU J J , HUO M M , et al . Review of human activity recognition based on wearable sensors [J ] . Journal of Computer Applications , 2018 , 38 ( 5 ): 1223 - 1229 , 1238 .

YATANI K , TRUONG K N . BodyScope:a wearable acoustic sensor for activity recognition [C ] // Proceedings of the 2012 ACM Conference on Ubiquitous Computing . New York:ACM Press , 2012 : 341 - 350 .

CHEN Z H , ZHU Q C , SOH Y C , et al . Robust human activity recognition using smartphone sensors via CT-PCA and online SVM [J ] . IEEE Transactions on Industrial Informatics , 2017 , 13 ( 6 ): 3070 - 3080 .

SADANAND S , CORSO J J . Action bank:a high-level representation of activity in video [C ] // Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2012 : 1234 - 1241 .

CHAKRABORTY I , ELGAMMAL A , BURD R S . Video based activity recognition in trauma resuscitation [C ] // Proceedings of 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) . Piscataway:IEEE Press , 2013 : 1 - 8 .

CHEN Y , YU L , OTA K , et al . Hierarchical posture representation for robust action recognition [J ] . IEEE Transactions on Computational Social Systems , 2019 , 6 ( 5 ): 1115 - 1125 .

SIGG S , BLANKE U , TRÖSTER G . The telepathic phone:frictionless activity recognition from Wi-Fi-RSSI [C ] // Proceedings of 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom) . Piscataway:IEEE Press , 2014 : 148 - 155 .

GU Y , REN F J , LI J . PAWS:passive human activity recognition based on Wi-Fi ambient signals [J ] . IEEE Internet of Things Journal , 2016 , 3 ( 5 ): 796 - 805 .

HALPERIN D , HU W J , SHETH A , et al . Tool release:gathering 802.11n traces with channel state information [J ] . ACM SIGCOMM Computer Communication Review , 2011 , 41 ( 1 ): 53 .

SEN S , LEE J , KIM K H , et al . Avoiding multipath to revive inbuilding Wi-Fi localization [C ] // Proceedings of MobiSys’13:Proceeding of the 11th annual international conference on Mobile systems,applications,and services . New York:ACM Press , 2013 : 249 - 262 .

ZHANG J , WU F X , WEI B , et al . Data augmentation and dense-LSTM for human activity recognition using Wi-Fi signal [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 6 ): 4628 - 4641 .

郭浩雨 , 冯秀芳 . 基于Bi-LSTM的CSI手势识别算法 [J ] . 计算机工程与设计 , 2022 , 43 ( 9 ): 2614 - 2621 .

GUO H Y , FENG X F . CSI gesture recognition algorithm based on Bi-LSTM [J ] . Computer Engineering and Design , 2022 , 43 ( 9 ): 2614 - 2621 .

HASEGAWA R , UCHIYAMA A , HIGASHINO T . Maneuver classification in wheelchair basketball using inertial sensors [C ] // Proceedings of 2019 Twelfth International Conference on Mobile Computing and Ubiquitous Network (ICMU) . Piscataway:IEEE Press , 2020 : 1 - 6 .

YAO B Q , GAO H , SU X . Human motion recognition by three-view kinect sensors in virtual basketball training [C ] // Proceedings of 2020 IEEE REGION 10 CONFERENCE (TENCON) . Piscataway:IEEE Press , 2020 : 1260 - 1265 .

熊小樵 , 冯秀芳 , 丁一 . 基于 CSI 的手势识别方法研究 [J ] . 计算机应用与软件 , 2022 , 39 ( 1 ): 181 - 187 .

XIONG X Q , FENG X F , DING Y . Research on hand gesture recognition method based on CSI [J ] . Computer Applications and Software , 2022 , 39 ( 1 ): 181 - 187 .

LIU W , CHANG S , LIU Y , et al . Wi-PSG:detecting rhythmic movement disorder using COTS Wi-Fi [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 6 ): 4681 - 4696 .

WAYT H J , KHAN T R . Integrated savitzky-golay filter from inverse taylor series approach [C ] // International Conference on Digital Signal Processing . Piscataway:IEEE Press , 2007 : 375 - 378 .

QIAN K , WU C S , YANG Z , et al . PADS:Passive detection of moving targets with dynamic speed using PHY layer information [C ] // Proceedings of 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS) . Piscataway:IEEE Press , 2015 : 1 - 8 .

KRESGE K , MARTINO S , ZHAO T M , et al . Wi-Fi-based contactless gesture recognition using lightweight CNN [C ] // Proceedings of 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems (MASS) . Piscataway:IEEE Press , 2021 : 645 - 650 .

DENIL M , BAZZANI L , LAROCHELLE H , et al . Learning where to attend with deep architectures for image tracking [J ] . Neural Computation , 2012 , 24 ( 8 ): 2151 - 2184 .

GUO L L , WANG L , LIU J L , et al . A novel benchmark on human activity recognition using Wi-Fi signals [C ] // Proceedings of 2017 IEEE 19th International Conference on e-Health Networking,Applications and Services (Healthcom) . Piscataway:IEEE Press , 2017 : 1 - 6 .

LIU S Q , ZHAO Y C , CHEN B . WiCount:a deep learning approach for crowd counting using Wi-Fi signals [C ] // Proceedings of 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC) . Piscataway:IEEE Press , 2018 : 967 - 974 .

CHEN Z H , ZHANG L , JIANG C Y , et al . Wi-Fi CSI based passive human activity recognition using attention based BLSTM [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 11 ): 2714 - 2724 .

Views

447

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Siamese GCN empowered fingerprinting indoor localization using few-shot transfer learning

Lightweight attention-based SAR ship detector

Research on multi-user identity recognition based on Wi-Fi sensing

Robust transmission algorithm for IRS-assisted NOMA network with hardware impairments and imperfect CSI

Wi-freshness: research on CSI-based pork freshness detecting system

Related Author

HUANG Hao

ZHU Hongbo

ZHAO Haitao

XIA Wenchao

WANG Yu

SHI Zheng

GU Hao

LU Ping

Related Institution

Jiangsu Province Key Laboratory of Wireless Communication and Internet of Things

School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications

School of Integrated Circuits, Southeast University

School of Internet of Things, Nanjing University of Posts and Telecommunications

State Key Laboratory of Mobile Network and Mobile Multimedia Technology

⁰