An environment adaptive gesture recognition system based on visible light

Zhu WANG; Hualei ZHANG; Qianhong HU; Zhiwen YU

doi:10.11959/j.issn.2096-3750.2023.00344

您当前的位置：

首页 >

文章列表页 >

An environment adaptive gesture recognition system based on visible light

Topic: Intellisense Technology | 更新时间：2024-08-16

- An environment adaptive gesture recognition system based on visible light
- Chinese Journal on Internet of Things Vol. 7, Issue 2, Pages: 15-25(2023)
- 作者机构：
  
  1. 西北工业大学计算机学院，陕西西安 710072
  2. 湖南文理学院计算机与电气工程学院，湖南常德 415006
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2023.00344
  CLC： TP391
- Online First：2023-06，
  
  Published：30 June 2023
- 稿件说明：
移动端阅览
Zhu WANG, Hualei ZHANG, Qianhong HU, et al. An environment adaptive gesture recognition system based on visible light[J]. Chinese Journal on Internet of Things, 2023, 7(2): 15-25.
DOI：

Zhu WANG, Hualei ZHANG, Qianhong HU, et al. An environment adaptive gesture recognition system based on visible light[J]. Chinese Journal on Internet of Things, 2023, 7(2): 15-25. DOI： 10.11959/j.issn.2096-3750.2023.00344.

摘要

手势日益成为一种重要的人机交互方式，可在电子游戏、虚拟现实等场景中为用户提供更优质的体验。近年来，研究者探索利用不同感知技术实现手势识别，如射频信号、声学信号等。与之相比，利用可见光识别手势具有更强普适性。基本原理为：不同手势遮挡可见光会产生独特的阴影模式，通过光电传感器捕捉阴影变化即可实现手势识别。针对可见光手势识别面临的环境依赖难题，设计了一种基于光电传感器阵列的数字手势识别系统，提出了基于图像的阵列感知数据抽象表示模型，结合图像固有特性发掘不同传感器数据之间的时间和空间关联性，利用时空特征设计了基于CNN-RNN的环境自适应手势识别方法。为了验证所提方法的有效性，设计了环境自适应手势识别系统Vi-Gesture，准确率相比基线方法提升10%以上。

Abstract

Gesture-based human-machine interaction is becoming more and more important

which can provide users with a better experience in scenarios such as video games and virtual reality.In recent years

researchers have explored different sensing technologies to facilitate gesture recognition

including RF signal

acoustic signal

etc.Compared with these approaches

visible light-based gesture recognition is a more pervasive option.The basic principle is that different gestures will produce unique shadow patterns as they block the visible light

and gesture recognition can be achieved by capturing shadow changes through photoelectric sensors.To address the environment-dependent problem faced by existing solutions

a digit gesture recognition system was designed based on the photoelectric sensor array.In particular

by modeling recordings of the sensor array as images

the temporal and spatial correlation between different sensor recordings was discovered.An environment adaptive gesture recognition method was designed based on CNN-RNN by fusing the spatio-temporal features.To verify the effectiveness of the proposed method

a prototype gesture recognition system was designed

named Vi-Gesture.Experimental results show that the proposed method outperforms baselines by more than 10% in recognition accuracy.

关键词

Keywords

references

SHEN S , GU K , CHEN X R , et al . Gesture recognition through sEMG with wearable device based on deep learning [J ] . Mobile Networksand Applications , 2020 , 25 ( 6 ): 2447 - 2458 .

CHAKRABORTY B K , SARMA D , BHUYAN M K , et al . Review of constraints on vision-based gesture recognition for human-computer interaction [J ] . IET Computer Vision , 2018 , 12 ( 1 ): 3 - 15 .

WANG Y W , SHEN J X , ZHENG Y Q . Push the limit of acoustic gesture recognition [C ] // Proceedings of IEEE INFOCOM 2020 - IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2020 : 566 - 575 .

GAO R Y , ZHANG M , ZHANG J , et al . Towards position-independent sensing for gesture recognition with Wi-Fi [J ] . Proceedings of the AC Mon Interactive,Mobile,Wearable and Ubiquitous Technologies , 2021 , 5 ( 2 ): 1 - 28 .

ZHANG C , TABOR J , ZHANG J L , et al . Extending mobile interaction through near-field visible light sensing [C ] // Proceedings of the 21st Annual International Conference on Mobile Computing and Networking . New York:ACM , 2015 : 345 - 357 .

LIT X , ANC K , TIAN Z , et al . Human sensing using visible light communication [C ] // Proceedings of the 21st Annual International Conference on Mobile Computing and Networking . New York:ACM , 2015 : 331 - 344 .

KAHOLOKULA D . Reusing ambient light to recognize hand gestures [EB ] . 2016 .

HU W J , GU H , PU Q F . Light sync:unsynchronized visual communication over screen-camera links [C ] // Proceedings of the 19th Annual International Conference on Mobile Computing ＆ Networking . New York:ACM Press , 2013 : 15 - 26 .

KIMH S , KIMD R , YANG S H , et al . An indoor visible light communication positioning system using a RF carrier allocation technique [J ] . Journal of Light Wave Technology , 2013 , 31 ( 1 ): 134 - 144 .

ELAMASSIE M , KARBALAYGHAREH M , MIRAMIRKHANI F , et al . Effect of fog and rain on the performance of vehicular visible light communications [C ] // Proceedings of 2018 IEEE 87th Vehicular Technology Conference (VTC Spring) . Piscataway:IEEE Press , 2018 : 1 - 6 .

SUN S Y , YANG F , SONG J . Sum rate maximization for intelligent reflecting surface-aided visible light communications [J ] . IEEE Communications Letters , 2021 , 25 ( 11 ): 3619 - 3623 .

KUOY S , PANNUTO P , HSIAOK J , et al . Luxapose:indoor positioning with mobile phones and visible light [C ] // Proceedings of the 20th Annual International Conference on Mobile Computing and Networking . New York:ACM Press , 2014 : 447 - 458 .

LI L , HU P , PENG C , et al . Epsilon:a visible light-based positioning system [C ] // Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI’ 14),331-343 , 2014 .

ALAM F , CHEW M T , WENGE T , et al . An accurate visible light positioning system using regenerated fingerprint database based on calibrated propagation model [J ] . IEEE Transactions on Instrumentation and Measurement , 2019 , 68 ( 8 ): 2714 - 2723 .

MAJEED K , HRANILOVIC S . Passive indoor visible light positioning system using deep learning [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 19 ): 14810 - 14821 .

YANG Y B , HAO J , LUO J , et al . Ceiling see:device-free occupancy inference through lighting infrastructure based LED sensing [C ] // Proceedings of 2017 IEEE International Conference on Pervasive Computing and Communications (Per Com) . Piscataway:IEEE Press , 2017 : 247 - 256 .

LI T X , LIU Q , ZHOU X . Practical human sensing in the light [C ] // Proceedings of the 14th Annual International Conference on Mobile Systems,Applications,and Services . New York:ACM Press , 2016 : 71 - 84 .

LIT X , XIONG X , XIE Y F , et al . Reconstructing hand poses using visible light [J ] . Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies , 2017 , 1 ( 3 ): 1 - 20 .

VENKATNARAYANR H , SHAHZAD M . Gesture recognition using ambient light [J ] . Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies , 2018 , 2 ( 1 ): 1 - 28 .

HU Q , YU Z , WANG Z , et al . Vi hand:gesture recognition with ambient light [C ] // Proceedings of the IEEE International Conference on Ubiquitous Intelligence and Computing ,[S.l.:s.n. ] , 2019 .

LAI K , YANUSHKEVICHS N . CNN RNN depth and skeleton based dynamic hand gesture recognition [C ] // Proceedings of 2018 24th International Conference on Pattern Recognition (ICPR) . Piscataway:IEEE Press , 2018 : 3451 - 3456 .

WEBBER J , MEHBODNIYA A , TENG R , et al . Human-machine interaction using probabilistic neural network for light communication systems [J ] . Electronics , 2022 , 11 ( 6 ): 932 .

LEYS C , LEY C , KLEIN C , et al . Detecting outliers:do not use standard deviation around the mean,use absolute deviation around the Median [J ] . Journal of Experimental Social Psychology , 2013 , 49 ( 4 ): 764 - 766 .

MA D , LAN G H , HU C S , et al . Recognizing hand gestures using solar cells [J ] . IEEE Transactions on Mobile Computing , 2022 , PP ( 99 ): 1 .

WEBBER J , MEHBODNIYA A , ARAFA A , et al . Gesture recognition using machine learning for light communication systems [C ] // Proceedings of 2022 International Mobile and Embedded Technology Conference (MECON) . Piscataway:IEEE Press , 2022 : 52 - 56 .

ZHANG S , LIU K H , ZHANG Y L , et al . Robust multi target device-free localization and tracking via visible light sensing [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 17 ): 16446 - 16462 .

ZHANG S , LIU K H , ZHANG Y L , et al . A coarse fingerprint-assisted multiple target indoor device-free localization with visible light sensing [J ] . IEEE Sensors Journal , 2022 , 22 ( 2 ): 1461 - 1473 .

YU L , ABUELLA H , ISLAM M Z , et al . Gesture recognition using reflected visible and infrared light wave signals [J ] . IEEE Transactions on Human-Machine Systems , 2021 , 51 ( 1 ): 44 - 55 .

MA D , LAN G H , HASSAN M , et al . Solar gest:ubiquitous and battery-free gesture recognition using solar cells [C ] // Proceedings of Mobi Com’ 19:The 25th Annual International Conference on Mobile Computing and Networking . New York:ACM Press , 2019 : 1 - 15 .

ZHANG D , PARK J W , ZHANG Y , et al . Op to sense:towards ubiquitous self-powered ambient light sensing surfaces [J ] . Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies,New York:ACM Press , 2020 , 4 ( 3 ): 1 - 27 .

Views

358

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

3D gesture recognition based on dynamic template matching algorithm for intelligent wearable devices

Related Author

Yunhe LI

Related Institution

No data

⁰