基于可见光的环境自适应手势识别系统

王柱; 张化磊; 胡千红; 於志文

doi:10.11959/j.issn.2096-3750.2023.00344

您当前的位置：

首页 >

文章列表页 >

基于可见光的环境自适应手势识别系统

专题：智能感知技术 | 更新时间：2024-08-16

- 基于可见光的环境自适应手势识别系统
- An environment adaptive gesture recognition system based on visible light
- 物联网学报 2023年7卷第2期页码：15-25
- 作者机构：
  
  1. 西北工业大学计算机学院，陕西西安 710072
  2. 湖南文理学院计算机与电气工程学院，湖南常德 415006
- 作者简介：
  
  [ "王柱（1983- ），男，博士，西北工业大学教授，主要研究方向为智能感知与普适计算" ]
  [ "张化磊（1998- ），男，西北工业大学计算机学院硕士生，主要研究方向为无线感知" ]
  [ "胡千红（1993- ），女，湖南文理学院计算机与电气工程学院讲师，主要研究方向为移动开发、无线感知" ]
  [ "於志文（1977- ），男，博士，西北工业大学教授，主要研究方向为智能感知、群智计算、人机计算" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61960206008);国家自然科学基金资助项目;The National Natural Science Foundation of China(62072375);湖南文理学院校级科研项目;The Research Project of Hunan University of Arts and Science(E06020042)
- DOI：10.11959/j.issn.2096-3750.2023.00344
  中图分类号： TP391
- 纸质出版日期：2023-06-30，
  
  网络出版日期：2023-06，
- 稿件说明：
移动端阅览
王柱, 张化磊, 胡千红, 等. 基于可见光的环境自适应手势识别系统[J]. 物联网学报, 2023,7(2):15-25.

ZHU WANG, HUALEI ZHANG, QIANHONG HU, et al. An environment adaptive gesture recognition system based on visible light. [J]. Chinese journal on internet of things, 2023, 7(2): 15-25.
王柱, 张化磊, 胡千红, 等. 基于可见光的环境自适应手势识别系统[J]. 物联网学报, 2023,7(2):15-25. DOI： 10.11959/j.issn.2096-3750.2023.00344.

ZHU WANG, HUALEI ZHANG, QIANHONG HU, et al. An environment adaptive gesture recognition system based on visible light. [J]. Chinese journal on internet of things, 2023, 7(2): 15-25. DOI： 10.11959/j.issn.2096-3750.2023.00344.

摘要

手势日益成为一种重要的人机交互方式，可在电子游戏、虚拟现实等场景中为用户提供更优质的体验。近年来，研究者探索利用不同感知技术实现手势识别，如射频信号、声学信号等。与之相比，利用可见光识别手势具有更强普适性。基本原理为：不同手势遮挡可见光会产生独特的阴影模式，通过光电传感器捕捉阴影变化即可实现手势识别。针对可见光手势识别面临的环境依赖难题，设计了一种基于光电传感器阵列的数字手势识别系统，提出了基于图像的阵列感知数据抽象表示模型，结合图像固有特性发掘不同传感器数据之间的时间和空间关联性，利用时空特征设计了基于CNN-RNN的环境自适应手势识别方法。为了验证所提方法的有效性，设计了环境自适应手势识别系统Vi-Gesture，准确率相比基线方法提升10%以上。

Abstract

Gesture-based human-machine interaction is becoming more and more important

which can provide users with a better experience in scenarios such as video games and virtual reality.In recent years

researchers have explored different sensing technologies to facilitate gesture recognition

including RF signal

acoustic signal

etc.Compared with these approaches

visible light-based gesture recognition is a more pervasive option.The basic principle is that different gestures will produce unique shadow patterns as they block the visible light

and gesture recognition can be achieved by capturing shadow changes through photoelectric sensors.To address the environment-dependent problem faced by existing solutions

a digit gesture recognition system was designed based on the photoelectric sensor array.In particular

by modeling recordings of the sensor array as images

the temporal and spatial correlation between different sensor recordings was discovered.An environment adaptive gesture recognition method was designed based on CNN-RNN by fusing the spatio-temporal features.To verify the effectiveness of the proposed method

a prototype gesture recognition system was designed

named Vi-Gesture.Experimental results show that the proposed method outperforms baselines by more than 10% in recognition accuracy.

关键词

可见光感知手势识别环境自适应时空特征CNN-RNN

Keywords

visible light sensinggesture recognitionenvironment adaptivespatio-temporal featureCNN-RNN

references

SHEN S, GU K, CHEN X R ,et al. Gesture recognition through sEMG with wearable device based on deep learning[J]. Mobile Networksand Applications, 2020,25(6): 2447-2458.

CHAKRABORTY B K, SARMA D, BHUYAN M K ,et al. Review of constraints on vision-based gesture recognition for human-computer interaction[J]. IET Computer Vision, 2018,12(1): 3-15.

WANG Y W, SHEN J X, ZHENG Y Q . Push the limit of acoustic gesture recognition[C]// Proceedings of IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2020: 566-575.

GAO R Y, ZHANG M, ZHANG J ,et al. Towards position-independent sensing for gesture recognition with Wi-Fi[J]. Proceedings of the AC Mon Interactive,Mobile,Wearable and Ubiquitous Technologies, 2021,5(2): 1-28.

ZHANG C, TABOR J, ZHANG J L ,et al. Extending mobile interaction through near-field visible light sensing[C]// Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. New York:ACM, 2015: 345-357.

LIT X, ANC K, TIAN Z ,et al. Human sensing using visible light communication[C]// Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. New York:ACM, 2015: 331-344.

KAHOLOKULA D . Reusing ambient light to recognize hand gestures[EB]. 2016.

HU W J, GU H, PU Q F . Light sync:unsynchronized visual communication over screen-camera links[C]// Proceedings of the 19th Annual International Conference on Mobile Computing ＆ Networking. New York:ACM Press, 2013: 15-26.

KIMH S, KIMD R, YANG S H ,et al. An indoor visible light communication positioning system using a RF carrier allocation technique[J]. Journal of Light Wave Technology, 2013,31(1): 134-144.

ELAMASSIE M, KARBALAYGHAREH M, MIRAMIRKHANI F ,et al. Effect of fog and rain on the performance of vehicular visible light communications[C]// Proceedings of 2018 IEEE 87th Vehicular Technology Conference (VTC Spring). Piscataway:IEEE Press, 2018: 1-6.

SUN S Y, YANG F, SONG J . Sum rate maximization for intelligent reflecting surface-aided visible light communications[J]. IEEE Communications Letters, 2021,25(11): 3619-3623.

KUOY S, PANNUTO P, HSIAOK J ,et al. Luxapose:indoor positioning with mobile phones and visible light[C]// Proceedings of the 20th Annual International Conference on Mobile Computing and Networking. New York:ACM Press, 2014: 447-458.

LI L, HU P, PENG C ,et al. Epsilon:a visible light-based positioning system[C]// Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI’ 14),331-343, 2014.

ALAM F, CHEW M T, WENGE T ,et al. An accurate visible light positioning system using regenerated fingerprint database based on calibrated propagation model[J]. IEEE Transactions on Instrumentation and Measurement, 2019,68(8): 2714-2723.

MAJEED K, HRANILOVIC S . Passive indoor visible light positioning system using deep learning[J]. IEEE Internet of Things Journal, 2021,8(19): 14810-14821.

YANG Y B, HAO J, LUO J ,et al. Ceiling see:device-free occupancy inference through lighting infrastructure based LED sensing[C]// Proceedings of 2017 IEEE International Conference on Pervasive Computing and Communications (Per Com). Piscataway:IEEE Press, 2017: 247-256.

LI T X, LIU Q, ZHOU X . Practical human sensing in the light[C]// Proceedings of the 14th Annual International Conference on Mobile Systems,Applications,and Services. New York:ACM Press, 2016: 71-84.

LIT X, XIONG X, XIE Y F ,et al. Reconstructing hand poses using visible light[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2017,1(3): 1-20.

VENKATNARAYANR H, SHAHZAD M . Gesture recognition using ambient light[J]. Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies, 2018,2(1): 1-28.

HU Q, YU Z, WANG Z ,et al. Vi hand:gesture recognition with ambient light[C]// Proceedings of the IEEE International Conference on Ubiquitous Intelligence and Computing,[S.l.:s.n.], 2019.

LAI K, YANUSHKEVICHS N . CNN RNN depth and skeleton based dynamic hand gesture recognition[C]// Proceedings of 2018 24th International Conference on Pattern Recognition (ICPR). Piscataway:IEEE Press, 2018: 3451-3456.

WEBBER J, MEHBODNIYA A, TENG R ,et al. Human-machine interaction using probabilistic neural network for light communication systems[J]. Electronics, 2022,11(6): 932.

LEYS C, LEY C, KLEIN C ,et al. Detecting outliers:do not use standard deviation around the mean,use absolute deviation around the Median[J]. Journal of Experimental Social Psychology, 2013,49(4): 764-766.

MA D, LAN G H, HU C S ,et al. Recognizing hand gestures using solar cells[J]. IEEE Transactions on Mobile Computing, 2022,PP(99): 1.

WEBBER J, MEHBODNIYA A, ARAFA A ,et al. Gesture recognition using machine learning for light communication systems[C]// Proceedings of 2022 International Mobile and Embedded Technology Conference (MECON). Piscataway:IEEE Press, 2022: 52-56.

ZHANG S, LIU K H, ZHANG Y L ,et al. Robust multi target device-free localization and tracking via visible light sensing[J]. IEEE Internet of Things Journal, 2022,9(17): 16446-16462.

ZHANG S, LIU K H, ZHANG Y L ,et al. A coarse fingerprint-assisted multiple target indoor device-free localization with visible light sensing[J]. IEEE Sensors Journal, 2022,22(2): 1461-1473.

YU L, ABUELLA H, ISLAM M Z ,et al. Gesture recognition using reflected visible and infrared light wave signals[J]. IEEE Transactions on Human-Machine Systems, 2021,51(1): 44-55.

MA D, LAN G H, HASSAN M ,et al. Solar gest:ubiquitous and battery-free gesture recognition using solar cells[C]// Proceedings of Mobi Com’ 19:The 25th Annual International Conference on Mobile Computing and Networking. New York:ACM Press, 2019: 1-15.

ZHANG D, PARK J W, ZHANG Y ,et al. Op to sense:towards ubiquitous self-powered ambient light sensing surfaces[J]. Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies,New York:ACM Press, 2020,4(3): 1-27.

浏览量

220

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

智能穿戴设备基于动态模板匹配算法的3D手势识别