基于超声波感知的手势重建和识别

田跃悦; 黄家栋; 尹华锐; 陈力

doi:null

您当前的位置：

首页 >

文章列表页 >

基于超声波感知的手势重建和识别

更新时间：2026-04-10

- 基于超声波感知的手势重建和识别
- Gesture reconstruction and recognition based on ultrasonic sensing
- 物联网学报 2026年
- 作者机构：
  
  中国科学技术大学，安徽合肥 230022
- 作者简介：
  
  陈力，chenli87@ustc.edu.cn
- 基金信息：
  
  国家自然科学基金青年基金（B类）(62522126);安徽省杰出青年基金(2308085J24)
- DOI：
  中图分类号： TN92
- 收稿：2025-12-16，
  
  修回：2026-01-14，
  
  录用：2026-03-23，
- 稿件说明：
移动端阅览
田跃悦, 黄家栋, 尹华锐, 等. 基于超声波感知的手势重建和识别[J/OL]. 物联网学报, 2026.

Tian Yueyue, Huang jiadong, Yin Huarui, et al. Gesture reconstruction and recognition based on ultrasonic sensing[J/OL]. Chinese Journal on Internet of Things, 2026.
田跃悦, 黄家栋, 尹华锐, 等. 基于超声波感知的手势重建和识别[J/OL]. 物联网学报, 2026. DOI：

Tian Yueyue, Huang jiadong, Yin Huarui, et al. Gesture reconstruction and recognition based on ultrasonic sensing[J/OL]. Chinese Journal on Internet of Things, 2026. DOI：

摘要

为解决基于视觉的手势感知中存在的隐私泄露风险以及其他非视觉信号感知成本高的问题，本文研究基于超声波信号的手势重建和识别。对采集的超声回波和手势图片数据处理构造出一个超声-手势框架数据集Ultrasonic Gesture。基于该数据集，提出了一种具有高性能局部感知与全局建模能力的CAMT-Net神经网络，实现从超声波信号到二维手势关键点坐标的高精度端到端映射。在包含六种静态手势的数据集上进行实验，所提方法生成的关键点精度接近基于RGB图像的重建方法；进一步基于重建关键点进行手势识别，准确率达到89%。结果表明，超声波信号可有效支持细粒度手势感知任务。

Abstract

To address the privacy leakage risks in vision-based gesture sensing and the high sensing costs of other non-visual signals

this paper investigated a gesture reconstruction and recognition method based on ultrasonic signals. The collected ultrasonic echo and gesture image data were processed to construct the Ultrasonic Gesture dataset. Based on this dataset

we proposed a CAMT-Net with high-performance local perception and global modeling capabilities

achieving high-precision end-to-end mapping from ultrasonic signals to 2D gesture keypoint coordinates. Experiments were conducted on the dataset containing six static gestures. The proposed method achieved accuracy close to the metods based on RGB images. Further gesture recognition based on the reconstructed keypoints reached an accuracy of 89%. The results indicate that ultrasonic signals can effectively support fine-grained gesture perception tasks.

关键词

Keywords

references

GAVGIOTAKI D , NTOA S , MARGETIS G , et al . Gesture-based Interaction for AR Systems: A Short Review [C ] // Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments . New York : ACM Press , 2023 : 284 - 292 .

GAO Q , LIU J G , JU Z J . Robust real-time hand detection and localization for space human–robot interaction based on deep learning [J ] . Neurocomputing , 2020 , 390 : 198 – 206 .

WANG R Y , POPOVIĆ J . Real-time hand-tracking with a color glove [J ] . ACM Transactions on Graphics , 2009 , 28 ( 3 ): 1 - 8 .

HU F , HE P , XU S L , et al . FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist [J ] . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , 2020 , 4 ( 2 ): 1 - 24 .

ZHANG F , ZHAO L , LI S , et al . 3D hand pose and shape estimation from monocular RGB via efficient 2D cues [J ] . Computational Visual Media , 2024 , 10 ( 1 ): 79 – 96 .

MUELLER F , BERNARD F , SOTNYCHENKO O , et al . GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 49 – 59 .

SMITH B , WU C , WEN H , et al . Constraining dense hand surface tracking with elasticity [J ] . ACM Transactions on Graphics , 2020 , 39 ( 6 ): 1 - 14 .

LIEN J , GILLIAN N , KARAGOZLER M E , et al . Soli: ubiquitous gesture sensing with millimeter wave radar [J ] . ACM Transactions on Graphics , 2016 , 35 ( 4 ): 1 - 19 .

GLANDON A , VIDYARATNE L , SADEGHZADEHYAZDI N , et al . 3D Skeleton Estimation and Human Identity Recognition Using Lidar Full Motion Video [C ] // 2019 International Joint Conference on Neural Networks (IJCNN) . Piscataway : IEEE Press , 2019 : 1 – 8 .

MAO W G , HE J , ZHENG H H , et al . High-precision acoustic motion tracking: demo [C ] // Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking . Piscataway : IEEE Press , 2016 : 491 - 492 .

SHARP T , KESKIN C , ROBERTSON D , et al . Accurate, Robust, and Flexible Real-time Hand Tracking [C ] // Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems . New York : ACM Press , 2015 : 3633 - 3642 .

SRIDHAR S , MUELLER F , OULASVIRTA A , et al . Fast and robust hand tracking using detection-guided optimization [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE Press , 2015 : 3213 - 3221 .

SRIDHAR S , OULASVIRTA A , THEOBALT C , et al . Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data [C ] // 2013 IEEE International Conference on Computer Vision . Piscataway : IEEE Press , 2013 : 2456 – 2463 .

KESKIN C , KIRAÇ F , KARA Y E , et al . Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests [C ] // Computer Vision – ECCV 2012 . Piscataway : IEEE Press , 2012 : 852 – 863 .

TANG D , CHANG H J , TEJANI A , et al . Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2014 : 3786 – 3793 .

TOMPSON J , STEIN M , LECUN Y , et al . Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks [J ] . ACM Transactions on Graphics , 2014 , 33 ( 5 ): 1 - 10 .

SIMON T , JOO H , MATTHEWS I , et al . Hand Keypoint Detection in Single Images Using Multiview Bootstrapping [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2017 : 4645 – 4653 .

WANG Y G , PENG C , LIU Y . Mask-Pose Cascaded CNN for 2D Hand Pose Estimation From Single Color Image [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2019 , 29 ( 11 ): 3258 – 3268 .

CHEN X , LIU Y , DONG Y , et al . MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2022 : 20512 – 20522 .

CHEN X , LIU Y , MA C , et al . Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2021 : 13269 – 13278 .

TANG X , WANG T , FU C W . Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE Press , 2021 : 11678 – 11687 .

KONG H , LYU H , YU J , et al . mmHand: 3D Hand Pose Estimation Leveraging mmWave Signals [C ] // 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS) . Piscataway : IEEE Press , 2024 : 1062 – 1073 .

JI S J , ZHANG X Y , ZHENG Y Q , et al . Construct 3D Hand Skeleton with Commercial WiFi [C ] // Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems . New York : ACM Press , 2024 : 322 - 334 .

LI D , LIU J L , LEE S I , et al . FM-track: pushing the limits of contactless multi-target tracking using acoustic signals [C ] // Proceedings of the 18th Conference on Embedded Networked Sensor Systems . New York : ACM Press , 2020 : 150 - 163 .

MAO W G , WANG M , SUN W , et al . RNN-Based Room Scale Hand Motion Tracking [C ] // The 25th Annual International Conference on Mobile Computing and Networking . New York : ACM Press , 2019 : 1 - 16 .

WANG W , LIU A X , SUN K . Device-free gesture tracking using acoustic signals [C ] // Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking . New York : ACM Press , 2016 : 82 - 94 .

YANG Y J , CHEN T , AN Z L , et al . LeakyFeeder: In-Air Gesture Control Through Leaky Acoustic Waves [C ] // Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems . New York : ACM Press , 2025 : 144 – 157 .

WANG S Y , PU H L , CAO Q M , et al . RAM-Hand: Robust Acoustic Multi-Hand Pose Reconstruction Using a Microphone Array [C ] // Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems . New York : ACM Press , 2025 : 130 – 143 .

FENG Z H , KITTLER J , AWAIS M , et al . Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE Press , 2018 : 2235 – 2245 .

YANG Y , RAMANAN D . Articulated Human Detection with Flexible Mixtures of Parts [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 12 ): 2878 – 2890 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

非视距条件下基于特征融合的UWB到达角估计方法

基于轻量并行去噪网络的OTFS信道估计算法

融合注意力机制的3D打印缺陷检测算法

深度学习在眼科疾病辅助诊断中的应用综述

物联网中模型剪枝技术：现状、方法和展望