基于DNN卷积核分割的边缘协作推理性能分析

郅佳琳; 滕颖蕾; 张新阳; 牛涛; 宋梅

doi:10.11959/j.issn.2096-3750.2022.00308

您当前的位置：

首页 >

文章列表页 >

基于DNN卷积核分割的边缘协作推理性能分析

理论与技术 | 更新时间：2024-08-16

- 基于DNN卷积核分割的边缘协作推理性能分析
- Cooperative inference analysis based on DNN convolutional kernel partitioning
- 物联网学报 2022年6卷第4期页码：72-81
- 作者机构：
- 作者简介：
  
  [ "郅佳琳（1999- ），女，北京邮电大学硕士生，主要研究方向为边缘计算、深度学习等" ]
  [ "滕颖蕾（1983- ），女，博士，北京邮电大学教授、博士生导师，IEEE高级会员。主要研究方向为AI与无线通信，边缘计算及毫米波技术等" ]
  [ "张新阳（2000- ），男，北京邮电大学硕士生，主要研究方向为边缘智能、资源分配等" ]
  [ "牛涛（1997- ），男，北京邮电大学博士生，主要研究方向为边缘计算、人工智能等" ]
  [ "宋梅（1960- ），女，北京邮电大学电子工程学院教授、博士生导师，中国电子教育学会研究生教育分会常务理事，中国电子学会通信分会委员，中国电子学会物联网专家委员会委员，中国铁道学会信息化委员会委员，主要研究方向为数据与服务、通信与管理等" ]
- 基金信息：
  
  国家重点研发计划;The National Key Research and Development Program of China(2021YFB3300100);国家自然科学基金资助项目;The National Natural Science Foundation of China(62171062)
- DOI：10.11959/j.issn.2096-3750.2022.00308
  中图分类号： TN911.22
- 纸质出版日期：2022-12-30，
  
  网络出版日期：2022-12，
- 稿件说明：
移动端阅览
郅佳琳, 滕颖蕾, 张新阳, 等. 基于DNN卷积核分割的边缘协作推理性能分析[J]. 物联网学报, 2022,6(4):72-81.

JIALIN ZHI, YINGLEI TENG, XINYANG ZHANG, et al. Cooperative inference analysis based on DNN convolutional kernel partitioning. [J]. Chinese journal on internet of things, 2022, 6(4): 72-81.
郅佳琳, 滕颖蕾, 张新阳, 等. 基于DNN卷积核分割的边缘协作推理性能分析[J]. 物联网学报, 2022,6(4):72-81. DOI： 10.11959/j.issn.2096-3750.2022.00308.

JIALIN ZHI, YINGLEI TENG, XINYANG ZHANG, et al. Cooperative inference analysis based on DNN convolutional kernel partitioning. [J]. Chinese journal on internet of things, 2022, 6(4): 72-81. DOI： 10.11959/j.issn.2096-3750.2022.00308.

摘要

随着智能芯片在边缘终端设备的普及，未来大量的AI应用将部署在更靠近数据源的网络边缘。基于DNN的分割方法可以实现深度学习模型在资源受限的终端设备上训练和部署，解决边缘 AI 算力瓶颈问题。在传统基于工作负载的分割方案（WPM

workload based partition method）的基础上，提出基于卷积核的分割方案（KPM

kernel based partition method），分别从计算量、内存占用、通信开销3个方面进行推理性能的定量分析，并从推理过程灵活性、鲁棒性、隐私性角度进行定性分析。最后搭建软硬件实验平台，使用PyTorch实现AlexNet和VGG11网络进一步验证所提方案在时延和能耗方面的性能优势，相比于传统工作负载分割方案，所提卷积核分割方案在大规模计算场景下有更好的DNN推理加速效果，且具有更低的内存占用和能量消耗。

Abstract

With the popularity of intelligent chip in the application of edge terminal devices

a large number of AI applications will be deployed on the edge of networks closer to data sources in the future.The method based on DNN partition can realize deep learning model training and deployment on resource-constrained terminal devices

and solve the bottleneck problem of edge AI computing ability.Thekernel based partition method (KPM) was proposed as a new scheme on the basis of traditional workload based partition method (WPM).The quantitative analysis of inference performance was carried out from three aspects of computation FLOPS

memory consumption and communication cost respectively

and the qualitative analysis of the above two schemes was carried out from the perspective of flexibility

robustness and privacy of inference process.Finally

a software and hardware experimental platform was built

and AlexNet and VGG11 networks were implemented using PyTorch to further verify the performance advantages of the proposed scheme in terms of delay and energy consumption.It was concluded that

compared with the WPM scheme

the KPM scheme had better DNN reasoning acceleration effect in large-scale computing scenarios.And it has lower memory usage and energy consumption.

关键词

边缘智能深度神经网络分割协作计算并行推理

Keywords

edge intelligencedeep neural network partitioncooperative computationparallel partition

references

ZHOU Z, CHEN X, LI E ,et al. Edge intelligence:paving the last mile of artificial intelligence with edge computing[J]. Proceedings of the IEEE, 2019,107(8): 1738-1762.

DENG S G, ZHAO H L, FANG W J ,et al. Edge intelligence:the confluence of edge computing and artificial intelligence[J]. IEEE Internet of Things Journal, 2020,7(8): 7457-7469.

ZHANG Y, MA X, ZHANG J ,et al. Edge intelligence in the cognitive Internet of Things:improving sensitivity and interactivity[J]. IEEE Network, 2019,33(3): 58-64.

CEVALLOS MORENO J F, SATTLER R, CAULIER CISTERNA R P ,et al. Online service function chain deployment for live-streaming in virtualized content delivery networks:a deep reinforcement learning approach[J]. Future Internet, 2021,13(11): 278.

CHIANG M, ZHANG T . Fog and IoT:an overview of research opportunities[J]. IEEE Internet of Things Journal, 2016,3(6): 854-864.

HUI H W, ZHOU C C, XU S G ,et al. A novel secure data transmission scheme in industrial Internet of things[J]. China Communications, 2020,17(1): 73-88.

XU Z C, ZHAO L Q, LIANG W F ,et al. Energy-aware inference offloading for DNN-driven applications in mobile edge clouds[J]. IEEE Transactions on Parallel and Distributed Systems, 2021,32(4): 799-814.

SUN Y, CUI Y N, HUANG Y H ,et al. SDMP:a secure detector for epidemic disease file based on DNN[J]. Information Fusion, 2021,68: 1-7.

TIAN X Z, ZHU J, XU T ,et al. Mobility-included DNN partition offloading from mobile devices to edge clouds[J]. Sensors (Basel,Switzerland), 2021,21(1): 229.

PARTHASARATHY A, KRISHNAMACHARI B . DEFER:distributed edge inference for deep neural networks[C]// 2022 14th International Conference on COMmunication Systems ＆ NETworkS (COMSNETS). Piscataway:IEEE Press, 2022: 749-753.

REN P, QIAO X Q, HUANG Y K ,et al. Fine-grained elastic partitioning for distributed DNN towards mobile web AR services in the 5G era[J]. IEEE Transactions on Services Computing, 2021,PP(99): 1.

TANG X, CHEN X, ZENG L K ,et al. Joint multiuser DNN partitioning and computational resource allocation for collaborative edge intelligence[J]. IEEE Internet of Things Journal, 2021,8(12): 9511-9522.

ESHRATIFAR A E, PEDRAM M . Energy and performance efficient computation offloading for deep neural networks in a mobile cloud computing environment[C]// GLSVLSI '18:Proceedings of the 2018 on Great Lakes Symposium on VLSI.[S.l.:s.n], 2018: 111-116.

ESHRATIFAR A E, ABRISHAMI M S, PEDRAM M . JointDNN:an efficient training and inference engine for intelligent mobile cloud computing services[J]. IEEE Transactions on Mobile Computing, 2021,20(2): 565-576.

MOHAMMED T, JOE WONG C, BABBAR R ,et al. Distributed inference acceleration with adaptive DNN partitioning and offloading[C]// IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2020: 854-863.

ZHAO Z R, BARIJOUGH K M, GERSTLAUER A . DeepThings:distributed adaptive deep learning inference on resource-constrained IoT edge clusters[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018,37(11): 2348-2359.

KRIZHEVSKY A, SUTSKEVER I, HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.

MOLCHANOV P, TYREE S, KARRAS T ,et al. Pruning convolutional neural networks for resource efficient transfer learning[EB]. 2016.

ZENG L K, CHEN X, ZHOU Z ,et al. CoEdge:cooperative DNN inference with adaptive workload partitioning over heterogeneous edge devices[J]. IEEE/ACM Transactions on Networking, 2021,29(2): 595-608.

SADEGHI A R, WACHSMANN C, WAIDNER M . Security and privacy challenges in industrial Internet of things[C]// Proceedings of the 52nd Annual Design Automation Conference. New York:ACM Press, 2015: 1-6.

DIN N, CHEN H P, KHAN D . Mobility-aware resource allocation in multi-access edge computing using deep reinforcement learning[C]// Proceedings of 2019 IEEE Intl Conf on Parallel ＆ Distributed Processing with Applications,Big Data ＆ Cloud Computing,Sustainable Computing ＆ Communications,Social Computing ＆ Networking. Piscataway:IEEE Press, 2019: 202-209.

KILCIOGLU E, MIRGHASEMI H, STUPIA I ,et al. An energy-efficient fine-grained deep neural network partitioning scheme for wireless collaborative fog computing[J]. IEEE Access, 2021: 79611-79627.

KOCHER P, HORN J, FOGH A ,et al. Spectre attacks:exploiting speculative execution[C]// Proceedings of 2019 IEEE Symposium on Security and Privacy. Piscataway:IEEE Press, 2019: 1-19.

TRAMÈR F, ZHANG F, JUELS A ,et al. Stealing machine learning models via prediction {APIs}[C]// 25th USENIX Security Symposium (USENIX Security 16).[S.l.:s.n], 2016: 601-618.

JUVEKAR C, VAIKUNTANATHAN V, CHANDRAKASAN A . {GAZELLE}:a low latency framework for secure neural network inference[C]// 27th USENIX Security Symposium (USENIX Security 18).[S.l.:s.n], 2018: 1651-1669.

GILAD-BACHRACH R, DOWLIN N, LAINE K ,et al. Cryptonets:applying neural networks to encrypted data with high throughput and accuracy[C]// International conference on machine learning. New York:PMLR, 2016: 201-210.

ZHAO L, TAN W A, XIE N ,et al. An optimal service selection approach for service-oriented business collaboration using crowd-based cooperative computing[J]. Applied Soft Computing, 2020,92: 106270.

HE K M, ZHANG X Y, REN S Q ,et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 770-778.

WU Y H, WANG Y H, ZHOU F H ,et al. Computation efficiency maximization in OFDMA-based mobile edge computing networks[J]. IEEE Communications Letters, 2020,24(1): 159-163.

浏览量

253

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向低空物联网的云-边协同演进模型与通信范式

车联网边缘智能：概念、架构、问题、实施和展望

边缘智能驱动的高能效无人机自主导航算法研究

面向万物智联的语义通信网络

边缘智能驱动的车联网