Cooperative inference analysis based on DNN convolutional kernel partitioning

Jialin ZHI; Yinglei TENG; Xinyang ZHANG; Tao NIU; Mei SONG

doi:10.11959/j.issn.2096-3750.2022.00308

您当前的位置：

首页 >

文章列表页 >

Cooperative inference analysis based on DNN convolutional kernel partitioning

Theory and Technology | 更新时间：2024-08-16

- Cooperative inference analysis based on DNN convolutional kernel partitioning
- Chinese Journal on Internet of Things Vol. 6, Issue 4, Pages: 72-81(2022)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2022.00308
  CLC： TN911.22
- Online First：2022-12，
  
  Published：30 December 2022
- 稿件说明：
移动端阅览
Jialin ZHI, Yinglei TENG, Xinyang ZHANG, et al. Cooperative inference analysis based on DNN convolutional kernel partitioning[J]. Chinese Journal on Internet of Things, 2022, 6(4): 72-81.
DOI：

Jialin ZHI, Yinglei TENG, Xinyang ZHANG, et al. Cooperative inference analysis based on DNN convolutional kernel partitioning[J]. Chinese Journal on Internet of Things, 2022, 6(4): 72-81. DOI： 10.11959/j.issn.2096-3750.2022.00308.

摘要

随着智能芯片在边缘终端设备的普及，未来大量的AI应用将部署在更靠近数据源的网络边缘。基于DNN的分割方法可以实现深度学习模型在资源受限的终端设备上训练和部署，解决边缘 AI 算力瓶颈问题。在传统基于工作负载的分割方案（WPM

workload based partition method）的基础上，提出基于卷积核的分割方案（KPM

kernel based partition method），分别从计算量、内存占用、通信开销3个方面进行推理性能的定量分析，并从推理过程灵活性、鲁棒性、隐私性角度进行定性分析。最后搭建软硬件实验平台，使用PyTorch实现AlexNet和VGG11网络进一步验证所提方案在时延和能耗方面的性能优势，相比于传统工作负载分割方案，所提卷积核分割方案在大规模计算场景下有更好的DNN推理加速效果，且具有更低的内存占用和能量消耗。

Abstract

With the popularity of intelligent chip in the application of edge terminal devices

a large number of AI applications will be deployed on the edge of networks closer to data sources in the future.The method based on DNN partition can realize deep learning model training and deployment on resource-constrained terminal devices

and solve the bottleneck problem of edge AI computing ability.Thekernel based partition method (KPM) was proposed as a new scheme on the basis of traditional workload based partition method (WPM).The quantitative analysis of inference performance was carried out from three aspects of computation FLOPS

memory consumption and communication cost respectively

and the qualitative analysis of the above two schemes was carried out from the perspective of flexibility

robustness and privacy of inference process.Finally

a software and hardware experimental platform was built

and AlexNet and VGG11 networks were implemented using PyTorch to further verify the performance advantages of the proposed scheme in terms of delay and energy consumption.It was concluded that

compared with the WPM scheme

the KPM scheme had better DNN reasoning acceleration effect in large-scale computing scenarios.And it has lower memory usage and energy consumption.

关键词

Keywords

references

ZHOU Z , CHEN X , LI E , et al . Edge intelligence:paving the last mile of artificial intelligence with edge computing [J ] . Proceedings of the IEEE , 2019 , 107 ( 8 ): 1738 - 1762 .

DENG S G , ZHAO H L , FANG W J , et al . Edge intelligence:the confluence of edge computing and artificial intelligence [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 8 ): 7457 - 7469 .

ZHANG Y , MA X , ZHANG J , et al . Edge intelligence in the cognitive Internet of Things:improving sensitivity and interactivity [J ] . IEEE Network , 2019 , 33 ( 3 ): 58 - 64 .

CEVALLOS MORENO J F , SATTLER R , CAULIER CISTERNA R P , et al . Online service function chain deployment for live-streaming in virtualized content delivery networks:a deep reinforcement learning approach [J ] . Future Internet , 2021 , 13 ( 11 ): 278 .

CHIANG M , ZHANG T . Fog and IoT:an overview of research opportunities [J ] . IEEE Internet of Things Journal , 2016 , 3 ( 6 ): 854 - 864 .

HUI H W , ZHOU C C , XU S G , et al . A novel secure data transmission scheme in industrial Internet of things [J ] . China Communications , 2020 , 17 ( 1 ): 73 - 88 .

XU Z C , ZHAO L Q , LIANG W F , et al . Energy-aware inference offloading for DNN-driven applications in mobile edge clouds [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 4 ): 799 - 814 .

SUN Y , CUI Y N , HUANG Y H , et al . SDMP:a secure detector for epidemic disease file based on DNN [J ] . Information Fusion , 2021 , 68 : 1 - 7 .

TIAN X Z , ZHU J , XU T , et al . Mobility-included DNN partition offloading from mobile devices to edge clouds [J ] . Sensors (Basel,Switzerland) , 2021 , 21 ( 1 ): 229 .

PARTHASARATHY A , KRISHNAMACHARI B . DEFER:distributed edge inference for deep neural networks [C ] // 2022 14th International Conference on COMmunication Systems ＆ NETworkS (COMSNETS) . Piscataway:IEEE Press , 2022 : 749 - 753 .

REN P , QIAO X Q , HUANG Y K , et al . Fine-grained elastic partitioning for distributed DNN towards mobile web AR services in the 5G era [J ] . IEEE Transactions on Services Computing , 2021 , PP ( 99 ): 1 .

TANG X , CHEN X , ZENG L K , et al . Joint multiuser DNN partitioning and computational resource allocation for collaborative edge intelligence [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 12 ): 9511 - 9522 .

ESHRATIFAR A E , PEDRAM M . Energy and performance efficient computation offloading for deep neural networks in a mobile cloud computing environment [C ] // GLSVLSI '18:Proceedings of the 2018 on Great Lakes Symposium on VLSI .[S.l.:s.n ] , 2018 : 111 - 116 .

ESHRATIFAR A E , ABRISHAMI M S , PEDRAM M . JointDNN:an efficient training and inference engine for intelligent mobile cloud computing services [J ] . IEEE Transactions on Mobile Computing , 2021 , 20 ( 2 ): 565 - 576 .

MOHAMMED T , JOE WONG C , BABBAR R , et al . Distributed inference acceleration with adaptive DNN partitioning and offloading [C ] // IEEE INFOCOM 2020 - IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2020 : 854 - 863 .

ZHAO Z R , BARIJOUGH K M , GERSTLAUER A . DeepThings:distributed adaptive deep learning inference on resource-constrained IoT edge clusters [J ] . IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2018 , 37 ( 11 ): 2348 - 2359 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

MOLCHANOV P , TYREE S , KARRAS T , et al . Pruning convolutional neural networks for resource efficient transfer learning [EB ] . 2016 .

ZENG L K , CHEN X , ZHOU Z , et al . CoEdge:cooperative DNN inference with adaptive workload partitioning over heterogeneous edge devices [J ] . IEEE/ACM Transactions on Networking , 2021 , 29 ( 2 ): 595 - 608 .

SADEGHI A R , WACHSMANN C , WAIDNER M . Security and privacy challenges in industrial Internet of things [C ] // Proceedings of the 52nd Annual Design Automation Conference . New York:ACM Press , 2015 : 1 - 6 .

DIN N , CHEN H P , KHAN D . Mobility-aware resource allocation in multi-access edge computing using deep reinforcement learning [C ] // Proceedings of 2019 IEEE Intl Conf on Parallel ＆ Distributed Processing with Applications,Big Data ＆ Cloud Computing,Sustainable Computing ＆ Communications,Social Computing ＆ Networking . Piscataway:IEEE Press , 2019 : 202 - 209 .

KILCIOGLU E , MIRGHASEMI H , STUPIA I , et al . An energy-efficient fine-grained deep neural network partitioning scheme for wireless collaborative fog computing [J ] . IEEE Access , 2021 : 79611 - 79627 .

KOCHER P , HORN J , FOGH A , et al . Spectre attacks:exploiting speculative execution [C ] // Proceedings of 2019 IEEE Symposium on Security and Privacy . Piscataway:IEEE Press , 2019 : 1 - 19 .

TRAMÈR F , ZHANG F , JUELS A , et al . Stealing machine learning models via prediction {APIs} [C ] // 25th USENIX Security Symposium (USENIX Security 16) .[S.l.:s.n ] , 2016 : 601 - 618 .

JUVEKAR C , VAIKUNTANATHAN V , CHANDRAKASAN A . {GAZELLE}:a low latency framework for secure neural network inference [C ] // 27th USENIX Security Symposium (USENIX Security 18) .[S.l.:s.n ] , 2018 : 1651 - 1669 .

GILAD-BACHRACH R , DOWLIN N , LAINE K , et al . Cryptonets:applying neural networks to encrypted data with high throughput and accuracy [C ] // International conference on machine learning . New York:PMLR , 2016 : 201 - 210 .

ZHAO L , TAN W A , XIE N , et al . An optimal service selection approach for service-oriented business collaboration using crowd-based cooperative computing [J ] . Applied Soft Computing , 2020 , 92 : 106270 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 770 - 778 .

WU Y H , WANG Y H , ZHOU F H , et al . Computation efficiency maximization in OFDMA-based mobile edge computing networks [J ] . IEEE Communications Letters , 2020 , 24 ( 1 ): 159 - 163 .

Views

407

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An edge-cloud collaborative model evolution and communication paradigm in Internet of low-altitude UAV

Edge intelligence empowered internet of vehicles: concept, framework, issues, implementation, and prospect

Research on power efficient autonomous UAV navigation algorithm: an edge intelligence driven approach

Semantic communication networking for the intelligence of everything

Internet of vehicles empowered by edge intelligence

Related Author

ZHANG Hongliang

ZHANG Shuhang

YU Xinbo

Yue CAO

Hai LIN

Yongdong ZHU

Xuefeng REN

Huan ZHOU

Related Institution

School of Electronics Engineering and Computer Science, Peking University

Pengcheng Laboratory

School of Electronics, Peking University

Wuhan University

Huali SmartWays Technology Co., Ltd.

⁰