A privacy budget adaptive optimization scheme for federated computing power Internet of things

MA Wenyu; CHEN Qian; HU Yuxiang; YAN Haonan; HU Tao; YI Peng

doi:10.11959/j.issn.2096-3750.2024.00440

您当前的位置：

首页 >

文章列表页 >

A privacy budget adaptive optimization scheme for federated computing power Internet of things

Topic: Computing Power Internet of Things | 更新时间：2025-01-20

- A privacy budget adaptive optimization scheme for federated computing power Internet of things
- Chinese Journal on Internet of Things Vol. 8, Issue 4, Pages: 54-69(2024)
- 作者机构：
  
  1.信息工程大学，河南郑州 450001
  2.西安电子科技大学，陕西西安 710126
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2022YFB2901500);The National Natural Science Foundation of China(62402373)
- DOI：10.11959/j.issn.2096-3750.2024.00440
  CLC： TP309.2
- Received：16 October 2024，
  
  Revised：2024-11-24，
  
  Published：10 December 2024
- 稿件说明：
移动端阅览
马文玉,陈谦,胡宇翔等.面向联邦算力物联网的隐私预算自适应优化方案[J].物联网学报,2024,08(04):54-69.

MA Wenyu,CHEN Qian,HU Yuxiang,et al.A privacy budget adaptive optimization scheme for federated computing power Internet of things[J].Chinese Journal on Internet of Things,2024,08(04):54-69.
马文玉,陈谦,胡宇翔等.面向联邦算力物联网的隐私预算自适应优化方案[J].物联网学报,2024,08(04):54-69. DOI： 10.11959/j.issn.2096-3750.2024.00440.

MA Wenyu,CHEN Qian,HU Yuxiang,et al.A privacy budget adaptive optimization scheme for federated computing power Internet of things[J].Chinese Journal on Internet of Things,2024,08(04):54-69. DOI： 10.11959/j.issn.2096-3750.2024.00440.

摘要

联邦算力物联网（IoT

Internet of things）旨在通过联邦学习深度融合算力与物联网资源，从而实现对泛在离散部署的海量物联网数据和异构资源的高效利用。为了应对联邦算力物联网中模型反演和梯度泄露等新兴隐私攻击威胁，学术界和产业界对差分隐私（DP

differential privacy）这一高效的隐私保护技术进行了广泛研究和应用。然而，现有差分隐私技术在设定隐私预算时，未考虑本地算力节点的数据特征和隐私预算分配公平性的问题

，造成了严重的模型精度损失。因此，提出了一种面向联邦算力物联网的隐私预算自适应优化方案——基于克拉美罗下界差分隐私的联邦学习（FedCDP

federated learning based on Cramér-Rao lower bound differential privacy）。首先，基于克拉美罗下界理论分析边缘算力节点的隐私预算估计值，实现自适应隐私预算规划；其次，通过计算边缘算力节点的上传模型与算力聚合服务器的聚合模型之间的相似度和隐私预算占比，分析得到每个节点的全局贡献度，进一步联合隐私预算估计值公平实时地优化隐私预算设定。理论分析证明了该方案可确保本地模型严格遵守

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=72515496&type=

2.28600001

https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=72515504&type=

1.26999998

-差分隐私，并保证全局模型收敛。基于多个公开数据集上的实验结果表明，在满足相同隐私保护需求的前提下，该方案将全局模型精确度最多提升了10.19%。

Abstract

Federated computing power Internet of things (IoT) is designed to deeply integrate computing power with IoT resources

facilitating the efficient utilization of vast and ubiquitously dispersed IoT data and heterogeneous resources through federated learning. Faced with the threats of emerging privacy attacks

e.g.

model inversion attacks and gradient leakage attacks

the academic and industrial communities have widely investigated and applied differential privacy (DP) as an effective privacy protection technique. However

two severe challenges have not been taken into account in the existing DP budget settings

i.e.

data heterogeneity issue of local computing power nodes and the fairness of privacy budget allocation

which lead to a significant loss in model accuracy. Therefore

an adaptive optimization scheme for privacy budget was proposed in federated computing power IoT

which was called federated learning based on Cramér-Rao lower bound differential privacy (FedCDP). In specific

to adaptively adjust privacy budgets

the privacy budget estimates for edge computing power nodes based

on the Cramér-Rao lower bound theory were analyzed. Furthermore

by assessing the similarity between the local model and the aggregated model

as well as their respective privacy budget proportions

the global contribution of each node was determined

which was used to fairly

also in real time

optimize and adjust the privacy budget settings in conjunction with the estimated privacy budget. Through rigorous theoretical analysis

FedCDP achieves

-DP for local models

and ensures the convergence of the global model. Experimental results on multiple public datasets show that the proposed scheme improves the accuracy of the global model by up to 10.19% under the premise of satisfying the same privacy protection requirements.

关键词

Keywords

references

王永江 , 孙文静 , 张凌 , 等 . 物联网环境下的边缘算力部署研究 [J ] . 智能物联技术 , 2024 , 7 ( 5 ): 149 - 152 .

WANG Y J , SUN W J , ZHANG L , et al . Research on edge computing deployment in the Internet of things environment [J ] . Technology of IoT & AI , 2024 , 7 ( 5 ): 149 - 152 .

陈心瑜 , 阮凯斌 . 基于物联网节点算力性能优化分析 [J ] . 新型工业化 , 2021 , 11 ( 2 ): 7 - 9, 13 .

CHEN X Y , RUAN K B . Optimization analysis of computing performance based on Internet of things nodes [J ] . The Journal of New Industrialization , 2021 , 11 ( 2 ): 7 - 9, 13 .

王志良 , 何刚 , 俞文心 , 等 . 边缘场景下动态联邦学习优化方法 [J ] . 计算机技术与发展 , 2024 , 34 ( 2 ): 98 - 104 .

WANG Z L , HE G , YU W X , et al . Dynamic federated learning optimization method in edge scenarios [J ] . Computer Technology and Development , 2024 , 34 ( 2 ): 98 - 104 .

程帆 , 王瑞锦 , 张凤荔 . 边缘场景下动态权重的联邦学习优化方法 [J ] . 计算机科学 , 2022 , 49 ( 12 ): 53 - 58 .

CHENG F , WANG R J , ZHANG F L . Federated learning optimization method for dynamic weights in edge scenarios [J ] . Computer Science , 2022 , 49 ( 12 ): 53 - 58 .

胡卓尔 . 基于边缘智能学习的安全可信物联网研究 [D ] . 北京 : 北京邮电大学 , 2022 .

HU Z E . Research on secure and trusted Internet of things based on edge intelligent learning [D ] . Beijing : Beijing University of Posts and Telecommunications , 2022 .

涂晓儒 , 刘子威 , 张更新 . 基于语义通信的卫星物联网边缘计算架构 [J ] . 通信技术 , 2024 , 57 ( 7 ): 680 - 687 .

TU X R , LIU Z W , ZHANG G X . Edge computing architecture of satellite IoT based on semantic communication [J ] . Communications Technology , 2024 , 57 ( 7 ): 680 - 687 .

岑伯维 . 基于云原生的电力物联网边缘计算模型与方法 [D ] . 广州 : 华南理工大学 , 2023 .

CEN B W . Cloud native-based edge computing model and method for power Internet of things [D ] . Guangzhou : South China University of Technology , 2023 .

WU Q , WANG X B , FAN P Y , et al . Vehicle selection for C-V2X mode 4 based federated edge learning systems [J ] . arXiv preprint , 2024 , arXiv: 2401.07224 .

FREDRIKSON M , JHA S , RISTENPART T . Model inversion attacks that exploit confidence information and basic countermeasures [C ] // Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security . New York : ACM , 2015 : 1322 - 1333 .

ZHU L , LIU Z , HAN S . Deep leakage from gradients [C ] // Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 , Piscataway : IEEE , 2019 : 14747 - 14756 .

中国移动通信集团有限公司．算力网络技术白皮书 [R ] . 2022 .

China Mobile Communications Group Co.,Ltd. . Computing force network technology whitepaper [R ] . 2022 .

李铭轩 , 曹畅 , 唐雄燕 , 等‍ . 面向算力网络的边缘资源调度解决方案研究 [J ] . 数据与计算发展前沿 , 2020 , 2 ( 4 ): 80 - 91 .

LI M X , CAO C , TANG X Y , et al . Research on edge resource scheduling solutions for computing power network [J ] . Frontiers of Data & Computing , 2020 , 2 ( 4 ): 80 - 91 .

YAO P , WANG H , ZHENG C , et al . Efficient federated learning aggregation protocol using approximate homomorphic encryption [C ] // Proceedings of the 2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD) . Piscataway : IEEE Press , 2023 : 1884 - 1889 .

YIN L H , FENG J Y , XUN H , et al . A privacy-preserving federated learning for multiparty data sharing in social IoTs [J ] . IEEE Transactions on Network Science and Engineering , 2021 , 8 ( 3 ): 2706 - 2718 .

徐茹枝 , 戴理朋 , 夏迪娅 , 等‍ . 基于联邦学习的中心化差分隐私保护算法研究 [J ] . 信息网络安全 , 2024 , 24 ( 1 ): 69 - 79 .

XU R Z , DAI L P , XIA D Y , et al . Research on centralized differential privacy algorithm for federated learning [J ] . Netinfo Security , 2024 , 24 ( 1 ): 69 - 79 .

CHEN Q , WANG Z L , CHEN J W , et al . Dap-FL: federated learning flourishes by adaptive tuning and secure aggregation [J ] . IEEE Transactions on Parallel and Distributed Systems , 2023 , 34 ( 6 ): 1923 - 1941 .

WEI K , LI J , DING M , et al . Federated learning with differential privacy: algorithms and performance analysis [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 3454 - 3469 .

CHENG S Y , LI P , WANG R C , et al . Differentially private federated learning with non-IID data [J ] . Computing , 2024 , 106 ( 7 ): 2459 - 2488 .

唐钰慧‍ . 针对联邦学习中梯度泄露攻击的改进方法 [J ] . 信息技术与信息化 , 2023 ( 9 ): 56 - 59 .

TANG Y H . An improved method for gradient leakage attack in federated learning [J ] . Information Technology and Informatization , 2023 ( 9 ): 56 - 59 .

马卓然‍ . 隐私保护的联邦学习关键技术研究 [D ] . 西安 : 西安电子科技大学 , 2022 .

MA Z R . Research on key technologies of federated learning for privacy protection [D ] . Xi’an : Xidian University , 2022 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [J ] . arXiv preprint , 2016 , arXiv: 1602.05629 .

BI M H , YAN C H , LIN R , et al . Client scheduling and bandwidth slicing for multiple federated learning tasks over multiple passive optical networks [J ] . Computer Networks , 2024 , 243 : 110309 .

HESSEL M , SOYER H , ESPEHOLT L , et al . Multi-task deep reinforcement learning with PopArt [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 3796 - 3803 .

TU J F , HUANG J M , YANG L , et al . Personalized federated learning with layer-wise feature transformation via meta-learning [J ] . ACM Transactions on Knowledge Discovery from Data , 2023 , 18 : 1 - 21 .

汤东汰‍ . 基于差分隐私的联邦学习算法优化研究 [D ] . 西安 : 西安理工大学 , 2023 .

TANG D T . Research on optimization of federated learning algorithm based on differential privacy [D ] . Xi’an : Xi’an University of Technology , 2023 .

SU L , XU J , YANG P . A non-parametric view of FedAvg and FedProx: beyond stationary points [J ] . The Journal of Machine Learning Research , 2024 , 24 ( 1 ): 9713 - 9760 .

张兰 , 程一航 , 李向阳‍ . 提升联邦学习在Non-IID和Mismatched场景下性能的方法 : CN113379067A [P ] . 2021 .

ZHANG L , CHENG Y H , LI X Y . Method for improving federated learning performance in Non-IID and mismatched scenarios . CN patent , CN 113379067A [P ] . 2021-09-10 .

侯铭楷‍‍ . 面向数据和系统异构的高效联邦学习算法研究 [D ] . 长春 : 吉林大学 , 2023 .

HOU M K . Research on efficient federated learning algorithm for heterogeneous data and systems [D ] . Changchun : Jilin University , 2023 .

LIANG P P , LIU T , LIU Z Y , et al . Think locally, act globally: federated learning with local and global representations [J ] . arXiv preprint , 2020 , arXiv: 2001.01523 .

ARIVAZHAGAN M G , AGGARWAL V , SINGH A K , et al . Federated learning with personalization layers [J ] . arXiv preprint , 2019 , arXiv: 1912.00818 .

WU Q , WANG W H , FAN P Y , et al . Cooperative edge caching based on elastic federated and multi-agent deep reinforcement learning in next-generation networks [J ] . IEEE Transactions on Network and Service Management , 2024 , 21 ( 4 ): 4179 - 4196 .

DEBADITYA S , OMER W , ULLAH K W . Federated learning and next generation wireless communications: a survey on bidirectional relationship [J ] . Transactions on Emerging Telecommunications Technologies , 2022 , 33 ( 7 ): e4458 .

XIE Y A , KANG J W , NIYATO D , et al . Securing federated learning: a covert communication-based approach [J ] . IEEE Network , 2023 , 37 ( 1 ): 118 - 124 .

宋祺鹏 , 王继东 , 张丽伟 , 等 . 本地化差分隐私下的电力物联网终端数据隐私保护方法 [J ] . 重庆邮电大学学报(自然科学版) , 2023 , 35 ( 6 ): 1001 - 1010 .

SONG Q P , WANG J D , ZHANG L W , et al . Terminal data privacy protection method for power Internet of things based on localized differential privacy [J ] . Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition) , 2023 , 35 ( 6 ): 1001 - 1010 .

KHAN L U , PANDEY S R , TRAN N H , et al . Federated learning for edge networks: resource optimization and incentive mechanism [J ] . IEEE Communications Magazine , 2020 , 58 ( 10 ): 88 - 93 .

王鑫 , 李美庆 , 王黎明 , 等 . 一种基于合同理论的可激励联邦学习模型 [J ] . 电子与信息学报 , 2023 , 45 ( 3 ): 874 - 883 .

WANG X , LI M Q , WANG L M , et al . An incentivized federated learning model based on contract theory [J ] . Journal of Electronics & Information Technology , 2023 , 45 ( 3 ): 874 - 883 .

LIU W , ZHANG Y H , HAN G , et al . Secure and efficient smart healthcare system based on federated learning [J ] . International Journal of Intelligent Systems , 2023 , 2023 : 8017489 .

刘洋‍ . 工业物联网环境下支持数据共享的联邦学习方法研究 [D ] . 南宁：广西大学， 2024 .

LIU Y . Research on federated learning method supporting data sharing in industrial Internet of things environment [D ] . Nanning : Guangxi University , 2024 .

LUO T , PAN M , THOLONIAT P , et al . Privacy budget scheduling [C ] // The 15th USENIX Symposium on Operating Systems Design and Implementation . Virtual Event , 2021 : 55 - 74 .

LIU Y , WANG Z B , ZHU Y F , et al . DPBalance: efficient and fair privacy budget scheduling for federated learning as a service [C ] // Proceedings of the IEEE INFOCOM 2024-IEEE Conference on Computer Communications . Piscataway : IEEE Press , 2024 : 21 - 30 .

THOLONIAT P , KOSTOPOULOU K , MCNEELY P , et al . Cookie monster: efficient on-device budgeting for differentially-private ad-measurement systems [J ] . arXiv Preprint , 2024 , arXiv: 2405. 16719 .

PELEG S , PORAT B . The Cramer-Rao lower bound for signals with constant amplitude and polynomial phase [J ] . IEEE Transactions on Signal Processing , 1991 , 39 ( 3 ): 749 - 752 .

李洪涛 , 任晓宇 , 王洁 , 等 . 基于差分隐私的连续位置隐私保护机制 [J ] . 通信学报 , 2021 , 42 ( 8 ): 164 - 175 .

LI H T , REN X Y , WANG J , et al . Continuous location privacy protection mechanism based on differential privacy [J ] . Journal on Communications , 2021 , 42 ( 8 ): 164 - 175 .

周赞 , 张笑燕 , 杨树杰 , 等 . 面向联邦算力网络的隐私计算自适激励机制 [J ] . 计算机学报 , 2023 , 46 ( 12 ): 2705 - 2725 .

ZHOU Z , ZHANG X Y , YANG S J , et al . Adaptive incentive mechanism for privacy computing in federated compute first networks [J ] . Chinese Journal of Computers , 2023 , 46 ( 12 ): 2705 - 2725 .

陈谦 , 柴政 , 王子龙 , 等 . 基于生成对抗网络的联邦学习中投毒攻击检测方案 [J ] . 计算机应用 , 2023 , 43 ( 12 ): 3790 - 3798 .

CHEN Q , CHAI Z , WANG Z L , et al . Poisoning attack detection scheme based on generative adversarial network for federated learning [J ] . Journal of Computer Applications , 2023 , 43 ( 12 ): 3790 - 3798 .

LI X , HUANG K , YANG W , et al . On the convergence of FedAvg on Non-IID data [C ] // Proceedings of 8th International Conference on Learning Representations . Washington DC : ICLR , 2020 .

邓志颖 , 潘建辉 . 巴拿赫不动点定理及其应用 [J ] . 高等数学研究 , 2013 , 16 ( 4 ): 78 - 80, 92 .

DENG Z Y , PAN J H . Banach fixed point theorem and its applications [J ] . Studies in College Mathematics , 2013 , 16 ( 4 ): 78 - 80, 92 .

谷峰 . 两个有限族一致L-Lipschitz映象的平行迭代算法的强收敛定理 [J ] . 数学学报 , 2010 , 53 ( 6 ): 1209 - 1216 .

GU F . Strong convergence theorems of a parallel iterative algorithm for two finite families of uniformly L-Lipschitzian mappings [J ] . Acta Mathematica Sinica , 2010 , 53 ( 6 ): 1209 - 1216 .

张少波‍ ，张激勇‍ ，朱更明‍ ，等．基于Bregman散度和差分隐私的个性化联邦学习方法 [J ] ．软件学报 , 2024 , 35 ( 11 ): 5249 - 5262 .

ZHANG S B , ZHANG J Y , ZHU G M , et al . Personalized federated learning method based on Bregman divergence and differential privacy [J ] . Journal of Software , 2024 , 35 ( 11 ): 5249 - 5262 .

LI T , SAHU A K , ZAHEER M , et al . Federated optimization in heterogeneous networks [C ] // Proceedings of the Third Conference on Machine Learning and Systems . TX, USA : MLSys , 2020 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . New York : ACM , 2016 : 308 - 318 .

Views

365

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Differential privacy budget optimization based on deep learning in IoT

Related Author

Dan LUO

Ruzhi XU

Zhitao GUAN

Related Institution

No data

⁰