物联网环境中基于深度学习的差分隐私预算优化方法

罗丹; 徐茹枝; 关志涛

doi:10.11959/j.issn.2096-3750.2022.00264

您当前的位置：

首页 >

文章列表页 >

物联网环境中基于深度学习的差分隐私预算优化方法

理论与技术 | 更新时间：2024-06-05

- 物联网环境中基于深度学习的差分隐私预算优化方法
- Differential privacy budget optimization based on deep learning in IoT
- 物联网学报 2022年6卷第2期页码：65-76
- 作者机构：
- 作者简介：
  
  [ "罗丹（1997− ），女，华北电力大学硕士生，主要研究方向为隐私计算、信息安全" ]
  [ "徐茹枝（1966− ），女，博士，华北电力大学教授，主要研究方向为电力信息安全" ]
  [ "关志涛（1979− ），男，博士，华北电力大学副教授、博士生导师，主要研究方向为物联网安全、区块链技术、人工智能安全" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61972148)
- DOI：10.11959/j.issn.2096-3750.2022.00264
  中图分类号： TP391
- 纸质出版日期：2022-06-30，
  
  网络出版日期：2022-06，
- 稿件说明：
移动端阅览
罗丹, 徐茹枝, 关志涛. 物联网环境中基于深度学习的差分隐私预算优化方法[J]. 物联网学报, 2022,6(2):65-76.

DAN LUO, RUZHI XU, ZHITAO GUAN. Differential privacy budget optimization based on deep learning in IoT. [J]. Chinese journal on internet of things, 2022, 6(2): 65-76.
罗丹, 徐茹枝, 关志涛. 物联网环境中基于深度学习的差分隐私预算优化方法[J]. 物联网学报, 2022,6(2):65-76. DOI： 10.11959/j.issn.2096-3750.2022.00264.

DAN LUO, RUZHI XU, ZHITAO GUAN. Differential privacy budget optimization based on deep learning in IoT. [J]. Chinese journal on internet of things, 2022, 6(2): 65-76. DOI： 10.11959/j.issn.2096-3750.2022.00264.

摘要

为有效处理物联网大规模应用所带来的海量数据，深度学习在物联网环境中得到广泛应用。然而，深度模型在训练过程中，存在推理攻击、模型逆向攻击等安全威胁，这会导致输入模型中的原始数据泄露。应用差分隐私对深度模型训练过程的参数进行保护，是解决该问题的有效方式。基于此提出一种物联网环境中基于深度学习的差分隐私预算优化方法，根据参数迭代变化规律，自适应地分配不同预算；为避免噪声过大的问题，引入正则化项对扰动项进行约束，既防止神经网络过拟合，又有助于学习模型的显著特征。实验表明，所提方法可有效增强模型的泛化能力；随着模型迭代次数增加，加噪后训练得到的模型，与使用原始数据训练得到的模型，二者精度差值低于0.5%。因此，所提方法既可实现用户隐私保护，同时有效保证模型可用性，实现了隐私性和可用性的平衡。

Abstract

In order to effectively process the massive data brought by the large-scale application of the internet of things (IoT)

deep learning is widely used in IoT environment.However

in the training process of deep learning

there are security threats such as reasoning attacks and model reverse attacks

which can lead to the leakage of the original data input to the model.Applying differential privacy to protect the training process parameters of the deep model is an effective way to solve this problem.A differential privacy budget optimization method was proposed based on deep learning in IoT

which adaptively allocates different budgets according to the iterative change of parameters.In order to avoid the excessive noise

a regularization term was introduced to constrain the disturbance term.Preventing the neural network from over fitting also helps to learn the salient features of the model.Experiments show that this method can effectively enhance the generalization ability of the model.As the number of iterations increases

the accuracy of the model trained after adding noise is almost the same as that obtained by training using the original data

which not only achieves privacy protection

but also guarantees the availability

which means balance the privacy and availability.

关键词

物联网差分隐私正则化深度学习隐私预算

Keywords

IoTdifferential privacyregularizationdeep learningprivacy budget

references

杨毅宇, 周威, 赵尚儒 ,等. 物联网安全研究综述：威胁、检测与防御[J]. 通信学报, 2021,42(8): 188-205.

YANG Y Y, ZHOU W, ZHAO S R ,et al. Survey of IoT security research:threats,detection and defense[J]. Journal on Communications, 2021,42(8): 188-205.

吕建新, 郑伟, 马林 ,等. 基于词向量语义扩展的网络文本特征选择方法研究[J]. 情报科学, 2019,37(12): 47-51.

LV J X, ZHENG W, MA L ,et al. Feature selection method of the network text based on semantic extension with word vector[J]. Information Science, 2019,37(12): 47-51.

孟仕林, 赵蕴龙, 关东海 ,等. 融合情感与语义信息的情感分析方法[J]. 计算机应用, 2019,39(7): 1931-1935.

MENG S L, ZHAO Y L, GUAN D H ,et al. Sentiment analysis method combining sentiment and semantic information[J]. Journal of Computer Applications, 2019,39(7): 1931-1935.

LI T, LI J, CHEN X F ,et al. NPMML:a framework for non-interactive privacy-preserving multi-party machine learning[J]. IEEE Transactions on Dependable and Secure Computing, 2021,18(6): 2969-2982.

ZHANG X L, FU A M, WANG H Q ,et al. A privacy-preserving and verifiable federated learning scheme[C]// Proceedings of ICC 2020 2020 IEEE International Conference on Communications. Piscataway:IEEE Press, 2020: 1-6.

SUH J, TANAKA T . Encrypted value iteration and temporal difference learning over leveled homomorphic encryption[C]// Proceedings of 2021 American Control Conference (ACC). Piscataway:IEEE Press, 2021.

WANG Y C, LIANG X L, HEI X H ,et al. Deep learning data privacy protection based on homomorphic encryption in AIoT[J]. Mobile Information Systems,2021, 2021:5510857.

YE H, LIU J Q, WANG W ,et al. Secure and efficient outsourcing differential privacy data release scheme in Cyber-physical system[J]. Future Generation Computer Systems, 2020,108: 1314-1323.

BU Z Q, WANG H, LONG Q ,et al. On the convergence of deep learning with differential privacy[EB]. 2021.

BU Z Q, GOPI S, KULKARNI J ,et al. Fast and memory efficient differentially private-SGD via JL projections[EB]. 2021.

CHEN X, WU S Z, HONG M . Understanding gradient clipping in private SGD:A geometric perspective[J]. Advances in Neural Information Processing Systems, 2020,33: 13773-13782.

KOSKELA A, JALKO J, HONKELA A . Computing tight differential privacy guarantees using fft[C]// International Conference on Artificial Intelligence and Statistics. Online:PMLR, 2020: 2560-2569.

GHAZI B, GOLOWICH N, KUMAR R ,et al. On deep learning with label differential privacy[EB]. 2021.

YUAN S, SHEN M, MIRONOV I ,et al. Practical,label private deep learning training based on secure multiparty computation and differential privacy[EB]. 2021.

ABADI M, CHU A, GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2016: 308-318.

DWORK C, . Differential privacy[C]// Proceedings of 33th International Colloquium on Automata,Languages and Programming. Berlin:Springer, 2006: 1-12.

DWORK C, . Differential privacy:a survey of results[C]// Proceedings of the 5th International Conference on Theory and Applications of Models of Computation. Berlin:Springer-Verlag, 2008: 1-19.

DWORK C, MCSHERRY F, NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// Theory of Cryptography. Berlin,Heidelberg:Springer, 2006: 265-284.

ZINKEVICH M, WEIMER M, LI L ,et al. Parallelized stochastic gradient descent[C]// Proceedings of Advances in neural information processing systems. Vancouver,Canada:NIPS, 2010: 2595-2603.

CHANG D Q, LIN M, ZHANG C S . On the generalization ability of online gradient descent algorithm under the quadratic growth condition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(10): 5008-5019.

BUKOVSKY I, HOMMA N . An approach to stable gradient-descent adaptation of higher order neural units[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017,28(9): 2022-2034.

LECUN Y, BOTTOU L, BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.

汪小寒, 韩慧慧, 张泽培 ,等. 树索引数据差分隐私预算分配方法[J]. 计算机应用, 2018,38(7): 1960-1966.

WANG X H, HAN H H, ZHANG Z P ,et al. Differential privacy budget allocation method for data of tree index[J]. Journal of Computer Applications, 2018,38(7): 1960-1966.

GONG M G, FENG J L, XIE Y . Privacy-enhanced multi-party deep learning[J]. Neural Networks, 2020,121: 484-496.

王璇 . 差分隐私保护中隐私预算的优化与应用[D]. 南京:南京邮电大学, 2019.

WANG X . Optimization and application of privacy budget in differential privacy protection[D]. Nanjing:Nanjing University of Posts and Telecommunications, 2019.

浏览量

801

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

认证加密算法SM4-GCM的低成本硬件架构设计与实现

超大规模MIMO-OTFS系统上行辅助的下行信道估计方法

基于最优运输理论的蜂窝网边缘卸载时延优化研究

基于可穿戴与可植入技术的人体健康物联网研究进展

基于AE和Transformer的运动想象脑电信号分类研究