Differential privacy budget optimization based on deep learning in IoT

Dan LUO; Ruzhi XU; Zhitao GUAN

doi:10.11959/j.issn.2096-3750.2022.00264

您当前的位置：

首页 >

文章列表页 >

Differential privacy budget optimization based on deep learning in IoT

Theory and Technology | 更新时间：2024-06-05

- Differential privacy budget optimization based on deep learning in IoT
- Chinese Journal on Internet of Things Vol. 6, Issue 2, Pages: 65-76(2022)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2022.00264
  CLC： TP391
- Published：30 June 2022，
  
  Published Online：2022-06，
- 稿件说明：
移动端阅览
DAN LUO, RUZHI XU, ZHITAO GUAN. Differential privacy budget optimization based on deep learning in IoT. [J]. Chinese journal on internet of things, 2022, 6(2): 65-76.
DOI：

DAN LUO, RUZHI XU, ZHITAO GUAN. Differential privacy budget optimization based on deep learning in IoT. [J]. Chinese journal on internet of things, 2022, 6(2): 65-76. DOI： 10.11959/j.issn.2096-3750.2022.00264.

摘要

为有效处理物联网大规模应用所带来的海量数据，深度学习在物联网环境中得到广泛应用。然而，深度模型在训练过程中，存在推理攻击、模型逆向攻击等安全威胁，这会导致输入模型中的原始数据泄露。应用差分隐私对深度模型训练过程的参数进行保护，是解决该问题的有效方式。基于此提出一种物联网环境中基于深度学习的差分隐私预算优化方法，根据参数迭代变化规律，自适应地分配不同预算；为避免噪声过大的问题，引入正则化项对扰动项进行约束，既防止神经网络过拟合，又有助于学习模型的显著特征。实验表明，所提方法可有效增强模型的泛化能力；随着模型迭代次数增加，加噪后训练得到的模型，与使用原始数据训练得到的模型，二者精度差值低于0.5%。因此，所提方法既可实现用户隐私保护，同时有效保证模型可用性，实现了隐私性和可用性的平衡。

Abstract

In order to effectively process the massive data brought by the large-scale application of the internet of things (IoT)

deep learning is widely used in IoT environment.However

in the training process of deep learning

there are security threats such as reasoning attacks and model reverse attacks

which can lead to the leakage of the original data input to the model.Applying differential privacy to protect the training process parameters of the deep model is an effective way to solve this problem.A differential privacy budget optimization method was proposed based on deep learning in IoT

which adaptively allocates different budgets according to the iterative change of parameters.In order to avoid the excessive noise

a regularization term was introduced to constrain the disturbance term.Preventing the neural network from over fitting also helps to learn the salient features of the model.Experiments show that this method can effectively enhance the generalization ability of the model.As the number of iterations increases

the accuracy of the model trained after adding noise is almost the same as that obtained by training using the original data

which not only achieves privacy protection

but also guarantees the availability

which means balance the privacy and availability.

关键词

物联网差分隐私正则化深度学习隐私预算

Keywords

IoTdifferential privacyregularizationdeep learningprivacy budget

references

杨毅宇, 周威, 赵尚儒 ,等. 物联网安全研究综述：威胁、检测与防御[J]. 通信学报, 2021,42(8): 188-205.

YANG Y Y, ZHOU W, ZHAO S R ,et al. Survey of IoT security research:threats,detection and defense[J]. Journal on Communications, 2021,42(8): 188-205.

吕建新, 郑伟, 马林 ,等. 基于词向量语义扩展的网络文本特征选择方法研究[J]. 情报科学, 2019,37(12): 47-51.

LV J X, ZHENG W, MA L ,et al. Feature selection method of the network text based on semantic extension with word vector[J]. Information Science, 2019,37(12): 47-51.

孟仕林, 赵蕴龙, 关东海 ,等. 融合情感与语义信息的情感分析方法[J]. 计算机应用, 2019,39(7): 1931-1935.

MENG S L, ZHAO Y L, GUAN D H ,et al. Sentiment analysis method combining sentiment and semantic information[J]. Journal of Computer Applications, 2019,39(7): 1931-1935.

LI T, LI J, CHEN X F ,et al. NPMML:a framework for non-interactive privacy-preserving multi-party machine learning[J]. IEEE Transactions on Dependable and Secure Computing, 2021,18(6): 2969-2982.

ZHANG X L, FU A M, WANG H Q ,et al. A privacy-preserving and verifiable federated learning scheme[C]// Proceedings of ICC 2020 2020 IEEE International Conference on Communications. Piscataway:IEEE Press, 2020: 1-6.

SUH J, TANAKA T . Encrypted value iteration and temporal difference learning over leveled homomorphic encryption[C]// Proceedings of 2021 American Control Conference (ACC). Piscataway:IEEE Press, 2021.

WANG Y C, LIANG X L, HEI X H ,et al. Deep learning data privacy protection based on homomorphic encryption in AIoT[J]. Mobile Information Systems,2021, 2021:5510857.

YE H, LIU J Q, WANG W ,et al. Secure and efficient outsourcing differential privacy data release scheme in Cyber-physical system[J]. Future Generation Computer Systems, 2020,108: 1314-1323.

BU Z Q, WANG H, LONG Q ,et al. On the convergence of deep learning with differential privacy[EB]. 2021.

BU Z Q, GOPI S, KULKARNI J ,et al. Fast and memory efficient differentially private-SGD via JL projections[EB]. 2021.

CHEN X, WU S Z, HONG M . Understanding gradient clipping in private SGD:A geometric perspective[J]. Advances in Neural Information Processing Systems, 2020,33: 13773-13782.

KOSKELA A, JALKO J, HONKELA A . Computing tight differential privacy guarantees using fft[C]// International Conference on Artificial Intelligence and Statistics. Online:PMLR, 2020: 2560-2569.

GHAZI B, GOLOWICH N, KUMAR R ,et al. On deep learning with label differential privacy[EB]. 2021.

YUAN S, SHEN M, MIRONOV I ,et al. Practical,label private deep learning training based on secure multiparty computation and differential privacy[EB]. 2021.

ABADI M, CHU A, GOODFELLOW I ,et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York:ACM Press, 2016: 308-318.

DWORK C, . Differential privacy[C]// Proceedings of 33th International Colloquium on Automata,Languages and Programming. Berlin:Springer, 2006: 1-12.

DWORK C, . Differential privacy:a survey of results[C]// Proceedings of the 5th International Conference on Theory and Applications of Models of Computation. Berlin:Springer-Verlag, 2008: 1-19.

DWORK C, MCSHERRY F, NISSIM K ,et al. Calibrating noise to sensitivity in private data analysis[C]// Theory of Cryptography. Berlin,Heidelberg:Springer, 2006: 265-284.

ZINKEVICH M, WEIMER M, LI L ,et al. Parallelized stochastic gradient descent[C]// Proceedings of Advances in neural information processing systems. Vancouver,Canada:NIPS, 2010: 2595-2603.

CHANG D Q, LIN M, ZHANG C S . On the generalization ability of online gradient descent algorithm under the quadratic growth condition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(10): 5008-5019.

BUKOVSKY I, HOMMA N . An approach to stable gradient-descent adaptation of higher order neural units[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017,28(9): 2022-2034.

LECUN Y, BOTTOU L, BENGIO Y ,et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998,86(11): 2278-2324.

汪小寒, 韩慧慧, 张泽培 ,等. 树索引数据差分隐私预算分配方法[J]. 计算机应用, 2018,38(7): 1960-1966.

WANG X H, HAN H H, ZHANG Z P ,et al. Differential privacy budget allocation method for data of tree index[J]. Journal of Computer Applications, 2018,38(7): 1960-1966.

GONG M G, FENG J L, XIE Y . Privacy-enhanced multi-party deep learning[J]. Neural Networks, 2020,121: 484-496.

王璇 . 差分隐私保护中隐私预算的优化与应用[D]. 南京:南京邮电大学, 2019.

WANG X . Optimization and application of privacy budget in differential privacy protection[D]. Nanjing:Nanjing University of Posts and Telecommunications, 2019.

Views

801

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Design and implementation of low-cost hardware architecture for authentication encryption algorithm SM4-GCM

Uplink assisted downlink channel estimation method of extra-large scale MIMO-OTFS system

Research on edge offloading delay optimization of cellular networks based on optimal transport theory

Research progress of human health IoT based on wearable and implantable techniques

Research on EEG signal classification of motor imagery based on AE and Transformer

Related Author

Rui CHEN

Chunqiang LI

Xumin PU

Kaiyuan DENG

Qianbin CHEN

Xiangyu LYU

Yong XIAO

Yi ZHONG

Related Institution

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications

National Mobile Communications Research Laboratory, Southeast University

Department of Electronic Engineering, School of Internet of Things Engineering, Jiangnan University

Collaborative Innovation Center of Advanced Microstructures, School of Electronic Science and Engineering, Nanjing University

Wuxi Central Rehabilitation Hospital, Affiliated Mental Health Center of Jiangnan University

⁰