Privacy-preserving Federated Learning based on Differential Privacy

Weng, Shangyin (2025) Privacy-preserving Federated Learning based on Differential Privacy. PhD thesis, University of Glasgow.

Abstract

Although FL is claimed to guarantee privacy protection, semi-honest servers and local clients can still reconstruct sensitive information from the gradients. Therefore, to enhance privacy protection, differential privacy (DP) is widely adopted in FL by randomizing the gradients before transmitting them to other parties. Nevertheless, randomizing gradients inevitably degrades FL performance in terms of lower accuracy and higher communication overhead. To solve this problem, this thesis focuses on exploring methods to enhance privacy protection and improve the overall utility of DP-based FL (DPFL) frameworks.
This thesis begins with the research question on improving the degraded accuracy performance and reducing communication overhead for centralized DPFL while maintaining a strong privacy protection guarantee. Two different frameworks are proposed to tackle this question. The first framework combines local DP (LDP) and central DP (CDP) to prevent both central servers and clients from recovering private information by adding noise to the local gradients before uploading and to the aggregated gradients on the server side before broadcasting, respectively. To improve the overall utility of the proposed DPFL, a novel sparse mechanism is adopted on the local gradients before adding noise and a global momentum gradient descent is introduced on the server side and the client side. For the second framework, a novel LDP-based FL framework with two performance improvement modifications is proposed. One modification is to calculate the difference between noisy and original gradients, and add the difference to the objective function to be minimized. The other modification is to calculate the expectation of the loss created by noise, which is also incorporated into the objective function to be optimized. For both modifications, the privacy protection levels are the same as those for plain DPFL since no modifications to DP settings have been made. This thesis presents the necessary convergence analysis for the proposed framework under convex and non-convex settings. A series of simulations is conducted to validate both frameworks’ effectiveness in terms of higher accuracy and lower communication costs. Specifically, the first framework can outperform other DPFL frameworks while saving 90% of communication costs since sparse mechanism can improve the performance under DP noise.
The second framework can save up to 40% of communication and training rounds while achieving better accuracy than plain LDP-based FL. The second research topic in this thesis is to investigate the impact of DP on privacy protection across various DP noise and clipping settings. To address this, an evaluation method for privacy leakage in the FL is proposed by utilizing reconstruction attacks to analyze the difference between the original images and reconstructed ones. Furthermore, this thesis studies the accumulative privacy loss under two different reconstruction attack settings and demonstrates that anonymizing local clients can decrease the probability of privacy leakage. Next, the effects of different clipping methods on privacy protection are analyzed. Simulations are conducted to characterize the trade-off between privacy protection and learning accuracy and demonstrate that there is an optimal DP setting to provide the desired privacy guarantee. The summarized theoretical findings and simulation results in this work can be utilized to guide heterogeneous DP settings for DPFL.
The third research topic of this thesis explores privacy enhancement and accuracy improvement in decentralized DPFL. To address these challenges, a novel anonymous decentralized DPFL framework is proposed. Specifically, two decentralized DPFL methods based on the gossip and fake-centralized manners are first introduced, where the training clients selection rate (TCSR) in each round for both methods and the model exchange rate (MER) in the gossip method are researched. To enhance privacy protection, an anonymous mechanism, is proposed where all clients are unaware of whom they are communicating with and cannot determine whether they are communicating with the same client across several rounds. Next, the required noise scale is derived in terms of the DP settings, TCSR and MER. Subsequently, the convergence bound for the proposed framework is provided, which suggests that an optimal number of clients for is needed to achieve the best convergence performance. Finally, a series of simulations is conducted to evaluate the performance. The simulation results show that the proposed framework only has a small degradation in accuracy compared to the non-private FL and validate our theoretical results.
In conclusion, this thesis provides insight into increasing overall utility and enhancing privacy protection in DPFL. The convergence and privacy analysis of the proposed frameworks provides a basis for future research focusing on further improving the performance of DPFL. Moreover, the proposed privacy leakage evaluation method can provide a more intuitive understanding of privacy loss, which can be utilized to improve accuracy and promote privacy audits for regulatory compliance and user assurance.

• Item Type: Thesis (PhD)
• Qualification Level: Doctoral
• Subjects: T Technology > TA Engineering (General). Civil engineering (General)
• Colleges/Schools: College of Science and Engineering > School of Engineering
• Supervisor's Name: Zhang, Professor Lei and Imran, Professor Muhammad
• Date of Award: 2025
• Depositing User: Theses Team
• Unique ID: glathesis:2025-84841
• Copyright: Copyright of this thesis is held by the author.
• Date Deposited: 29 Jan 2025 11:31
• Last Modified: 29 Jan 2025 12:17
• Thesis DOI: 10.5525/gla.thesis.84841
• URI: https://theses.gla.ac.uk/id/eprint/84841

Related URLs

• Conference proceeding
• Article DOI
• Article DOI