Skip to main content
Article
On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks
IEEE International Conference on Communications
  • Minsu Kim, Virginia Polytechnic Institute and State University
  • Walid Saad, Virginia Polytechnic Institute and State University
  • Mohammad Mozaffari, Ericsson Inc.
  • Merouane Debbah, Technology Innovation Institute & Mohamed bin Zayed University of Artificial Intelligence
Document Type
Conference Proceeding
Abstract

Deploying federated learning (FL) over wireless networks with resource-constrained devices requires balancing between accuracy, energy efficiency, and precision. Prior art on FL often requires devices to train deep neural networks (DNNs) using a 32-bit precision level for data representation to improve accuracy. However, such algorithms are impractical for resource-constrained devices since DNNs could require execution of millions of operations. Thus, training DNNs with a high precision level incurs a high energy cost for FL. In this paper, a quantized FL framework, that represents data with a finite level of precision in both local training and uplink transmission, is proposed. Here, the finite level of precision is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. In the considered FL model, each device trains its QNN and transmits a quantized training result to the base station. Energy models for the local training and the transmission with the quantization are rigorously derived. An energy minimization problem is formulated with respect to the level of precision while ensuring convergence. To solve the problem, we first analytically derive the FL convergence rate and use a line search method. Simulation results show that our FL framework can reduce energy consumption by up to 53% compared to a standard FL model. The results also shed light on the tradeoff between precision, energy, and accuracy in FL over wireless networks.

DOI
10.1109/ICC45855.2022.9838362
Publication Date
8-11-2022
Keywords
  • Training,
  • Wireless networks,
  • Simulation,
  • Search methods,
  • Neural networks,
  • Minimization,
  • Energy efficiency
Comments

IR conditions: non-described

Citation Information
M. Kim, W. Saad, M. Mozaffari and M. Debbah, "On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks," ICC 2022 - IEEE International Conference on Communications, Seoul, Korea, Republic of, 2022, pp. 2194-2199, doi: 10.1109/ICC45855.2022.9838362.