Learning the Optimal Partition for Collaborative DNN Training with Privacy Requirements

Letian Zhang, Jie Xu

Research output: Contribution to journalArticlepeer-review

Abstract

With the growth of intelligent Internet-of-Things (IoT) applications and services, deep neural network (DNN) has become the core method to power and enable increased functionality in many smart IoT devices. However, DNN training is difficult to carry out on end devices because it requires a great deal of computational power. The conventional approach to DNN training is generally implemented on a powerful computation server; nevertheless, this approach violates privacy because it exposes the training data to curious service providers. In this paper, we consider a collaborative DNN training system between a resource-constrained end device and a powerful edge server, aiming at partitioning a DNN into a front-end part running on the end device and a back-end part running on the edge server to accelerate the training process while preserving the privacy of the training data. With the key challenge being how to locate the optimal partition point to minimize the end-to-end training delay, we propose an online learning module, called Learn-to-Split (L2S), to adaptively learn the optimal partition point on-the-fly. This approach is unlike existing efforts on DNN partitioning that relies heavily on a dedicated offline profiling stage. In particular, we design a new contextual bandit learning algorithm called LinUCB-E as the basis of L2S, which has provable theoretical learning performance and is ultra-lightweight for easy real-world implementation. We implement a prototype system consisting of an end device and an edge server, and experimental results demonstrate that L2S can significantly outperform state-of-the-art benchmarks in terms of reducing the end-to-end training delay and preserving privacy.

Original languageEnglish (US)
JournalIEEE Internet of Things Journal
DOIs
StateAccepted/In press - 2021

Keywords

  • Collaboration
  • Data privacy
  • deep learning
  • Deep learning
  • Delays
  • edge computing
  • Edge intelligence
  • Internet of Things
  • online learning.
  • Servers
  • Training

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Learning the Optimal Partition for Collaborative DNN Training with Privacy Requirements'. Together they form a unique fingerprint.

Cite this