Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing

Jie Xu, Lixing Chen, Shaolei Ren

Research output: Contribution to journalArticlepeer-review

203 Scopus citations


Mobile edge computing (also known as fog computing) has recently emerged to enable in-situ processing of delay-sensitive applications at the edge of mobile networks. Providing grid power supply in support of mobile edge computing, however, is costly and even infeasible (in certain rugged or under-developed areas), thus mandating on-site renewable energy as a major or even sole power supply in increasingly many scenarios. Nonetheless, the high intermittency and unpredictability of renewable energy make it very challenging to deliver a high quality of service to users in energy harvesting mobile edge computing systems. In this paper, we address the challenge of incorporating renewables into mobile edge computing and propose an efficient reinforcement learning-based resource management algorithm, which learns on-the-fly the optimal policy of dynamic workload offloading (to the centralized cloud) and edge server provisioning to minimize the long-term system cost (including both service delay and operational cost). Our online learning algorithm uses a decomposition of the (offline) value iteration and (online) reinforcement learning, thus achieving a significant improvement of learning rate and run-time performance when compared to standard reinforcement learning algorithms such as ${Q}$ -learning. We prove the convergence of the proposed algorithm and analytically show that the learned policy has a simple monotone structure amenable to practical implementation. Our simulation results validate the efficacy of our algorithm, which significantly improves the edge computing performance compared to fixed or myopic optimization schemes and conventional reinforcement learning algorithms.

Original languageEnglish (US)
Article number7973020
Pages (from-to)361-373
Number of pages13
JournalIEEE Transactions on Cognitive Communications and Networking
Issue number3
StatePublished - Sep 2017


  • Mobile edge computing
  • energy harvesting
  • online learning

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing'. Together they form a unique fingerprint.

Cite this