An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework

Ramin Moghaddass, Shuangwen Sheng

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. The numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

Original languageEnglish (US)
Pages (from-to)561-582
Number of pages22
JournalApplied Energy
Volume240
DOIs
StatePublished - Apr 15 2019

Fingerprint

Dynamical systems
anomaly
Sensors
sensor
Wind turbines
Markov processes
Large scale systems
Availability
catastrophic event
detection
wind turbine
Markov chain
trade-off
Costs
Experiments
cost
modeling
experiment

Keywords

  • Anomaly detection
  • Bayesian Modeling
  • Dynamic systems
  • Sensor-intensive energy systems
  • Wind turbine

ASJC Scopus subject areas

  • Building and Construction
  • Energy(all)
  • Mechanical Engineering
  • Management, Monitoring, Policy and Law

Cite this

An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework. / Moghaddass, Ramin; Sheng, Shuangwen.

In: Applied Energy, Vol. 240, 15.04.2019, p. 561-582.

Research output: Contribution to journalArticle

@article{931481e613db4f99ac762b641e901d8a,
title = "An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework",
abstract = "Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. The numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.",
keywords = "Anomaly detection, Bayesian Modeling, Dynamic systems, Sensor-intensive energy systems, Wind turbine",
author = "Ramin Moghaddass and Shuangwen Sheng",
year = "2019",
month = "4",
day = "15",
doi = "10.1016/j.apenergy.2019.02.025",
language = "English (US)",
volume = "240",
pages = "561--582",
journal = "Applied Energy",
issn = "0306-2619",
publisher = "Elsevier BV",

}

TY - JOUR

T1 - An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework

AU - Moghaddass, Ramin

AU - Sheng, Shuangwen

PY - 2019/4/15

Y1 - 2019/4/15

N2 - Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. The numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

AB - Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. The numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

KW - Anomaly detection

KW - Bayesian Modeling

KW - Dynamic systems

KW - Sensor-intensive energy systems

KW - Wind turbine

UR - http://www.scopus.com/inward/record.url?scp=85061751773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061751773&partnerID=8YFLogxK

U2 - 10.1016/j.apenergy.2019.02.025

DO - 10.1016/j.apenergy.2019.02.025

M3 - Article

AN - SCOPUS:85061751773

VL - 240

SP - 561

EP - 582

JO - Applied Energy

JF - Applied Energy

SN - 0306-2619

ER -