Benchmarking deep networks for predicting residue-specific quality of individual protein models in CASP11

Tong Liu, Yiheng Wang, Jesse Eickholt, Zheng Wang

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Quality assessment of a protein model is to predict the absolute or relative quality of a protein model using computational methods before the native structure is available. Single-model methods only need one model as input and can predict the absolute residue-specific quality of an individual model. Here, we have developed four novel single-model methods (Wang-deep-1, Wang-deep-2, Wang-deep-3, and Wang-SVM) based on stacked denoising autoencoders (SdAs) and support vector machines (SVMs). We evaluated these four methods along with six other methods participating in CASP11 at the global and local levels using Pearson € s correlation coefficients and ROC analysis. As for residue-specific quality assessment, our four methods achieved better performance than most of the six other CASP11 methods in distinguishing the reliably modeled residues from the unreliable measured by ROC analysis; and our SdA-based method Wang-deep-1 has achieved the highest accuracy, 0.77, compared to SVM-based methods and our ensemble of an SVM and SdAs. However, we found that Wang-deep-2 and Wang-deep-3, both based on an ensemble of multiple SdAs and an SVM, performed slightly better than Wang-deep-1 in terms of ROC analysis, indicating that integrating an SVM with deep networks works well in terms of certain measurements.

Original languageEnglish (US)
Article number19301
JournalScientific Reports
Volume6
DOIs
StatePublished - Jan 14 2016
Externally publishedYes

Fingerprint

Benchmarking
Proteins
ROC Curve
Support Vector Machine

ASJC Scopus subject areas

  • General

Cite this

Benchmarking deep networks for predicting residue-specific quality of individual protein models in CASP11. / Liu, Tong; Wang, Yiheng; Eickholt, Jesse; Wang, Zheng.

In: Scientific Reports, Vol. 6, 19301, 14.01.2016.

Research output: Contribution to journalArticle

@article{89ad3643a5ca4e1c8094fdffaed30b30,
title = "Benchmarking deep networks for predicting residue-specific quality of individual protein models in CASP11",
abstract = "Quality assessment of a protein model is to predict the absolute or relative quality of a protein model using computational methods before the native structure is available. Single-model methods only need one model as input and can predict the absolute residue-specific quality of an individual model. Here, we have developed four novel single-model methods (Wang-deep-1, Wang-deep-2, Wang-deep-3, and Wang-SVM) based on stacked denoising autoencoders (SdAs) and support vector machines (SVMs). We evaluated these four methods along with six other methods participating in CASP11 at the global and local levels using Pearson € s correlation coefficients and ROC analysis. As for residue-specific quality assessment, our four methods achieved better performance than most of the six other CASP11 methods in distinguishing the reliably modeled residues from the unreliable measured by ROC analysis; and our SdA-based method Wang-deep-1 has achieved the highest accuracy, 0.77, compared to SVM-based methods and our ensemble of an SVM and SdAs. However, we found that Wang-deep-2 and Wang-deep-3, both based on an ensemble of multiple SdAs and an SVM, performed slightly better than Wang-deep-1 in terms of ROC analysis, indicating that integrating an SVM with deep networks works well in terms of certain measurements.",
author = "Tong Liu and Yiheng Wang and Jesse Eickholt and Zheng Wang",
year = "2016",
month = "1",
day = "14",
doi = "10.1038/srep19301",
language = "English (US)",
volume = "6",
journal = "Scientific Reports",
issn = "2045-2322",
publisher = "Nature Publishing Group",

}

TY - JOUR

T1 - Benchmarking deep networks for predicting residue-specific quality of individual protein models in CASP11

AU - Liu, Tong

AU - Wang, Yiheng

AU - Eickholt, Jesse

AU - Wang, Zheng

PY - 2016/1/14

Y1 - 2016/1/14

N2 - Quality assessment of a protein model is to predict the absolute or relative quality of a protein model using computational methods before the native structure is available. Single-model methods only need one model as input and can predict the absolute residue-specific quality of an individual model. Here, we have developed four novel single-model methods (Wang-deep-1, Wang-deep-2, Wang-deep-3, and Wang-SVM) based on stacked denoising autoencoders (SdAs) and support vector machines (SVMs). We evaluated these four methods along with six other methods participating in CASP11 at the global and local levels using Pearson € s correlation coefficients and ROC analysis. As for residue-specific quality assessment, our four methods achieved better performance than most of the six other CASP11 methods in distinguishing the reliably modeled residues from the unreliable measured by ROC analysis; and our SdA-based method Wang-deep-1 has achieved the highest accuracy, 0.77, compared to SVM-based methods and our ensemble of an SVM and SdAs. However, we found that Wang-deep-2 and Wang-deep-3, both based on an ensemble of multiple SdAs and an SVM, performed slightly better than Wang-deep-1 in terms of ROC analysis, indicating that integrating an SVM with deep networks works well in terms of certain measurements.

AB - Quality assessment of a protein model is to predict the absolute or relative quality of a protein model using computational methods before the native structure is available. Single-model methods only need one model as input and can predict the absolute residue-specific quality of an individual model. Here, we have developed four novel single-model methods (Wang-deep-1, Wang-deep-2, Wang-deep-3, and Wang-SVM) based on stacked denoising autoencoders (SdAs) and support vector machines (SVMs). We evaluated these four methods along with six other methods participating in CASP11 at the global and local levels using Pearson € s correlation coefficients and ROC analysis. As for residue-specific quality assessment, our four methods achieved better performance than most of the six other CASP11 methods in distinguishing the reliably modeled residues from the unreliable measured by ROC analysis; and our SdA-based method Wang-deep-1 has achieved the highest accuracy, 0.77, compared to SVM-based methods and our ensemble of an SVM and SdAs. However, we found that Wang-deep-2 and Wang-deep-3, both based on an ensemble of multiple SdAs and an SVM, performed slightly better than Wang-deep-1 in terms of ROC analysis, indicating that integrating an SVM with deep networks works well in terms of certain measurements.

UR - http://www.scopus.com/inward/record.url?scp=84955127996&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84955127996&partnerID=8YFLogxK

U2 - 10.1038/srep19301

DO - 10.1038/srep19301

M3 - Article

C2 - 26763289

AN - SCOPUS:84955127996

VL - 6

JO - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

M1 - 19301

ER -