A large-scale evaluation of computational protein function prediction

Predrag Radivojac; Wyatt T. Clark; Tal Ronnen Oron; Alexandra M. Schnoes; Tobias Wittkop; Artem Sokolov; Kiley Graim; Christopher Funk; Karin Verspoor; Asa Ben-Hur; Gaurav Pandey; Jeffrey M. Yunes; Ameet S. Talwalkar; Susanna Repo; Michael L. Souza; Damiano Piovesan; Rita Casadio; Zheng Wang; Jianlin Cheng; Hai Fang; Julian Gough; Patrik Koskinen; Petri Törönen; Jussi Nokso-Koivisto; Liisa Holm; Domenico Cozzetto; Daniel W.A. Buchan; Kevin Bryson; David T. Jones; Bhakti Limaye; Harshal Inamdar; Avik Datta; Sunitha K. Manjari; Rajendra Joshi; Meghana Chitale; Daisuke Kihara; Andreas M. Lisewski; Serkan Erdin; Eric Venner; Olivier Lichtarge; Robert Rentzsch; Haixuan Yang; Alfonso E. Romero; Prajwal Bhat; Alberto Paccanaro; Tobias Hamp; Rebecca Kaßner; Stefan Seemayer; Esmeralda Vicedo; Christian Schaefer; Dominik Achten; Florian Auer; Ariane Boehm; Tatjana Braun; Maximilian Hecht; Mark Heron; Peter Hönigschmid; Thomas A. Hopf; Stefanie Kaufmann; Michael Kiening; Denis Krompass; Cedric Landerer; Yannick Mahlich; Manfred Roos; Jari Björne; Tapio Salakoski; Andrew Wong; Hagit Shatkay; Fanny Gatzmann; Ingolf Sommer; Mark N. Wass; Michael J.E. Sternberg; Nives Škunca; Fran Supek; Matko Bošnjak; Panče Panov; Sašo Džeroski; Tomislav Šmuc; Yiannis A.I. Kourmpetis; Aalt D.J. Van Dijk; Cajo J.F. Ter Braak; Yuanpeng Zhou; Qingtian Gong; Xinran Dong; Weidong Tian; Marco Falda; Paolo Fontana; Enrico Lavezzo; Barbara Di Camillo; Stefano Toppo; Liang Lan; Nemanja Djuric; Yuhong Guo; Slobodan Vucetic; Amos Bairoch; Michal Linial; Patricia C. Babbitt; Steven E. Brenner; Christine Orengo; Burkhard Rost; Sean D. Mooney; Iddo Friedberg

doi:10.1038/nmeth.2340

A large-scale evaluation of computational protein function prediction

Predrag Radivojac, Wyatt T. Clark, Tal Ronnen Oron, Alexandra M. Schnoes, Tobias Wittkop, Artem Sokolov, Kiley Graim, Christopher Funk, Karin Verspoor, Asa Ben-Hur, Gaurav Pandey, Jeffrey M. Yunes, Ameet S. Talwalkar, Susanna Repo, Michael L. Souza, Damiano Piovesan, Rita Casadio, Zheng Wang, Jianlin Cheng, Hai FangJulian Gough, Patrik Koskinen, Petri Törönen, Jussi Nokso-Koivisto, Liisa Holm, Domenico Cozzetto, Daniel W.A. Buchan, Kevin Bryson, David T. Jones, Bhakti Limaye, Harshal Inamdar, Avik Datta, Sunitha K. Manjari, Rajendra Joshi, Meghana Chitale, Daisuke Kihara, Andreas M. Lisewski, Serkan Erdin, Eric Venner, Olivier Lichtarge, Robert Rentzsch, Haixuan Yang, Alfonso E. Romero, Prajwal Bhat, Alberto Paccanaro, Tobias Hamp, Rebecca Kaßner, Stefan Seemayer, Esmeralda Vicedo, Christian Schaefer, Dominik Achten, Florian Auer, Ariane Boehm, Tatjana Braun, Maximilian Hecht, Mark Heron, Peter Hönigschmid, Thomas A. Hopf, Stefanie Kaufmann, Michael Kiening, Denis Krompass, Cedric Landerer, Yannick Mahlich, Manfred Roos, Jari Björne, Tapio Salakoski, Andrew Wong, Hagit Shatkay, Fanny Gatzmann, Ingolf Sommer, Mark N. Wass, Michael J.E. Sternberg, Nives Škunca, Fran Supek, Matko Bošnjak, Panče Panov, Sašo Džeroski, Tomislav Šmuc, Yiannis A.I. Kourmpetis, Aalt D.J. Van Dijk, Cajo J.F. Ter Braak, Yuanpeng Zhou, Qingtian Gong, Xinran Dong, Weidong Tian, Marco Falda, Paolo Fontana, Enrico Lavezzo, Barbara Di Camillo, Stefano Toppo, Liang Lan, Nemanja Djuric, Yuhong Guo, Slobodan Vucetic, Amos Bairoch, Michal Linial, Patricia C. Babbitt, Steven E. Brenner, Christine Orengo, Burkhard Rost, Sean D. Mooney, Iddo Friedberg

Computer Science

Research output: Contribution to journal › Article

432 Scopus citations

Abstract

Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the overwhelming majority of protein products can only be annotated computationally. If computational predictions are to be relied upon, it is crucial that the accuracy of these methods be high. Here we report the results from the first large-scale community-based critical assessment of protein function annotation (CAFA) experiment. Fifty-four methods representing the state of the art for protein function prediction were evaluated on a target set of 866 proteins from 11 organisms. Two findings stand out: (i) today's best protein function prediction algorithms substantially outperform widely used first-generation methods, with large gains on all types of targets; and (ii) although the top methods perform well enough to guide experiments, there is considerable need for improvement of currently available tools.

Original language	English (US)
Pages (from-to)	221-227
Number of pages	7
Journal	Nature Methods
Volume	10
Issue number	3
DOIs	https://doi.org/10.1038/nmeth.2340
State	Published - Mar 1 2013

Fingerprint

ASJC Scopus subject areas

Biotechnology
Biochemistry
Molecular Biology
Cell Biology

Cite this

Radivojac, P., Clark, W. T., Oron, T. R., Schnoes, A. M., Wittkop, T., Sokolov, A., Graim, K., Funk, C., Verspoor, K., Ben-Hur, A., Pandey, G., Yunes, J. M., Talwalkar, A. S., Repo, S., Souza, M. L., Piovesan, D., Casadio, R., Wang, Z., Cheng, J., ... Friedberg, I. (2013). A large-scale evaluation of computational protein function prediction. Nature Methods, 10(3), 221-227. https://doi.org/10.1038/nmeth.2340

A large-scale evaluation of computational protein function prediction. / Radivojac, Predrag; Clark, Wyatt T.; Oron, Tal Ronnen; Schnoes, Alexandra M.; Wittkop, Tobias; Sokolov, Artem; Graim, Kiley; Funk, Christopher; Verspoor, Karin; Ben-Hur, Asa; Pandey, Gaurav; Yunes, Jeffrey M.; Talwalkar, Ameet S.; Repo, Susanna; Souza, Michael L.; Piovesan, Damiano; Casadio, Rita; Wang, Zheng; Cheng, Jianlin; Fang, Hai; Gough, Julian; Koskinen, Patrik; Törönen, Petri; Nokso-Koivisto, Jussi; Holm, Liisa; Cozzetto, Domenico; Buchan, Daniel W.A.; Bryson, Kevin; Jones, David T.; Limaye, Bhakti; Inamdar, Harshal; Datta, Avik; Manjari, Sunitha K.; Joshi, Rajendra; Chitale, Meghana; Kihara, Daisuke; Lisewski, Andreas M.; Erdin, Serkan; Venner, Eric; Lichtarge, Olivier; Rentzsch, Robert; Yang, Haixuan; Romero, Alfonso E.; Bhat, Prajwal; Paccanaro, Alberto; Hamp, Tobias; Kaßner, Rebecca; Seemayer, Stefan; Vicedo, Esmeralda; Schaefer, Christian; Achten, Dominik; Auer, Florian; Boehm, Ariane; Braun, Tatjana; Hecht, Maximilian; Heron, Mark; Hönigschmid, Peter; Hopf, Thomas A.; Kaufmann, Stefanie; Kiening, Michael; Krompass, Denis; Landerer, Cedric; Mahlich, Yannick; Roos, Manfred; Björne, Jari; Salakoski, Tapio; Wong, Andrew; Shatkay, Hagit; Gatzmann, Fanny; Sommer, Ingolf; Wass, Mark N.; Sternberg, Michael J.E.; Škunca, Nives; Supek, Fran; Bošnjak, Matko; Panov, Panče; Džeroski, Sašo; Šmuc, Tomislav; Kourmpetis, Yiannis A.I.; Van Dijk, Aalt D.J.; Ter Braak, Cajo J.F.; Zhou, Yuanpeng; Gong, Qingtian; Dong, Xinran; Tian, Weidong; Falda, Marco; Fontana, Paolo; Lavezzo, Enrico; Di Camillo, Barbara; Toppo, Stefano; Lan, Liang; Djuric, Nemanja; Guo, Yuhong; Vucetic, Slobodan; Bairoch, Amos; Linial, Michal; Babbitt, Patricia C.; Brenner, Steven E.; Orengo, Christine; Rost, Burkhard; Mooney, Sean D.; Friedberg, Iddo.

In: Nature Methods, Vol. 10, No. 3, 01.03.2013, p. 221-227.

Research output: Contribution to journal › Article

Radivojac, P, Clark, WT, Oron, TR, Schnoes, AM, Wittkop, T, Sokolov, A, Graim, K, Funk, C, Verspoor, K, Ben-Hur, A, Pandey, G, Yunes, JM, Talwalkar, AS, Repo, S, Souza, ML, Piovesan, D, Casadio, R, Wang, Z, Cheng, J, Fang, H, Gough, J, Koskinen, P, Törönen, P, Nokso-Koivisto, J, Holm, L, Cozzetto, D, Buchan, DWA, Bryson, K, Jones, DT, Limaye, B, Inamdar, H, Datta, A, Manjari, SK, Joshi, R, Chitale, M, Kihara, D, Lisewski, AM, Erdin, S, Venner, E, Lichtarge, O, Rentzsch, R, Yang, H, Romero, AE, Bhat, P, Paccanaro, A, Hamp, T, Kaßner, R, Seemayer, S, Vicedo, E, Schaefer, C, Achten, D, Auer, F, Boehm, A, Braun, T, Hecht, M, Heron, M, Hönigschmid, P, Hopf, TA, Kaufmann, S, Kiening, M, Krompass, D, Landerer, C, Mahlich, Y, Roos, M, Björne, J, Salakoski, T, Wong, A, Shatkay, H, Gatzmann, F, Sommer, I, Wass, MN, Sternberg, MJE, Škunca, N, Supek, F, Bošnjak, M, Panov, P, Džeroski, S, Šmuc, T, Kourmpetis, YAI, Van Dijk, ADJ, Ter Braak, CJF, Zhou, Y, Gong, Q, Dong, X, Tian, W, Falda, M, Fontana, P, Lavezzo, E, Di Camillo, B, Toppo, S, Lan, L, Djuric, N, Guo, Y, Vucetic, S, Bairoch, A, Linial, M, Babbitt, PC, Brenner, SE, Orengo, C, Rost, B, Mooney, SD & Friedberg, I 2013, 'A large-scale evaluation of computational protein function prediction', Nature Methods, vol. 10, no. 3, pp. 221-227. https://doi.org/10.1038/nmeth.2340

Radivojac, Predrag ; Clark, Wyatt T. ; Oron, Tal Ronnen ; Schnoes, Alexandra M. ; Wittkop, Tobias ; Sokolov, Artem ; Graim, Kiley ; Funk, Christopher ; Verspoor, Karin ; Ben-Hur, Asa ; Pandey, Gaurav ; Yunes, Jeffrey M. ; Talwalkar, Ameet S. ; Repo, Susanna ; Souza, Michael L. ; Piovesan, Damiano ; Casadio, Rita ; Wang, Zheng ; Cheng, Jianlin ; Fang, Hai ; Gough, Julian ; Koskinen, Patrik ; Törönen, Petri ; Nokso-Koivisto, Jussi ; Holm, Liisa ; Cozzetto, Domenico ; Buchan, Daniel W.A. ; Bryson, Kevin ; Jones, David T. ; Limaye, Bhakti ; Inamdar, Harshal ; Datta, Avik ; Manjari, Sunitha K. ; Joshi, Rajendra ; Chitale, Meghana ; Kihara, Daisuke ; Lisewski, Andreas M. ; Erdin, Serkan ; Venner, Eric ; Lichtarge, Olivier ; Rentzsch, Robert ; Yang, Haixuan ; Romero, Alfonso E. ; Bhat, Prajwal ; Paccanaro, Alberto ; Hamp, Tobias ; Kaßner, Rebecca ; Seemayer, Stefan ; Vicedo, Esmeralda ; Schaefer, Christian ; Achten, Dominik ; Auer, Florian ; Boehm, Ariane ; Braun, Tatjana ; Hecht, Maximilian ; Heron, Mark ; Hönigschmid, Peter ; Hopf, Thomas A. ; Kaufmann, Stefanie ; Kiening, Michael ; Krompass, Denis ; Landerer, Cedric ; Mahlich, Yannick ; Roos, Manfred ; Björne, Jari ; Salakoski, Tapio ; Wong, Andrew ; Shatkay, Hagit ; Gatzmann, Fanny ; Sommer, Ingolf ; Wass, Mark N. ; Sternberg, Michael J.E. ; Škunca, Nives ; Supek, Fran ; Bošnjak, Matko ; Panov, Panče ; Džeroski, Sašo ; Šmuc, Tomislav ; Kourmpetis, Yiannis A.I. ; Van Dijk, Aalt D.J. ; Ter Braak, Cajo J.F. ; Zhou, Yuanpeng ; Gong, Qingtian ; Dong, Xinran ; Tian, Weidong ; Falda, Marco ; Fontana, Paolo ; Lavezzo, Enrico ; Di Camillo, Barbara ; Toppo, Stefano ; Lan, Liang ; Djuric, Nemanja ; Guo, Yuhong ; Vucetic, Slobodan ; Bairoch, Amos ; Linial, Michal ; Babbitt, Patricia C. ; Brenner, Steven E. ; Orengo, Christine ; Rost, Burkhard ; Mooney, Sean D. ; Friedberg, Iddo. / A large-scale evaluation of computational protein function prediction. In: Nature Methods. 2013 ; Vol. 10, No. 3. pp. 221-227.

@article{16f42f11c6c048ed9ee1f48d38f94047,

title = "A large-scale evaluation of computational protein function prediction",

abstract = "Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the overwhelming majority of protein products can only be annotated computationally. If computational predictions are to be relied upon, it is crucial that the accuracy of these methods be high. Here we report the results from the first large-scale community-based critical assessment of protein function annotation (CAFA) experiment. Fifty-four methods representing the state of the art for protein function prediction were evaluated on a target set of 866 proteins from 11 organisms. Two findings stand out: (i) today's best protein function prediction algorithms substantially outperform widely used first-generation methods, with large gains on all types of targets; and (ii) although the top methods perform well enough to guide experiments, there is considerable need for improvement of currently available tools.",

author = "Predrag Radivojac and Clark, {Wyatt T.} and Oron, {Tal Ronnen} and Schnoes, {Alexandra M.} and Tobias Wittkop and Artem Sokolov and Kiley Graim and Christopher Funk and Karin Verspoor and Asa Ben-Hur and Gaurav Pandey and Yunes, {Jeffrey M.} and Talwalkar, {Ameet S.} and Susanna Repo and Souza, {Michael L.} and Damiano Piovesan and Rita Casadio and Zheng Wang and Jianlin Cheng and Hai Fang and Julian Gough and Patrik Koskinen and Petri T{\"o}r{\"o}nen and Jussi Nokso-Koivisto and Liisa Holm and Domenico Cozzetto and Buchan, {Daniel W.A.} and Kevin Bryson and Jones, {David T.} and Bhakti Limaye and Harshal Inamdar and Avik Datta and Manjari, {Sunitha K.} and Rajendra Joshi and Meghana Chitale and Daisuke Kihara and Lisewski, {Andreas M.} and Serkan Erdin and Eric Venner and Olivier Lichtarge and Robert Rentzsch and Haixuan Yang and Romero, {Alfonso E.} and Prajwal Bhat and Alberto Paccanaro and Tobias Hamp and Rebecca Ka{\ss}ner and Stefan Seemayer and Esmeralda Vicedo and Christian Schaefer and Dominik Achten and Florian Auer and Ariane Boehm and Tatjana Braun and Maximilian Hecht and Mark Heron and Peter H{\"o}nigschmid and Hopf, {Thomas A.} and Stefanie Kaufmann and Michael Kiening and Denis Krompass and Cedric Landerer and Yannick Mahlich and Manfred Roos and Jari Bj{\"o}rne and Tapio Salakoski and Andrew Wong and Hagit Shatkay and Fanny Gatzmann and Ingolf Sommer and Wass, {Mark N.} and Sternberg, {Michael J.E.} and Nives {\v S}kunca and Fran Supek and Matko Bo{\v s}njak and Pan{\v c}e Panov and Sa{\v s}o D{\v z}eroski and Tomislav {\v S}muc and Kourmpetis, {Yiannis A.I.} and {Van Dijk}, {Aalt D.J.} and {Ter Braak}, {Cajo J.F.} and Yuanpeng Zhou and Qingtian Gong and Xinran Dong and Weidong Tian and Marco Falda and Paolo Fontana and Enrico Lavezzo and {Di Camillo}, Barbara and Stefano Toppo and Liang Lan and Nemanja Djuric and Yuhong Guo and Slobodan Vucetic and Amos Bairoch and Michal Linial and Babbitt, {Patricia C.} and Brenner, {Steven E.} and Christine Orengo and Burkhard Rost and Mooney, {Sean D.} and Iddo Friedberg",

year = "2013",

month = mar,

day = "1",

doi = "10.1038/nmeth.2340",

language = "English (US)",

volume = "10",

pages = "221--227",

journal = "PLoS Medicine",

issn = "1549-1277",

publisher = "Nature Publishing Group",

number = "3",

}

TY - JOUR

T1 - A large-scale evaluation of computational protein function prediction

AU - Radivojac, Predrag

AU - Clark, Wyatt T.

AU - Oron, Tal Ronnen

AU - Schnoes, Alexandra M.

AU - Wittkop, Tobias

AU - Sokolov, Artem

AU - Graim, Kiley

AU - Funk, Christopher

AU - Verspoor, Karin

AU - Ben-Hur, Asa

AU - Pandey, Gaurav

AU - Yunes, Jeffrey M.

AU - Talwalkar, Ameet S.

AU - Repo, Susanna

AU - Souza, Michael L.

AU - Piovesan, Damiano

AU - Casadio, Rita

AU - Wang, Zheng

AU - Cheng, Jianlin

AU - Fang, Hai

AU - Gough, Julian

AU - Koskinen, Patrik

AU - Törönen, Petri

AU - Nokso-Koivisto, Jussi

AU - Holm, Liisa

AU - Cozzetto, Domenico

AU - Buchan, Daniel W.A.

AU - Bryson, Kevin

AU - Jones, David T.

AU - Limaye, Bhakti

AU - Inamdar, Harshal

AU - Datta, Avik

AU - Manjari, Sunitha K.

AU - Joshi, Rajendra

AU - Chitale, Meghana

AU - Kihara, Daisuke

AU - Lisewski, Andreas M.

AU - Erdin, Serkan

AU - Venner, Eric

AU - Lichtarge, Olivier

AU - Rentzsch, Robert

AU - Yang, Haixuan

AU - Romero, Alfonso E.

AU - Bhat, Prajwal

AU - Paccanaro, Alberto

AU - Hamp, Tobias

AU - Kaßner, Rebecca

AU - Seemayer, Stefan

AU - Vicedo, Esmeralda

AU - Schaefer, Christian

AU - Achten, Dominik

AU - Auer, Florian

AU - Boehm, Ariane

AU - Braun, Tatjana

AU - Hecht, Maximilian

AU - Heron, Mark

AU - Hönigschmid, Peter

AU - Hopf, Thomas A.

AU - Kaufmann, Stefanie

AU - Kiening, Michael

AU - Krompass, Denis

AU - Landerer, Cedric

AU - Mahlich, Yannick

AU - Roos, Manfred

AU - Björne, Jari

AU - Salakoski, Tapio

AU - Wong, Andrew

AU - Shatkay, Hagit

AU - Gatzmann, Fanny

AU - Sommer, Ingolf

AU - Wass, Mark N.

AU - Sternberg, Michael J.E.

AU - Škunca, Nives

AU - Supek, Fran

AU - Bošnjak, Matko

AU - Panov, Panče

AU - Džeroski, Sašo

AU - Šmuc, Tomislav

AU - Kourmpetis, Yiannis A.I.

AU - Van Dijk, Aalt D.J.

AU - Ter Braak, Cajo J.F.

AU - Zhou, Yuanpeng

AU - Gong, Qingtian

AU - Dong, Xinran

AU - Tian, Weidong

AU - Falda, Marco

AU - Fontana, Paolo

AU - Lavezzo, Enrico

AU - Di Camillo, Barbara

AU - Toppo, Stefano

AU - Lan, Liang

AU - Djuric, Nemanja

AU - Guo, Yuhong

AU - Vucetic, Slobodan

AU - Bairoch, Amos

AU - Linial, Michal

AU - Babbitt, Patricia C.

AU - Brenner, Steven E.

AU - Orengo, Christine

AU - Rost, Burkhard

AU - Mooney, Sean D.

AU - Friedberg, Iddo

PY - 2013/3/1

Y1 - 2013/3/1

N2 - Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the overwhelming majority of protein products can only be annotated computationally. If computational predictions are to be relied upon, it is crucial that the accuracy of these methods be high. Here we report the results from the first large-scale community-based critical assessment of protein function annotation (CAFA) experiment. Fifty-four methods representing the state of the art for protein function prediction were evaluated on a target set of 866 proteins from 11 organisms. Two findings stand out: (i) today's best protein function prediction algorithms substantially outperform widely used first-generation methods, with large gains on all types of targets; and (ii) although the top methods perform well enough to guide experiments, there is considerable need for improvement of currently available tools.

AB - Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the overwhelming majority of protein products can only be annotated computationally. If computational predictions are to be relied upon, it is crucial that the accuracy of these methods be high. Here we report the results from the first large-scale community-based critical assessment of protein function annotation (CAFA) experiment. Fifty-four methods representing the state of the art for protein function prediction were evaluated on a target set of 866 proteins from 11 organisms. Two findings stand out: (i) today's best protein function prediction algorithms substantially outperform widely used first-generation methods, with large gains on all types of targets; and (ii) although the top methods perform well enough to guide experiments, there is considerable need for improvement of currently available tools.

UR - http://www.scopus.com/inward/record.url?scp=84874663959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874663959&partnerID=8YFLogxK

U2 - 10.1038/nmeth.2340

DO - 10.1038/nmeth.2340

M3 - Article

C2 - 23353650

AN - SCOPUS:84874663959

VL - 10

SP - 221

EP - 227

JO - PLoS Medicine

JF - PLoS Medicine

SN - 1549-1277

IS - 3

ER -

Access to Document

10.1038/nmeth.2340