Joint adaptive meanvariance regularization and variance stabilization of high dimensional data

Jean Eudes Dazard, Jonnagadda S Rao

Research output: Contribution to journalArticle

7 Scopus citations

Abstract

The paper addresses a common problem in the analysis of high-dimensional high-throughput "omics" data, which is parameter estimation across multiple variables in a set of data where the number of variables is much larger than the sample size. Among the problems posed by this type of data are that variable-specific estimators of variances are not reliable and variable-wise tests statistics have low power, both due to a lack of degrees of freedom. In addition, it has been observed in this type of data that the variance increases as a function of the mean. We introduce a non-parametric adaptive regularization procedure that is innovative in that (i) it employs a novel "similarity statistic"-based clustering technique to generate local-pooled or regularized shrinkage estimators of population parameters, (ii) the regularization is done jointly on population moments, benefiting from C. Stein's result on inadmissibility, which implies that usual sample variance estimator is improved by a shrinkage estimator using information contained in the sample mean. From these joint regularized shrinkage estimators, we derived regularized t-like statistics and show in simulation studies that they offer more statistical power in hypothesis testing than their standard sample counterparts, or regular common value-shrinkage estimators, or when the information contained in the sample mean is simply ignored. Finally, we show that these estimators feature interesting properties of variance stabilization and normalization that can be used for preprocessing high-dimensional multivariate data. The method is available as an R package, called 'MVR' ('MeanVariance Regularization'), downloadable from the CRAN website.

Original languageEnglish
Pages (from-to)2317-2333
Number of pages17
JournalComputational Statistics and Data Analysis
Volume56
Issue number7
DOIs
StatePublished - Jul 1 2012

    Fingerprint

Keywords

  • Bioinformatics
  • Inadmissibility
  • Normalization
  • Regularization
  • Shrinkage estimators
  • Variance stabilization

ASJC Scopus subject areas

  • Computational Mathematics
  • Computational Theory and Mathematics
  • Statistics and Probability
  • Applied Mathematics

Cite this