TY - JOUR

T1 - Joint adaptive meanvariance regularization and variance stabilization of high dimensional data

AU - Dazard, Jean Eudes

AU - Sunil Rao, J.

N1 - Funding Information:
We thank Dr. Hemant Ishwaran for helpful discussion and for providing the R code of his CART Variance Stabilization and Regularization procedure (CVSR). This work was supported in part by the National Institutes of Health [ P30-CA043703 to J-E.D., R01-GM085205 to J.S.R.]; and the National Science Foundation [ DMS-0806076 to J.S.R.]. Conflict of Interest : None declared.

PY - 2012/7

Y1 - 2012/7

N2 - The paper addresses a common problem in the analysis of high-dimensional high-throughput "omics" data, which is parameter estimation across multiple variables in a set of data where the number of variables is much larger than the sample size. Among the problems posed by this type of data are that variable-specific estimators of variances are not reliable and variable-wise tests statistics have low power, both due to a lack of degrees of freedom. In addition, it has been observed in this type of data that the variance increases as a function of the mean. We introduce a non-parametric adaptive regularization procedure that is innovative in that (i) it employs a novel "similarity statistic"-based clustering technique to generate local-pooled or regularized shrinkage estimators of population parameters, (ii) the regularization is done jointly on population moments, benefiting from C. Stein's result on inadmissibility, which implies that usual sample variance estimator is improved by a shrinkage estimator using information contained in the sample mean. From these joint regularized shrinkage estimators, we derived regularized t-like statistics and show in simulation studies that they offer more statistical power in hypothesis testing than their standard sample counterparts, or regular common value-shrinkage estimators, or when the information contained in the sample mean is simply ignored. Finally, we show that these estimators feature interesting properties of variance stabilization and normalization that can be used for preprocessing high-dimensional multivariate data. The method is available as an R package, called 'MVR' ('MeanVariance Regularization'), downloadable from the CRAN website.

AB - The paper addresses a common problem in the analysis of high-dimensional high-throughput "omics" data, which is parameter estimation across multiple variables in a set of data where the number of variables is much larger than the sample size. Among the problems posed by this type of data are that variable-specific estimators of variances are not reliable and variable-wise tests statistics have low power, both due to a lack of degrees of freedom. In addition, it has been observed in this type of data that the variance increases as a function of the mean. We introduce a non-parametric adaptive regularization procedure that is innovative in that (i) it employs a novel "similarity statistic"-based clustering technique to generate local-pooled or regularized shrinkage estimators of population parameters, (ii) the regularization is done jointly on population moments, benefiting from C. Stein's result on inadmissibility, which implies that usual sample variance estimator is improved by a shrinkage estimator using information contained in the sample mean. From these joint regularized shrinkage estimators, we derived regularized t-like statistics and show in simulation studies that they offer more statistical power in hypothesis testing than their standard sample counterparts, or regular common value-shrinkage estimators, or when the information contained in the sample mean is simply ignored. Finally, we show that these estimators feature interesting properties of variance stabilization and normalization that can be used for preprocessing high-dimensional multivariate data. The method is available as an R package, called 'MVR' ('MeanVariance Regularization'), downloadable from the CRAN website.

KW - Bioinformatics

KW - Inadmissibility

KW - Normalization

KW - Regularization

KW - Shrinkage estimators

KW - Variance stabilization

UR - http://www.scopus.com/inward/record.url?scp=84857633258&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84857633258&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2012.01.012

DO - 10.1016/j.csda.2012.01.012

M3 - Article

AN - SCOPUS:84857633258

VL - 56

SP - 2317

EP - 2333

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

IS - 7

ER -