NIPY logo

Site Navigation

NIPY Community

Table Of Contents

Previous topic

algorithms.statistics.bench.bench_intvol

This Page

algorithms.statistics.empirical_pvalue

Module: algorithms.statistics.empirical_pvalue

Inheritance diagram for nipy.algorithms.statistics.empirical_pvalue:

This module contains several routines to get corrected p-values estimates, based on the observation of data/p-values. It yields 3 main approaches: - benjamini-Hochberg fdr - a class that fits a gaussian model to the central part of an histogram, following schwartzman et al, 2009. This is typically necessary to estimate a fdr when one is not certain that the data behaves as a standard normal under H_0. - a model based on gaussian mixture modelling

Author : Bertrand Thirion, 2008-2011

Classes

FDR

class nipy.algorithms.statistics.empirical_pvalue.FDR(pv=None)

Bases: object

Basic class to handle false discovery rate computation Members: x the samples from which the fdr is derived, assumed to be a normal variate

The Benjamini-Horchberg procedure is used

Methods

all_fdr
check_pv
pth_from_pvals
__init__(pv=None)
Parameters :pv: array of p-values :
all_fdr(pv=None, verbose=0)

Returns the fdr associated with each the values

Parameters :

pv : ndarray of shape (n)

The samples p-value

Returns :

q : array of shape(n)

The corresponding fdrs

check_pv(pv)

Do some basic checks on the pv array: each value should be within [0,1]

Parameters :

pv : array of shape (n)

The sample p-values

Returns :

pv : array of shape (n)

The sample p-values

pth_from_pvals(pv=None, alpha=0.05)

Returns the critical p-value associated with an FDR alpha

Parameters :

pv : array of shape (n), optional

The samples p-value

alpha : float, optional

The desired FDR significance

Returns :

pth: float :

The p value corresponding to the FDR alpha

NormalEmpiricalNull

class nipy.algorithms.statistics.empirical_pvalue.NormalEmpiricalNull(x)

Bases: object

Class to compute the empirical null normal fit to the data.

The data which is used to estimate the FDR, assuming a gaussian null from Schwartzmann et al., NeuroImage 44 (2009) 71–82

Methods

fdr
fdrcurve
learn
plot
threshold
uncorrected_threshold
__init__(x)

Initialize an empirical null normal object.

Parameters :

x : 1D ndarray

The data used to estimate the empirical null.

fdr(theta)

Given a threshold theta, find the estimated fdr

Returns :afp: value of array of shape(n) :
fdrcurve()

Returns the fdr associated with any point of self.x

learn(left=0.2, right=0.8)

Estimate the proportion, mean and variance of a gaussian distribution for a fraction of the data

Parameters :

left: float, optional :

Left cut parameter to prevent fitting non-gaussian data

right: float, optional :

Right cut parameter to prevent fitting non-gaussian data

plot(efp=None, alpha=0.05, bar=1, mpaxes=None)

plot the histogram of x

Parameters :

efp : float, optional

The empirical fdr (corresponding to x) if efp==None, the false positive rate threshod plot is not drawn.

alpha : float, optional

The chosen fdr threshold

bar=1 : bool, optional

mpaxes=None: if not None, handle to an axes where the fig :

will be drawn. Avoids creating unnecessarily new figures :

threshold(alpha=0.05, verbose=0)

Compute the threshold correponding to an alpha-level fdr for x

Parameters :

alpha : float, optional

the chosen false discovery rate threshold.

verbose : boolean, optional

the verbosity level, if True a plot is generated.

uncorrected_threshold(alpha=0.001, verbose=0)

Compute the threshold correponding to a specificity alpha for x

Parameters :

alpha : float, optional

the chosen false discovery rate threshold.

verbose : boolean, optional

the verbosity level, if True a plot is generated.

Functions

nipy.algorithms.statistics.empirical_pvalue.Gamma_Gaussian_fit(x, test=None, verbose=0, mpaxes=None, bias=1, gaussian_mix=0, return_estimator=False)

Computing some prior probabilities that the voxels of a certain map are in class disactivated, null or active uning a gamma-Gaussian mixture

Parameters :

x: array of shape (nvox,) :

the map to be analysed

test: array of shape (nbitems,), optional :

the test values for which the p-value needs to be computed by default, test = x

verbose: 0, 1 or 2, optional :

verbosity mode, 0 is quiet, and 2 calls matplotlib to display graphs.

mpaxes: matplotlib axes, option. :

axes handle used to plot the figure in verbose mode if None, new axes are created

bias: float, optional :

lower bound on the gaussian variance (to avoid shrinkage)

gaussian_mix: float, optional :

if nonzero, lower bound on the gaussian mixing weight (to avoid shrinkage)

return_estimator: boolean, optional :

If return_estimator is true, the estimator object is returned.

Returns :

bfp: array of shape (nbitems,3) :

The probability of each component in the mixture model for each test value

estimator: nipy.labs.clustering.ggmixture.GGGM object :

The estimator object, returned only if return_estimator is true.

nipy.algorithms.statistics.empirical_pvalue.all_fdr_gaussian(x)

Return the fdr of all values assuming a Gaussian distribution

nipy.algorithms.statistics.empirical_pvalue.gaussian_fdr_threshold(x, alpha=0.05)

Given an array x of normal variates, this function returns the critical p-value associated with alpha. x is explicitly assumed to be normal distributed under H_0

Parameters :

x: ndarray, the input data :

alpha: float, optional, the desired significance :

Returns :

th: float, :

The threshold in variate value

nipy.algorithms.statistics.empirical_pvalue.smoothed_histogram_from_samples(x, bins=None, nbins=256, normalized=False)

Returns the smooth histogram corresponding to the density underlying the samples in x

Parameters :

x: array of shape(n_samples), :

input data

bins: array of shape(nbins+1), optional, :

the bins location

nbins: int, optional, :

the number of bins of the resulting histogram

Normalized: bool, optional :

if True, the result is returned as a density value

Returns :

h: array of shape (nbins) :

the histogram

bins: array of shape(nbins+1), :

the bins location

nipy.algorithms.statistics.empirical_pvalue.three_classes_GMM_fit(x, test=None, alpha=0.01, prior_strength=100, verbose=0, fixed_scale=False, mpaxes=None, bias=0, theta=0, return_estimator=False)

Fit the data with a 3-classes Gaussian Mixture Model, i.e. computing some probability that the voxels of a certain map are in class disactivated, null or active

Parameters :

x array of shape (nvox,1): the map to be analysed :

test=None array of shape(nbitems,1): :

the test values for which the p-value needs to be computed by default, test=x

alpha = 0.01 the prior weights of the positive and negative classes :

prior_strength = 100 the confidence on the prior :

(should be compared to size(x))

verbose=0 : verbosity mode

fixed_scale = False, boolean, variance parameterization :

if True, the variance is locked to 1 otherwise, it is estimated from the data

mpaxes=None: axes handle used to plot the figure in verbose mode :

if None, new axes are created

bias = 0: allows a recaling of the posterior probability :

that takes into account the thershold theta. Not rigorous.

theta = 0 the threshold used to correct the posterior p-values :

when bias=1; normally, it is such that test>theta note that if theta = -np.infty, the method has a standard behaviour

return_estimator: boolean, optional :

If return_estimator is true, the estimator object is returned.