Standard Error of Statistics

The term standard error has already been introduced very briefly in previous tutorials while discussing the sampling distribution of means. In this tutorial we will discuss standard error in more detail.

In the broader sense of the term, standard errors of mean, median, standard deviation, coefficient of correlation, regression coefficients, etc., all refer to the sample and are therefore known as sample statistics. The standard error of the mean is very common and widely used.

The standard error of any statistic may be determined by first drawing all possible samples (with replacement) of size $n$ from the given population, and then the required statistic (mean, median or standard deviation, etc.) is computed for each sample. We therefore have a number of values of that statistic (one value for each sample). These values form the so-called distribution of the sample statistic (e.g., the distribution of sample mean). If we calculate the standard deviation of such a distribution it will be referred to as the standard error of that statistic.

This method of determining the standard error is sometimes impractical. Fortunately we have relationships between the population’s standard deviation, the standard error and the sample size which can be empirically verified. The standard error of some important statistics are presented in the table below, where $\sigma$ is the standard deviation of the population and $n$ is the sample size.

 Statistics Standard Errors SAMPLE MEAN $\sigma /\sqrt n$ SAMPLE MEDIAN $\sigma /\sqrt {\frac{\pi }{{2n}}}$ SAMPLE STANDRAD DEVIATION $\sigma /\sqrt {2n}$ SAMPLE REGRESSION COEFFIECENT $\frac{{{\sigma _X}}}{{{\sigma _Y}}}\sqrt {\frac{{1 – {\gamma ^2}}}{n}}$ SAMPLE COEFFICIENT OF CORRELATION $\sqrt {\frac{{1 – {\gamma ^2}}}{n}}$

From the list of formulae given above it can be seen that the standard error depends, in most of the cases, on two factors: the sample size and the standard deviation of the population. It measures the precision of the concerned statistics in estimating the parameters. In the above formulae, if the population is large and the population’s standard deviation $\sigma$ is unknown, then $\sigma$ can be replaced by $S$ (the sample standard deviation), as defined in previous tutorials without significantly affecting our estimate of the standard error, especially when the sample size is reasonably large.