Comparisons of t-distribution and Normal distribution

This tutorial compares t-distribution and normal distribution by explaining the similarities and connections between t-distribution and normal distribution.

Similarities between t-distribution and normal distribution

There are a few similarities between t-distribution and normal distribution.

1. Both have bell-shaped density curves (see the figure below).
2. Both standard normal distribution and t-distribution have a mean of zero (see the figure below).
3. As the sample size increases, the t-distribution becomes more similar to a normal distribution (see the figure below).

The following figure shows the t-distribution density function curve and the standard normal curve. As we can see, as the sample size increases (i.e., the degree of freedom increases as well), t-distribution becomes more similar to standard normal distribution.

Connections between t-distribution and normal distribution

t-distribution actually starts from a normal distribution. In particular, assuming that X follows the normal distribution. We can get a sample (i.e., sample 1) and calculate its mean (i.e., mean 1). We can do this 100 times (i.e., 100 different samples) and get 100 different means. Assuming that we know the standard deviation of the population $$\sigma$$, all these means will follow a normal distribution as follows,

$$\bar{X} \sim N(\mu, \frac{\sigma }{\sqrt{n}} )$$

Note that, here we are talking about the random variable of mean $$\bar{X}$$. For the mean, its standard deviation is also called standard error (SE). Thus,

$$SE = \frac{\sigma }{ \sqrt{n}}$$

Note that, such a normal distribution assumes that the population standard deviation $$\sigma$$ is known. In reality, we probably do not know the population standard deviation. Thus, the t-statistic comes in. t-statistic follows t-distribution and is defined as follows. The t-distribution needs to estimate the standard deviation $$s$$ from a sample.

$$t= \frac{\bar{x} -\mu}{ s/\sqrt{n}}$$

where,

$$\bar{x} = \frac{\sum_{i=1}^n x_i}{n}$$

$$\mu: population \; mean$$

$$s = \sqrt {\frac{1}{n-1} \sum_{i=1}^n (x_i – \bar{x})^2}$$

Since we need to estimate standard deviation, $$\bar{X}$$ does not follow the Central Limit Theorem anymore and thus $$\bar{X}$$ does not follow normal distribution anymore. Instead, $$\bar{X}$$ follows t-distribution. The t-distribution is defined by the degrees of freedom. The degree of freedom is related to the sample size. That is, t-distribution has only one parameter. In contrast, the normal distribution is defined by the mean and the standard deviation.

References

(PDF) THE t TEST: An Introduction

The t-Distribution (JMP website)

Student’s t-distribution (Wikipedia)