07. Comparing Variances

Our hypothesis tests thus far exclusively considered the means of datasets. It is also possible to compare the variances or standard deviations of tests. In this lesson, we will touch upon the two probability density functions, the

\chi^{2}

and

F

distributions. We will use these to compare our standard deviation with a reference, and compare two variances.

Learning Goals #

Calculate the confidence interval of the variance.
Conduct hypothesis tests using the $\chi^2$ and $F$ -distributions.
Compare two variances in a hypothesis test.
Compare a standard deviation with a reference value.

Analytical Separation Science by B.W.J. Pirok and P.J. Schoenmakers

READ SECTION 9.3.6

Comparison of standard deviation with reference

1. Comparison of standard deviation with reference #

The first variance test that we will learn about compares a variance or standard deviation with a reference value. For analytical chemists this is highly useful as it allows us to assess whether our analytical instrument $s$ meets the certified precision $\sigma_0$ .

CASE: PRECISION OF INSTRUMENT

Let’s see how this test works by taking a case as example. An LC autosampler injection system is capable of injecting volumes of 10 μL. The manufacturer guarantees that the precision (i.e. standard deviation) is 0.07 μL or less. A calibration standard is measured 10 times and $s$ of 0.1 is found. Is the precision of our instrument worse than 0.07?

To do the comparison, the $\chi^2$ -test for the variance can be used. It employs the $\chi^2$ -distribution, which is given by

Equation 9.33: $f(x)=\Bigg\{\begin{matrix}{\frac{x^{(\nu /2)-1}\cdot e^{-(x/2)}}{2^{\nu/2} \cdot \Gamma(\nu/2)}} & {x>0} \\{0} & \text{otherwise}\end{matrix}$

The book addresses this distribution in further detail. For now, we consider that the degrees of freedom are given by $\nu=n-1$ . Figure 1 demonstrates the strong dependence of the distribution on $\nu$ .

Figure 1. The

\chi^2

distribution plotted for different number of degrees of freedom.

This test functions similar to the earlier hypothesis tests that we have encountered. Our hypotheses are

$H_0: \sigma^2\leq\sigma^{2}_0$ and $H_1: \sigma^2>\sigma^{2}_0$

NOTE

Do not confuse the direction of precision and the standard deviation. A higher precision means a smaller standard deviation and variance!

Similar to the other hypothesis test, a critical value is compared with an observed value (or a $p$ -value is compared with $\alpha$ ). The $\chi_{\text{obs}}^2$ statistic can be calculated as:

Equation 9.34: $\chi_{\text{obs}}^2 = \frac{(n-1)s^2}{\sigma_{0}^2}$

The critical value is computed as

				
					chi2_crit = icdf('Chi2',probability,degrees_of_freedom);

The CHISQ.INV() function can be used. It is operated with two input arguments:

CHISQ.INV(probability,degrees_of_freedom)

				
					using Distributions

# Define the Chi-squared distribution 
chi2_dist   = Chisq(degrees_of_freedom)

# Compute the critical value
chi2_crit   = quantile(chi2_dist, probability)

whereas the $p$ -value is calculated through

				
					p = cdf('Chi2',chi2_obs,degrees_of_freedom);

The CHISQ.DIST() function can be used. It is operated with two input arguments:

CHISQ.DIST(chi2_obs,degrees_of_freedom,’TRUE’)

				
					using Distributions

# Define the Chi-squared distribution 
chi2_dist   = Chisq(degrees_of_freedom)

# Compute the cumulative probability
p           = cdf(chi1, chi2_obs)

Calculate the relevant statistics and conduct the hypothesis test at a significance level of 0.05.

Correct! The observed value of 18.37 exceeds the critical value of 16.91 in this right-tail test. This is confirmed by the p-value which is 0.031 and larger than 0.05.

This is incorrect. These hints may help you on: you should arrive at an observed value of 18.37. This is a right-tail test.

2. Confidence interval of variance #

A useful ability that we can also touch upon is computing the confidence interval of the variance. This is given by

Equation 9.35: $\frac{(n-1)s^2}{\chi^{2}_{1-\alpha/2,n-1}} < \sigma^2 < \frac{(n-1)s^2}{\chi^{2}_{\alpha/2,n-1}}$

At this stage you may realise that you rarely have encountered a confidence interval for the variance. This is not surprising, because they are generally considered less reliable due to their heavy dependency on the underlying data to be normally distributed.

Calculate the 95% confidence interval for the instrument based on the data from the 10 repeated measurements from Exercise 1. Report it according to Equation 9.35.

This is the correct answer!

Unfortunately this is not correct.

3. Comparison of two variances #

The final test type that we are covering in this lesson is the comparison of two variances. The so-called $F$ -distribution can be used. $F$ stands for Fisher, named after British scientist Aylmer Fisher. This distribution requires the degrees of freedom of each variance to be specified, and is given by

Equation 9.36: $f(x)=\frac{\sqrt{\frac{{\nu_A x}^{\nu_A} \nu_{B}^{\nu_B}}{{(\nu_A x+\nu_B)}^{\nu_A+\nu_B}}}}{x \Beta(\frac{\nu_A}{2},\frac{\nu_B}{2})}$

Here, $\Beta$ is the beta function, which is treated further in the book. The distribution is shown in Figure 2. As can be seen, the order in which the degrees of freedom are specified seriously affect the distribution shape. You are therefore recommended to ensure that these are correct.

Figure 2. The

F

distribution plotted for different number of degrees of freedom. We can see that the order in which we specify the degrees of freedom matters a lot, therefore we must be very consistent!

The requirements are similar to those of other tests, but the $F$ test is extremely sensitive to deviations from normality. Outliers are therefore also not desirable.

NOTE

Figure 2 demonstrates how the $F$ -distribution strongly depends on the order in which the degrees of freedom are specified of the two samples. If you encounter an $F_{\text{obs}}$ statistic that is smaller than 1 and decide to flip the variances in Equation 9.38 so as to end up with a value above 1, make sure that you adjust the $F$ -distribution parameters for the degrees of freedom accordingly!

The $F$ statistic is calculated through

Equation 9.38:

F_{\text{obs}}=\frac{s^{2}_A}{s^{2}_B}

where for the test the larger variance is placed in the numerator so that $F_{\text{obs}} \ge 1$ . From this point on, the hypothesis test works similar to the others. The critical value

				
					F_crit = icdf('F',probability,dof_A,dof_B);

and/or $p$ -value

				
					p = cdf('F',F_obs,dof_A,dof_B);

can be computed and then compared to $F_{\text{obs}}$ and/or $\alpha$ , respectively.

CASE: RETENTION TIME STABILITY

Suppose we want to compare the stability of the retention time for an analyse on an old system relative to a newer system. Ten repetitions on the old system yield $\bar{x}_{\text{old}}$ = 1.37 min and $s_{\text{old}}$ = 0.033 min. The new system yields $\bar{x}_{\text{new}}$ = 1.42 min and $s_{\text{new}}$ = 0.028 min, based on 7 repetitions. Does the new system yield a better retention time stability?

The F.DIST() function can be used. It is operated with four input arguments:

CHISQ.DIST(F_obs,dof_A,dof_B,’TRUE’)

				
					using Distributions

# Define the F distribution
f_dist  	= FDist(dof_A, dof_B)

# Compute the cumulative probability
p           = cdf(f_dist, F_obs)

The F.INV() function can be used. It is operated with three input arguments:

F.INV(probability,dof_A,dof_B)

				
					using Distributions

# Define the F distribution
f_dist  	= FDist(dof_A, dof_B)

# Compute the critical value
F_crit      = quenatile(f_dist, probability)

This is clearly a one-sided test, given that the case information specifies that the interest is to find out if the new system is “better”. Two sets of hypotheses are then possible:

Option A:

H_0: \sigma^{2}_{\text{old}}\leq\sigma^{2}_{\text{new}}

and

H_1: \sigma^{2}_{\text{old}}>\sigma^{2}_{\text{new}}

Option B:

H_0: \sigma^{2}_{\text{new}}\leq\sigma^{2}_{\text{old}}

and

H_1: \sigma^{2}_{\text{new}}>\sigma^{2}_{\text{old}}

Which of the two sets of hypotheses is correct? Explain (in your mind) your answer.

This is correct! Note that the statistical effect that we are investigating (i.e. the new instrument to be better) should be in the alternative hypothesis. Crucially, we must remember that a better precision means a lower standard deviation and variance! For Option A, the alternative hypothesis suggests mathematically “the newer system has a better precision than the older one”. This is what we would like. An additional advantage is that we now obtain a right-tail test and thus an $F$ -statistic that is larger than 1.

Unfortunately, this is not correct. Note that the statistical effect that we are investigating (i.e. the new instrument to be better) should be in the alternative hypothesis. Crucially, we must remember that a better precision means a lower standard deviation and variance! For Option A, the alternative hypothesis suggests mathematically “the newer system has a better precision than the older one”. This is what we would like. An additional advantage is that we now obtain a right-tail test and thus an $F$ -statistic that is larger than 1.

Conduct the $F$ -test at $\alpha$ . What is the $p$ -value? Round to 3 decimals.

This is correct! The variance pertaining to the old system features 9 degrees of freedom, and the new system 6 degrees of freedom. We thus accept $H_0$ and conclude that there is no evidence to suggest that the system is significantly better in terms of precision.

This is unfortunately not correct, yet! Did you place the degrees of freedom in the correct order? You may want to try swapping them around.

Concluding remarks #

We now also have the capability of comparing variances with hypothesis tests. This skill will prove useful for several applications which are addressed in the upcoming lessons.