Confidence Interval

A Confidence Interval is a technique in Statistics, where instead of estimating a Parameter by a Point Estimator alone, we create an interval that we know contains the unknown parameter with a certain Probability.

A $100(1-\alpha )\%$ Confidence Interval for an unknown parameter $\theta$ is given by $L,U\in \text{RV}$ satisfying

$$P(L\le \theta \le U)=1-\alpha$$

• where $L$ is the lower bound
• where $U$ is the upper bound
• where $1-\alpha$ is the Confidence Coefficient

Confidence Interval Scenarios

• 1 population Confidence Interval
  • $\mu$
    • ${\sigma }^2$ known
      • $Z=\frac{\overline{X}-\mu }{\sigma /\sqrt{n}}\sim N(0,1)$
      • $\mu \in \overline{X}\pm z_{\alpha /2}\frac{\sigma }{\sqrt{n}}$
    • ${\sigma }^2$ unknown
      • $T=\frac{\overline{X}-\mu }{S/\sqrt{n}}\sim t_{(n-1)}$
      • $\mu \in \overline{X}\pm t_{\alpha /2,(n-1)}\frac{S}{\sqrt{n}}$
  • ${\sigma }^2$
    • ${\chi }^2=\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }_{(n-1)}^2$
    • ${\sigma }^2\in \left[\frac{(n-1)S^2}{{\chi }_{1-\alpha /2,(n-1)}^2},\frac{(n-1)S^2}{{\chi }_{\alpha /2,(n-1)}^2}\right]$
  • $p$
    • $Z=\frac{\overline{X}-p}{\sqrt{pq/n}}=\frac{\hat{p}-E(\hat{p})}{\sqrt{\text{Var}(\hat{p})}}\sim N(0,1)$
    • $p\in \overline{X}\pm z_{\alpha /2}\sqrt{\frac{\overline{X}(1-\overline{X})}{n}}$

• 2 population Confidence Interval
  • ${\mu }_X-{\mu }_Y$
    • $\forall {\sigma }^2$ known
      • $Z=\frac{(\overline{X}-\overline{Y})-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{{\sigma }_X^2}{n_X}+\frac{{\sigma }_Y^2}{n_Y}}}\sim N(0,1)$
      • $({\mu }_X-{\mu }_Y)\in (\overline{X}-\overline{Y})\pm z_{\alpha /2}\sqrt{\frac{{\sigma }_X^2}{n_X}+\frac{{\sigma }_Y^2}{n_Y}}$
    • $\forall {\sigma }^2\text{ unknown}\wedge {\sigma }_X^2={\sigma }_Y^2$
      • $T=\frac{(\overline{X}-\overline{Y})-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{S_p^2}{n_X}+\frac{S_p^2}{n_Y}}}\sim t_{(n_X+n_Y-2)}$
      • $S_p^2=\frac{(n_X-1)S_X^2+(n_Y-1)S_Y^2}{n_X+n_Y-2}$; bonus: is unbiased estimator for ${\sigma }^2$
      • $({\mu }_X-{\mu }_Y)\in (\overline{X}-\overline{Y})\pm t_{\alpha /2,(n_X+n_Y-2)}\sqrt{\frac{S_p^2}{n_X}+\frac{S_p^2}{n_Y}}$
    • $\forall {\sigma }^2\text{ unknown}\wedge {\sigma }_X^2\ne {\sigma }_Y^2$
      • $T=\frac{(\overline{X}-\overline{Y})-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{S_X^2}{n_X}+\frac{S_Y^2}{n_Y}}}\sim t_{(\nu )}$
      • $\nu =\frac{{\left(\frac{S_X^2}{n_X}+\frac{S_Y^2}{n_Y}\right)}^2}{\frac{(S_X^2/n_X{)}^2}{n_X+1}+\frac{(S_Y^2/n_Y{)}^2}{n_Y+1}}-2$
      • $({\mu }_X-{\mu }_Y)\in (\overline{X}-\overline{Y})\pm t_{\alpha /2,\nu }\sqrt{\frac{S_X^2}{n_X}+\frac{S_Y^2}{n_Y}}$
  • $\frac{{\sigma }_X^2}{{\sigma }_Y^2}$
    • $F=\frac{S_X^2/{\sigma }_X^2}{S_Y^2/{\sigma }_Y^2}\sim \frac{{\chi }_{n_X-1}^2/(n_X-1)}{{\chi }_{n_Y-1}^2/(n_Y-1)}\sim F_{(n_X-1,n_Y-1)}$
    • $\frac{{\sigma }_X^2}{{\sigma }_Y^2}\in \left[\frac{S_X^2}{S_Y^2}\frac{1}{F_{\alpha /2,(n_X-1,n_Y-1)}},\frac{S_X^2}{S_Y^2}\frac{1}{F_{1-\alpha /2,(n_X-1,n_Y-1)}}\right]$
  • $p_X-p_Y$
    • $Z=\frac{({\hat{p}}_X-{\hat{p}}_Y)-(p_X-p_Y)}{\sqrt{\frac{{\hat{p}}_X(1-{\hat{p}}_X)}{n_X}+\frac{{\hat{p}}_Y(1-{\hat{p}}_Y)}{n_Y}}}\sim N(0,1)$
    • $(p_X-p_Y)\in ({\hat{p}}_X-{\hat{p}}_Y)\pm z_{\alpha /2}\sqrt{\frac{{\hat{p}}_X(1-{\hat{p}}_X)}{n_X}+\frac{{\hat{p}}_Y(1-{\hat{p}}_Y)}{n_Y}}$

Variation of Sidedness

$$\begin{array}{rlr}\theta \in \hat{\theta }\pm H& & \text{2 sided CI}\\ \theta <\hat{\theta }+H& & \text{1 sided lower CI}\\ \theta >\hat{\theta }-H& & \text{1 sided upper CI}\end{array}$$

Calculating Sample Size from Half Width

Example using 1 population, $\mu$, unknown ${\sigma }^2$

$$\begin{array}{rl}H& \le ϵ\\ t_{\alpha /2,(n-1)}\frac{S}{\sqrt{n}}& \le ϵ\\ \frac{t_{\alpha /2,(n-1)}S}{ϵ}& \le \sqrt{n}\\ {\left(\frac{t_{\alpha /2,(n-1)}S}{ϵ}\right)}^2& \le n\\ n& \ge {\left(\frac{t_{\alpha /2,(n-1)}S}{ϵ}\right)}^2\end{array}$$

Mean CI Variance Known

$$\begin{array}{c}\mu \in \overline{X}\pm z_{\alpha /2}\frac{\sigma }{\sqrt{n}}\end{array}$$

$H$ is called half-width

$$H\equiv z_{\alpha /2}\frac{\sigma }{\sqrt{n}}$$ $$\begin{array}{c}\begin{array}{ccc}{X}_{\mathrm{1..}n}{\sim }^{\text{iid}}N(\mu ,{\sigma }^2)& {\sigma }^2\text{ known}& \hat{\theta }=\overline{X}={\sum }_i{X}_i/n\end{array}\\ Z\equiv \frac{\overline{X}-\mu }{\sqrt{{\sigma }^2/n}}\sim N(0,1)\\ \begin{array}{lc}1-\alpha & =P(-z_{\alpha /2}\le Z\le z_{\alpha /2})\\ & =P(-z_{\alpha /2}\le \frac{\overline{X}-\mu }{\sqrt{{\sigma }^2/n}}\le z_{\alpha /2})\\ & =P(-z_{\alpha /2}\sqrt{{\sigma }^2/n}\le \overline{X}-\mu \le z_{\alpha /2}\sqrt{{\sigma }^2/n})\\ & =P\left(\underset{L}{\underbrace{\overline{X}-z_{\alpha /2}\sqrt{{\sigma }^2/n}}}\le \mu \le \underset{U}{\underbrace{\overline{X}+z_{\alpha /2}\sqrt{{\sigma }^2/n}}}\right)\end{array}\end{array}$$

In order to reduce half length $H$ below a tolerance error $ϵ$

$$H=z_{\alpha /2}\frac{\sigma }{\sqrt{n}}\le ϵ⟺n\ge (\sigma z_{\alpha /2}/ϵ{)}^2$$

Mean CI Variance Unknown

$$\begin{array}{c}\mu \in \overline{X}\pm t_{\alpha /2(n-1)}\frac{S}{\sqrt{n}}\\ H\equiv t_{\alpha /2(n-1)}\frac{S}{\sqrt{n}}\end{array}$$