21. Grad Homework Three: The Z-test and Hagelloch Measles Epidemic

In the paper linked here (https://www.sciencedirect.com/science/article/pii/S0163445322000743#bib0014), the authors write that

Hagelloch was an isolated village with 577 inhabitants in 1861. When the first cases of measles from Hagelloch were detected at the hospital in Tübingen in November, an investigation was initiated. From November 29 to December 18, the village was visited every day by a young physician, Albert Pfeilsticker, in order to examine and treat all cases of measles. With a local assistant, houses were numbered, and a complete enumeration of the population carried out.

Age and residence house number was registered for all children. Twelve of 56 houses were occupied by several families, but for all children in these multi-household houses it was indicated to which household the child belonged. Pfeilsticker’s study focused on incubation period, prodromal period, infectiousness, and complications during the different phases of measles infection. Clinical information reported in the study was date of onset of prodromes (symptoms), date of rash, fever, complications, and survival. Information was collected retrospectively for the few children who had measles before the medical investigation was initiated. Sex was not reported by Pfeilsticker,14 except for a few severe cases. However, through parish registers, it has been possible to determine sex of 177 of the 188 children infected during the epidemic.

One girl aged 4 years who had prodromes on January 24, 1862 and rash on January 27, 1862 is not classified as a secondary case; measles virus infection had started on December 5 and 6, 1861, in her sister aged 5 and her brother aged 10. It is therefore unclear from where this girl was infected since all the other cases occurred in 1861. We have excluded this case and focused on the 187 measles cases with onset in 1861.

The dataset that was collected in 1861 has been formatted as a CSV here https://skgallagher.github.io/EpiCompare/reference/hagelloch_raw.html? and there is a link to the dataset provided on course site.

The two-sample Z-test (with known standard deviation)

Lets suppose that we wanted to compare the proportion of females who experienced death due to measles to the proportion of males. We might decide to turn to a familar topic called: null hypothesis significance testing (sometimes abbreivated NHST).

Our null hypothesis (and alternative hypothesis) might supopse that the probability of death due to measles is equal (unequal) in females and in males, or

(21.1)\[\begin{align} H_{\text{null}}: p_{\text{female}} - p_{\text{male}} &= 0 \\ H_{\text{alte}}: p_{\text{female}} - p_{\text{male}} &\neq 0 \end{align}\]

Most individuals would look at the above and think nothing of this statement. Except, the possible values for \(p_{\text{female}}\) are an infinite number between 0 and one. Same for p_{\text{male}}, and so the same for their difference. If we used a continuous random variable to describe this difference then we know the probability of any single value is zero.

Question 1: Whats going on with the above hypothesis? i just want to hear your thoughts. Below i discuss and we work through why. DONT READ AHEAD! i want to hear how you approach this problem and how you think about hypothesis testing. This question is a 3 if you complete it and 0 if you skip it.

A good statistic to help provide information about the above hypotheses.

A good estimator for the probability of an event is the proportion of times this event has occurred. Let \(n_{f}\) equal the number of observed females and \(n_{m}\) males in the dataset above. Let \(f_{i}\) equal the value one if the \(i^{\text{th}}\) female died due to measles and zero otherwise. Let \(m_{j}\) equal the value one if the \(j^{\text{th}}\) male died due to measles and zero otherwise

Then we can compute the statistic

(21.2)\[\begin{align} S = \frac{1}{n_{f}} \sum^{n_{f}}_{i=1} f_{i} - \frac{1}{n_{m}}\sum^{n_{m}}_{j=1} m_{j} \end{align}\]

Question 2 If the observations are independent (reasonable to assume) then what does the Central Limit Theorem say about the distribution of

\( \frac{1}{n_{f}} \sum^{n_{f}}_{i=1} f_{i} \) ?

Question 3 If the observations are independent (reasonable to assume) then what does the Central Limit Theorem say about the distribution of

\( \frac{1}{n_{m}} \sum^{n_{m}}_{j=1} m_{j} \) ?

Question 4 Assume the above two distribtuions are represented by the random variables \(X\) (for females) and \(Y\) (for males). Then what is the distribution of the new random variables \(Z = X-Y\)?

Ok i can help with Question 1. For hypothesis testing, we make statements about parameters and potential values that they can take. This includes also any algebraic statement of paramters like their difference or one parameter divided by another etc. Though the hypothesis is a statement about parameters, the probability (or uncertainity) statement is that: given the null hypothesis is true then what is the probability of observing the sample that i collected/observed?

This is why you can provide evidence for or against the above null hypothesis. The evidence, and the probability, are contained in the sample.

Z test for Measles

Question 5 Please compute the following test statistic for our measles data where we assume that the null hypotheis is true.

(21.3)\[\begin{align} T = \frac{ \bar{f} - \bar{m} }{ \text{SE}_{\text{H}_{\text{null}}}( \bar{f} - \bar{m} ) } \end{align}\]

(21.4)\[\begin{align} \text{SE}_{\text{H}_{\text{null}}}( \bar{f} - \bar{m} ) = \sqrt{ \frac{p(1-p)}{ n_{f}} + \frac{p(1-p)}{n_{m}} } \end{align}\]

where \(p\) is the unkown, but assumed to be equal based on our null hypothesis proportion of deaths due to measles. We can estimate \(p\) as \(\frac{ \sum f_{i} + \sum m_{j} }{ n_{f} + n_{m} }\)

Question 6 Ok, i dont get it. The test statistic above is supposed to assume that the null hypothesis is true: that the two proportions of deaths due to measles are equal. In the denominator we assume that the probabilities are equal but i dont see that happening in the numerator? Is this test statistic measuring “surprise” or “difference” from the expected null distribution? What does a big versus small T value indicate? Discuss.

p value

Question 7 Please compute the pvalue for the above Z-test. That is, please compute the probability that we would observe the above T statistic or values more extreme than this T statistic: under the assumption the null distribution of statistics we would have expected to compute has the following distribution \(\mathcal{N}(0,1)\), where \(\mathcal{N}(\mu,\sigma^{2})\) is a Normal density with expected value \(\mu\) and variance \(\sigma^{2}\).

Question 8 What does it mean that the null distribution is \(\mathcal{N}(0,1)\)? The null distribution of what? Discuss.