Aim of this tutorial
Upon this exercise you can
- conduct a statistical hypothesis test for a two group comparison in R and to interpret the results.
- critically evaluate the assumptions for a two sample t-test
- use simulations to assist you when evaluating the assumptions of a two group comparison
- select the correct test based on the data exploration.
Background
Researchers wanted to study the immune response on pertussis. They have set up an experiments with 40 rats. 16 rats were infected with pertussis and 24 rats received a control treatment. Researchers measured the white blood cell concentration (WBC) in each rat (count per mm\(^3\)).
De data consists of two variables:
Import the dataset
Load the libraries
Data path:
https://raw.githubusercontent.com/statOmics/PSLSData/main/wbcon.csv
wbcon <- read_csv("https://raw.githubusercontent.com/statOmics/PSLSData/main/wbcon.csv")
## Rows: 40 Columns: 2
## ── Column specification ──────────────────────────────────────────────
## Delimiter: ","
## chr (1): trt
## dbl (1): WBC
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
## Rows: 40
## Columns: 2
## $ WBC <dbl> 10252, 10467, 10601, 10638, 10901, 11071, 11092, 11371, …
## $ trt <chr> "control", "control", "control", "control", "control", "…
Aim of the study
The overarching goal of this study was to assess if the white blood cell count changes upon pertussis infection. To this end, researchers randomized 40 rats to two treatments: A control treatment and a treatment in which the rat was infected with pertussis.
Data exploration
A crucial first step in a data analysis is to visualize and to explore the raw data.
This will allow us to gain insight in the data.
A secondary goal of the data exploration is to check the assumptions of the test that we will perform.
What are the assumptions of this test?
- The data are independent
- The data in both groups have the same variance
- The data in both groups are normally distributed.
Boxplots
We can assess the second assumption using boxplots.
wbcon %>% ggplot(aes(x = trt, y = WBC, fill = trt)) +
geom_boxplot(outlier.shape = NA) +
geom_point(position = "jitter") +
ylab("WBC (count/mm3)") +
xlab("treatment") +
stat_summary(
fun = mean, geom = "point",
shape = 5, size = 3, color = "black"
)
![](05_2_pertussis_sol_files/figure-html/unnamed-chunk-4-1.png)
What do you observe?
Both the mean and variance of the data seems to differ between control rats and rats infected with pertussis. So the second assumption is not valid. If the data are normally distributed we can compare the groups using a Welch modified t-test, which is valid if the data in both groups are exhibiting a different variance.
QQ-plots
To assess the assumption that the data are normally distributed in each treatment group, we will use QQ plots.
wbcon %>%
ggplot(aes(sample = WBC)) +
geom_qq() +
geom_qq_line() +
facet_grid(cols = vars(trt))
![](05_2_pertussis_sol_files/figure-html/unnamed-chunk-5-1.png)
What do you observe?
The white blood cell counts appear to be normally distributed in both treatment groups.
Assess the research question with the appropriate t-test
Analysis
output <- t.test(WBC ~ trt, data = wbcon, var.equal = FALSE)
output
##
## Welch Two Sample t-test
##
## data: WBC by trt
## t = -5.7114, df = 15.132, p-value = 3.984e-05
## alternative hypothesis: true difference in means between group control and group pertussis is not equal to 0
## 95 percent confidence interval:
## -32311.32 -14758.47
## sample estimates:
## mean in group control mean in group pertussis
## 12215.92 35750.81
Conclusion
The average white blood cell count is extremely different between rats that are infected with pertussis and rads that received a control treatment (p << 0.001).
The white blood cell count is on average 23535 blood cells/mm\(^3\) higher for rats with pertussis than for rats that received a control treatment (95% CI [14758, 32311]).
ADDENDUM: Train yourself in checking the assumptions
In order for the learners to get more proficient in evaluating the assumptions we will simulate 8 dataset with sample sizes similar to our data for which the assumptions of normality and equal variance do hold. For the QQ-plots we will only plot the one from one of the groups.
Simulate data
We simulate 8 datasets with the same sample sizes, means and pooled variance as in the sample.
set.seed(21532)
nSim <- 8
## descriptive statistics
wbcSum <- wbcon %>%
group_by(trt) %>%
summarize(
mean = mean(WBC, na.rm = TRUE),
sd = sd(WBC, na.rm = TRUE),
n = n()
) %>%
mutate(se = sd / sqrt(n))
sigma <- sqrt(sum(wbcSum$sd^2 * (wbcSum$n - 1)) / (sum(wbcSum$n) - 2))
normSim <- matrix(
c(
wbcon$WBC,
rnorm(sum(wbcSum$n) * nSim,
mean = c(
rep(wbcSum$mean[1], wbcSum$n[1]),
rep(wbcSum$mean[2], wbcSum$n[2])
),
sd = sigma)
),
nrow = sum(wbcSum$n)) %>%
as.data.frame() %>%
mutate(trt = wbcon$trt)
Comparisons of variances
names(normSim)[1:(nSim+1)] <- c("orig",paste0("sim",1:nSim))
normSim %>%
gather(samp, data, -trt) %>%
ggplot(aes(x = trt, y = data)) +
geom_boxplot() +
facet_wrap(~samp)
![](05_2_pertussis_sol_files/figure-html/unnamed-chunk-8-1.png)
We observe that the difference in variability of the original observations (top left plot) is larger than in the each simulations. Indicating that there is indeed evidence that the variances in each group differ.
Evaluation of normality
Control group
normSim %>%
gather(samp, data, -trt) %>%
filter(trt == "control") %>%
ggplot(aes(sample = data)) +
geom_qq() +
geom_qq_line() +
facet_wrap(~samp)
![](05_2_pertussis_sol_files/figure-html/unnamed-chunk-9-1.png)
Pertussis group
normSim %>%
gather(samp, data, -trt) %>%
filter(trt == "pertussis") %>%
ggplot(aes(sample = data)) +
geom_qq() +
geom_qq_line() +
facet_wrap(~samp)
![](05_2_pertussis_sol_files/figure-html/unnamed-chunk-10-1.png)
The data in each group does not show larger deviations from normality than what we observe in each of the eight datasets that were simulated under the model assumptions.
LS0tCnRpdGxlOiAiRXhlcmNpc2UgNS4yOiBIeXBvdGhlc2lzIHRlc3Rpbmcgb24gdGhlIHBlcnR1c3NpcyBleGFtcGxlIC0gc29sdXRpb24iCmF1dGhvcjogIkxpZXZlbiBDbGVtZW50LCBhbmQgTWlsYW4gTWFsZmFpdCIKZGF0ZTogInN0YXRPbWljcywgR2hlbnQgVW5pdmVyc2l0eSAoaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSIKLS0tCgojIEFpbSBvZiB0aGlzIHR1dG9yaWFsCgpVcG9uIHRoaXMgZXhlcmNpc2UgeW91IGNhbgoKMS4gY29uZHVjdCBhIHN0YXRpc3RpY2FsIGh5cG90aGVzaXMgdGVzdCBmb3IgYSB0d28gZ3JvdXAgY29tcGFyaXNvbiBpbiBSIGFuZCB0byBpbnRlcnByZXQgdGhlIHJlc3VsdHMuCjIuIGNyaXRpY2FsbHkgZXZhbHVhdGUgdGhlIGFzc3VtcHRpb25zIGZvciBhIHR3byBzYW1wbGUgdC10ZXN0CjMuIHVzZSBzaW11bGF0aW9ucyB0byBhc3Npc3QgeW91IHdoZW4gZXZhbHVhdGluZyB0aGUgYXNzdW1wdGlvbnMgb2YgYSB0d28gZ3JvdXAgY29tcGFyaXNvbgo0LiBzZWxlY3QgdGhlIGNvcnJlY3QgdGVzdCBiYXNlZCBvbiB0aGUgZGF0YSBleHBsb3JhdGlvbi4KCiMgQmFja2dyb3VuZAoKUmVzZWFyY2hlcnMgd2FudGVkIHRvIHN0dWR5IHRoZSBpbW11bmUgcmVzcG9uc2Ugb24gcGVydHVzc2lzLgpUaGV5IGhhdmUgc2V0IHVwIGFuIGV4cGVyaW1lbnRzIHdpdGggNDAgcmF0cy4KMTYgcmF0cyB3ZXJlIGluZmVjdGVkIHdpdGggcGVydHVzc2lzIGFuZCAyNCByYXRzIHJlY2VpdmVkIGEgY29udHJvbCB0cmVhdG1lbnQuClJlc2VhcmNoZXJzIG1lYXN1cmVkIHRoZSB3aGl0ZSBibG9vZCBjZWxsIGNvbmNlbnRyYXRpb24gKFdCQykgaW4gZWFjaCByYXQgKGNvdW50IHBlciBtbSReMyQpLgoKRGUgZGF0YSBjb25zaXN0cyBvZiB0d28gdmFyaWFibGVzOgoKLSBXQkM6IHdoaXRlIGJsb29kIGNlbGwgY291bnQgKGNvdW50cy9tbSReMyQpLgotIHRydDogdHJlYXRtZW50CgogICAgLSBjb250cm9sOiByYXQgcmVjaWV2ZWQgY29udHJvbCB0cmVhdG1lbnQKICAgIC0gcGVydHVzc2lzOiByYXQgd2FzIGluZmVjdGVkIHdpdGggcGVydHVzc2lzCgoKIyBJbXBvcnQgdGhlIGRhdGFzZXQKCkxvYWQgdGhlIGxpYnJhcmllcwoKYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CmxpYnJhcnkodGlkeXZlcnNlKQpgYGAKCkRhdGEgcGF0aDoKCiAgYGh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9zdGF0T21pY3MvUFNMU0RhdGEvbWFpbi93YmNvbi5jc3ZgCgoKYGBge3J9CndiY29uIDwtIHJlYWRfY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1BTTFNEYXRhL21haW4vd2Jjb24uY3N2IikKYGBgCgpgYGB7cn0KZ2xpbXBzZSh3YmNvbikKYGBgCgojIyBBaW0gb2YgdGhlIHN0dWR5CgpUaGUgb3ZlcmFyY2hpbmcgZ29hbCBvZiB0aGlzIHN0dWR5IHdhcyB0byBhc3Nlc3MgaWYgdGhlIHdoaXRlIGJsb29kIGNlbGwgY291bnQgY2hhbmdlcyB1cG9uIHBlcnR1c3NpcyBpbmZlY3Rpb24uIFRvIHRoaXMgZW5kLCByZXNlYXJjaGVycyByYW5kb21pemVkIDQwIHJhdHMKdG8gdHdvIHRyZWF0bWVudHM6IEEgY29udHJvbCB0cmVhdG1lbnQgYW5kIGEgdHJlYXRtZW50IGluIHdoaWNoIHRoZSByYXQgd2FzIGluZmVjdGVkIHdpdGggcGVydHVzc2lzLgoKIyBEYXRhIGV4cGxvcmF0aW9uCgpBIGNydWNpYWwgZmlyc3Qgc3RlcCBpbiBhIGRhdGEgYW5hbHlzaXMgaXMgdG8gdmlzdWFsaXplIGFuZCB0byBleHBsb3JlIHRoZSByYXcgZGF0YS4KClRoaXMgd2lsbCBhbGxvdyB1cyB0byBnYWluIGluc2lnaHQgaW4gdGhlIGRhdGEuCgpBIHNlY29uZGFyeSBnb2FsIG9mIHRoZSBkYXRhIGV4cGxvcmF0aW9uIGlzIHRvIGNoZWNrIHRoZSBhc3N1bXB0aW9ucyBvZiB0aGUgdGVzdCB0aGF0IHdlIHdpbGwgcGVyZm9ybS4KCiMjIFdoaWNoIHRlc3Qgd2lsbCB5b3UgcGVyZm9ybSB0byBjb21wYXJlIHRoZSB3aGl0ZSBibG9vZCBjZWxsIGNvdW50IGJldHdlZW4gaW5mZWN0ZWQgYW5kIGNvbnRyb2wgcmF0cz8KCjQwIGRpZmZlcmVudCByYXRzIGFyZSBhc3NpZ25lZCBhdCByYW5kb20gdG8gYSBjb250cm9sIHRyZWF0bWVudCBvciBhIHRyZWF0bWVudCB3aGVyZSB0aGV5IGFyZSBpbmZlY3RlZCB3aXRoIHBlcnR1c3Npcy4KVGhlIHdoaXRlIGJsb29kIGNlbGxzIG9mIHRoZSByYXRzIGFyZSBtZWFzdXJlZCBvbiBkaWZmZXJlbnQgYW5pbWFscyBhbmQgd2UgY2FuIGFzc3VtZSB0aGF0IHRoZSBkYXRhIGJldHdlZW4gcmF0cyBhcmUgaW5kZXBlbmRlbnQuCk5vdGUsIHRoYXQgdGhpcyB3aWxsIG5vdCBiZSB0aGUgY2FzZSBpZiBtdWx0aXBsZSByYXRzIGFyZSBob3VzZWQgaW4gdGhlIHNhbWUgY2FnZXMhCgpXZSB3aWxsIHRoZXJlZm9yZSBwZXJmb3JtIGEgdHdvLXNhbXBsZSB0LXRlc3QgdG8gYXNzZXNzIGlmIHRoZSBhdmVyYWdlIHdoaXRlIGJsb29kIGNlbGwgY291bnQgZGlmZmVycyBiZXR3ZWVuIGluZmVjdGVkIHJhdHMgYW5kIHJhdHMgdGhhdCByZWNlaXZlZCB0aGUgY29udHJvbCB0cmVhdG1lbnQuCgojIyBXaGF0IGFyZSB0aGUgYXNzdW1wdGlvbnMgb2YgdGhpcyB0ZXN0PwoKMS4gVGhlIGRhdGEgYXJlIGluZGVwZW5kZW50CjIuIFRoZSBkYXRhIGluIGJvdGggZ3JvdXBzIGhhdmUgdGhlIHNhbWUgdmFyaWFuY2UKMy4gVGhlIGRhdGEgaW4gYm90aCBncm91cHMgYXJlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkLgoKIyMgQm94cGxvdHMKCldlIGNhbiBhc3Nlc3MgdGhlIHNlY29uZCBhc3N1bXB0aW9uIHVzaW5nIGJveHBsb3RzLgoKYGBge3J9CndiY29uICU+JSBnZ3Bsb3QoYWVzKHggPSB0cnQsIHkgPSBXQkMsIGZpbGwgPSB0cnQpKSArCiAgZ2VvbV9ib3hwbG90KG91dGxpZXIuc2hhcGUgPSBOQSkgKwogIGdlb21fcG9pbnQocG9zaXRpb24gPSAiaml0dGVyIikgKwogIHlsYWIoIldCQyAoY291bnQvbW0zKSIpICsKICB4bGFiKCJ0cmVhdG1lbnQiKSArCiAgc3RhdF9zdW1tYXJ5KAogICAgZnVuID0gbWVhbiwgZ2VvbSA9ICJwb2ludCIsCiAgICBzaGFwZSA9IDUsIHNpemUgPSAzLCBjb2xvciA9ICJibGFjayIKICApCmBgYAoKV2hhdCBkbyB5b3Ugb2JzZXJ2ZT8KCkJvdGggdGhlIG1lYW4gYW5kIHZhcmlhbmNlIG9mIHRoZSBkYXRhIHNlZW1zIHRvIGRpZmZlciBiZXR3ZWVuIGNvbnRyb2wgcmF0cyBhbmQgcmF0cyBpbmZlY3RlZCB3aXRoIHBlcnR1c3Npcy4KU28gdGhlIHNlY29uZCBhc3N1bXB0aW9uIGlzIG5vdCB2YWxpZC4KSWYgdGhlIGRhdGEgYXJlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIHdlIGNhbiBjb21wYXJlIHRoZSBncm91cHMgdXNpbmcgYSBXZWxjaCBtb2RpZmllZCB0LXRlc3QsIHdoaWNoIGlzIHZhbGlkIGlmIHRoZSBkYXRhIGluIGJvdGggZ3JvdXBzIGFyZSBleGhpYml0aW5nIGEgZGlmZmVyZW50IHZhcmlhbmNlLgoKIyMgUVEtcGxvdHMKClRvIGFzc2VzcyB0aGUgYXNzdW1wdGlvbiB0aGF0IHRoZSBkYXRhIGFyZSBub3JtYWxseSBkaXN0cmlidXRlZCBpbiBlYWNoIHRyZWF0bWVudCBncm91cCwgd2Ugd2lsbCB1c2UgUVEgcGxvdHMuCgpgYGB7cn0Kd2Jjb24gJT4lCiAgZ2dwbG90KGFlcyhzYW1wbGUgPSBXQkMpKSArCiAgZ2VvbV9xcSgpICsKICBnZW9tX3FxX2xpbmUoKSArCiAgZmFjZXRfZ3JpZChjb2xzID0gdmFycyh0cnQpKQpgYGAKCldoYXQgZG8geW91IG9ic2VydmU/CgpUaGUgd2hpdGUgYmxvb2QgY2VsbCBjb3VudHMgYXBwZWFyIHRvIGJlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkIGluIGJvdGggdHJlYXRtZW50IGdyb3Vwcy4KCgojIEFzc2VzcyB0aGUgcmVzZWFyY2ggcXVlc3Rpb24gd2l0aCB0aGUgYXBwcm9wcmlhdGUgdC10ZXN0CgojIyBBbmFseXNpcwoKYGBge3J9Cm91dHB1dCA8LSB0LnRlc3QoV0JDIH4gdHJ0LCBkYXRhID0gd2Jjb24sIHZhci5lcXVhbCA9IEZBTFNFKQpvdXRwdXQKYGBgCgojIyBDb25jbHVzaW9uCgpUaGUgYXZlcmFnZSB3aGl0ZSBibG9vZCBjZWxsIGNvdW50IGlzIGV4dHJlbWVseSBkaWZmZXJlbnQgYmV0d2VlbiByYXRzIHRoYXQgYXJlIGluZmVjdGVkIHdpdGggcGVydHVzc2lzIGFuZCByYWRzIHRoYXQgcmVjZWl2ZWQgYSBjb250cm9sIHRyZWF0bWVudCAgKHAgPDwgMC4wMDEpLgoKVGhlIHdoaXRlIGJsb29kIGNlbGwgY291bnQgaXMgb24gYXZlcmFnZSBgciBvdXRwdXQkZXN0aW1hdGUgJT4lIGRpZmYgJT4lIGFicyAlPiUgZm9ybWF0KC4sZGlnaXRzID0gNSlgIGJsb29kIGNlbGxzL21tJF4zJCBoaWdoZXIgZm9yIHJhdHMgd2l0aCBwZXJ0dXNzaXMgdGhhbiBmb3IgcmF0cyB0aGF0IHJlY2VpdmVkIGEgY29udHJvbCB0cmVhdG1lbnQgKDk1JSBDSSBbYHIgb3V0cHV0JGNvbmYuaW50ICU+JSBhYnMgJT4lIHNvcnQgJT4lIGZvcm1hdCguLGRpZ2l0cyA9IDUpYF0pLgoKCiMgQURERU5EVU06IFRyYWluIHlvdXJzZWxmIGluIGNoZWNraW5nIHRoZSBhc3N1bXB0aW9ucwoKSW4gb3JkZXIgZm9yIHRoZSBsZWFybmVycyB0byBnZXQgbW9yZSBwcm9maWNpZW50IGluIGV2YWx1YXRpbmcgdGhlIGFzc3VtcHRpb25zIHdlIHdpbGwgc2ltdWxhdGUgOCBkYXRhc2V0IHdpdGggc2FtcGxlIHNpemVzIHNpbWlsYXIgdG8gb3VyIGRhdGEgZm9yIHdoaWNoIHRoZSBhc3N1bXB0aW9ucyBvZiBub3JtYWxpdHkgYW5kIGVxdWFsIHZhcmlhbmNlIGRvIGhvbGQuIEZvciB0aGUgUVEtcGxvdHMgd2Ugd2lsbCBvbmx5IHBsb3QgdGhlIG9uZSBmcm9tIG9uZSBvZiB0aGUgZ3JvdXBzLgoKIyMgU2ltdWxhdGUgZGF0YQoKV2Ugc2ltdWxhdGUgOCBkYXRhc2V0cyB3aXRoIHRoZSBzYW1lIHNhbXBsZSBzaXplcywgbWVhbnMgYW5kIHBvb2xlZCB2YXJpYW5jZSBhcyBpbiB0aGUgc2FtcGxlLgoKYGBge3J9CnNldC5zZWVkKDIxNTMyKQpuU2ltIDwtIDgKIyMgZGVzY3JpcHRpdmUgc3RhdGlzdGljcwp3YmNTdW0gPC0gd2Jjb24gJT4lCiAgZ3JvdXBfYnkodHJ0KSAlPiUKICBzdW1tYXJpemUoCiAgICAgIG1lYW4gPSBtZWFuKFdCQywgbmEucm0gPSBUUlVFKSwKICAgICAgc2QgPSBzZChXQkMsIG5hLnJtID0gVFJVRSksCiAgICAgIG4gPSBuKCkKICApICU+JQogIG11dGF0ZShzZSA9IHNkIC8gc3FydChuKSkKCnNpZ21hIDwtIHNxcnQoc3VtKHdiY1N1bSRzZF4yICogKHdiY1N1bSRuIC0gMSkpIC8gKHN1bSh3YmNTdW0kbikgLSAyKSkKCm5vcm1TaW0gPC0gbWF0cml4KAogIGMoCiAgICB3YmNvbiRXQkMsCiAgICBybm9ybShzdW0od2JjU3VtJG4pICogblNpbSwKICAgICAgICAgIG1lYW4gPSBjKAogICAgICAgICAgICByZXAod2JjU3VtJG1lYW5bMV0sIHdiY1N1bSRuWzFdKSwKICAgICAgICAgICAgcmVwKHdiY1N1bSRtZWFuWzJdLCB3YmNTdW0kblsyXSkKICAgICAgICAgICAgKSwKICAgICAgICAgIHNkID0gc2lnbWEpCiAgICApLAogIG5yb3cgPSBzdW0od2JjU3VtJG4pKSAlPiUKICBhcy5kYXRhLmZyYW1lKCkgJT4lCiAgbXV0YXRlKHRydCA9IHdiY29uJHRydCkKYGBgCgojIyBDb21wYXJpc29ucyBvZiB2YXJpYW5jZXMKCmBgYHtyfQpuYW1lcyhub3JtU2ltKVsxOihuU2ltKzEpXSA8LSBjKCJvcmlnIixwYXN0ZTAoInNpbSIsMTpuU2ltKSkKbm9ybVNpbSAlPiUKICBnYXRoZXIoc2FtcCwgZGF0YSwgLXRydCkgJT4lCiAgZ2dwbG90KGFlcyh4ID0gdHJ0LCB5ID0gZGF0YSkpICsKICBnZW9tX2JveHBsb3QoKSArCiAgZmFjZXRfd3JhcCh+c2FtcCkKYGBgCgpXZSBvYnNlcnZlIHRoYXQgdGhlIGRpZmZlcmVuY2UgaW4gdmFyaWFiaWxpdHkgb2YgdGhlIG9yaWdpbmFsIG9ic2VydmF0aW9ucyAodG9wIGxlZnQgcGxvdCkgaXMgbGFyZ2VyIHRoYW4gaW4gdGhlIGVhY2ggc2ltdWxhdGlvbnMuIEluZGljYXRpbmcgdGhhdCB0aGVyZSBpcyBpbmRlZWQgZXZpZGVuY2UgdGhhdCB0aGUgdmFyaWFuY2VzIGluIGVhY2ggZ3JvdXAgZGlmZmVyLgoKIyMgRXZhbHVhdGlvbiBvZiBub3JtYWxpdHkKCiMjIyBDb250cm9sIGdyb3VwCgpgYGB7cn0Kbm9ybVNpbSAlPiUKICBnYXRoZXIoc2FtcCwgZGF0YSwgLXRydCkgJT4lCiAgZmlsdGVyKHRydCA9PSAiY29udHJvbCIpICU+JQogIGdncGxvdChhZXMoc2FtcGxlID0gZGF0YSkpICsKICBnZW9tX3FxKCkgKwogIGdlb21fcXFfbGluZSgpICsKICBmYWNldF93cmFwKH5zYW1wKQpgYGAKCiMjIyBQZXJ0dXNzaXMgZ3JvdXAKCmBgYHtyfQpub3JtU2ltICU+JQogIGdhdGhlcihzYW1wLCBkYXRhLCAtdHJ0KSAlPiUKICBmaWx0ZXIodHJ0ID09ICJwZXJ0dXNzaXMiKSAlPiUKICBnZ3Bsb3QoYWVzKHNhbXBsZSA9IGRhdGEpKSArCiAgZ2VvbV9xcSgpICsKICBnZW9tX3FxX2xpbmUoKSArCiAgZmFjZXRfd3JhcCh+c2FtcCkKYGBgCgpUaGUgZGF0YSBpbiBlYWNoIGdyb3VwIGRvZXMgbm90IHNob3cgbGFyZ2VyIGRldmlhdGlvbnMgZnJvbSBub3JtYWxpdHkgdGhhbiB3aGF0IHdlIG9ic2VydmUgaW4gZWFjaCBvZiB0aGUgZWlnaHQgZGF0YXNldHMgdGhhdCB3ZXJlIHNpbXVsYXRlZCB1bmRlciB0aGUgbW9kZWwgYXNzdW1wdGlvbnMuCg==