In this short tutorial, we perform a hypothesis test on the “cuckoo” dataset.

1 The cuckoo dataset

The common cuckoo does not build its own nest: it prefers to lay its eggs in another birds’ nest. It is known, since 1892, that the type of cuckoo bird eggs are different between different locations. In a study from 1940, it was shown that cuckoos return to the same nesting area each year, and that they always pick the same bird species to be a “foster parent” for their eggs.

Over the years, this has lead to the development of geographically determined subspecies of cuckoos. These subspecies have evolved in such a way that their eggs look as similar as possible as those of their foster parents.

The cuckoo dataset contains information on 120 Cuckoo eggs, obtained from randomly selected “foster” nests. For these eggs, researchers have measured the length (in mm) and established the type (species) of foster parent. The type column is coded as follows:

  • type=1: Meadow pipit
  • type=2: Tree pipit
  • type=3: Dunnock
  • type=4: European robin
  • type=5: White wagtail
  • type=6: Eurasian wren

2 Goal

The researchers want totest if the type of foster parent has an effect on the average length of the cuckoo eggs.

In theory, they want to study this for all six species. Previously, we looked at a single pairwise comparison between the European robin and the Eurasian wren with a t-test. Here, we will analyse all types simultaneously with ANOVA.

Load the required libraries

library(tidyverse)

2.1 Import the data

Cuckoo <- read_tsv("https://raw.githubusercontent.com/GTPB/PSLS20/master/data/Cuckoo.txt")
head(Cuckoo)
## # A tibble: 6 x 2
##   length  type
##    <dbl> <dbl>
## 1   21.8     1
## 2   21.7     1
## 3   21.7     1
## 4   24.0     1
## 5   22.4     1
## 6   23.0     1

3 Data tidying

It seems that the tpye column is a double rather than a factor. Let’s fix this:

Cuckoo <- Cuckoo %>%
  mutate(type = as.factor(type))

4 Data exploration

How many birds do we have for each type?

Cuckoo %>%
  count(type)
## # A tibble: 6 x 2
##   type      n
##   <fct> <int>
## 1 1        45
## 2 2        15
## 3 3        14
## 4 4        16
## 5 5        15
## 6 6        15

Visualize the data

Cuckoo %>%
  ggplot(aes(x=type,y=length,fill=type)) +
  theme_bw() +
  scale_fill_brewer(palette="RdGy") +
  geom_boxplot() +
  geom_jitter(width = 0.2) +
  ggtitle("Boxplot of the length of eggs per type") +
  ylab("length (mm)") + 
  stat_summary(fun.y=mean, geom="point", shape=5, size=3, color="black", fill="black")

4.1 Check the assumptions for ANOVA

To study whether or not the observed differences in average egg length between the different foster bird types are significant, we may perform an ANOVA.

The null hypothesis of ANOVA states that: \(H0\): The mean egg length is equal between the different bird tpyes.

The alternative hypothesis of ANOVA states that: \(HA\): The mean egg length for at least one bird type is different from the mean egg length in at least one other bird type.

Before we may proceed with the analysis, we must make sure that all assumptions for ANOVA are met. ANOVA has three assumptions:

  1. The observations are independent of each other (in all groups)
  2. The data (length) must be normally distributed (in all groups)
  3. The variability within all groups is similar

The first assumption is met, as we may assume that there are no specific patterns of correlation between the randomly selected nests.

To check the normality assumption, we will use QQ plots.

Cuckoo %>% 
  ggplot(aes(sample=length)) +
  geom_qq() +
  geom_qq_line() + 
  facet_grid(~type)

There seem to be no clear deviations from normality.

The third assumption of equal variances seems to be met based on the visualization with the boxplots (see above).

As such, we may proceed with the ANOVA analysis.

5 Data analysis

5.1 ANOVA

fit <- lm(length~type, Cuckoo)
fit_anova <- anova(fit)
fit_anova
## Analysis of Variance Table
## 
## Response: length
##            Df  Sum Sq Mean Sq F value    Pr(>F)    
## type        5  41.161  8.2322  4.7467 0.0005621 ***
## Residuals 114 197.710  1.7343                      
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The p-value of the ANOVA analysis is extremely significant (p-value = 0.0005621), so we reject the null hypothesis that the mean egg length is equal between the different bird types. We can say that the mean egg length is significantly different between at least two bird types on the 5% significance level.

Based on this analysis, we do not yet know between which particular bird types there is a significant difference. To study this, we will perfrom the Tuckey post-hoc analysis.

5.2 Post-hoc analysis

We will perform a post-hoc analysis, to look at the difference in egg length between each pairwise comparison of bird types. Importantly, with this strategy, the p-values will be correctly adjusted for multiple testing.

The null hypothesis for each pairwise test is that there is no difference in the mean egg length between both bird types.

The alternative hypothesis for each pairwise test states that there is indeed a difference in the mean egg length between both bird types.

We will also calculate the confidence interval on the mean differences.

library(multcomp, quietly = TRUE)
mcp <- glht(fit, linfct = mcp(type = "Tukey"))
summary(mcp)
## 
##   Simultaneous Tests for General Linear Hypotheses
## 
## Multiple Comparisons of Means: Tukey Contrasts
## 
## 
## Fit: lm(formula = length ~ type, data = Cuckoo)
## 
## Linear Hypotheses:
##            Estimate Std. Error t value Pr(>|t|)   
## 2 - 1 == 0  0.59000    0.39263   1.503  0.65690   
## 3 - 1 == 0  0.62143    0.40301   1.542  0.63151   
## 4 - 1 == 0  0.07500    0.38332   0.196  0.99996   
## 5 - 1 == 0  0.40333    0.39263   1.027  0.90518   
## 6 - 1 == 0 -1.37000    0.39263  -3.489  0.00865 **
## 3 - 2 == 0  0.03143    0.48939   0.064  1.00000   
## 4 - 2 == 0 -0.51500    0.47330  -1.088  0.88201   
## 5 - 2 == 0 -0.18667    0.48087  -0.388  0.99878   
## 6 - 2 == 0 -1.96000    0.48087  -4.076  0.00115 **
## 4 - 3 == 0 -0.54643    0.48195  -1.134  0.86268   
## 5 - 3 == 0 -0.21810    0.48939  -0.446  0.99764   
## 6 - 3 == 0 -1.99143    0.48939  -4.069  0.00116 **
## 5 - 4 == 0  0.32833    0.47330   0.694  0.98170   
## 6 - 4 == 0 -1.44500    0.47330  -3.053  0.03188 * 
## 6 - 5 == 0 -1.77333    0.48087  -3.688  0.00446 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## (Adjusted p values reported -- single-step method)
confint(mcp)
## 
##   Simultaneous Confidence Intervals
## 
## Multiple Comparisons of Means: Tukey Contrasts
## 
## 
## Fit: lm(formula = length ~ type, data = Cuckoo)
## 
## Quantile = 2.8888
## 95% family-wise confidence level
##  
## 
## Linear Hypotheses:
##            Estimate lwr      upr     
## 2 - 1 == 0  0.59000 -0.54424  1.72424
## 3 - 1 == 0  0.62143 -0.54279  1.78565
## 4 - 1 == 0  0.07500 -1.03234  1.18234
## 5 - 1 == 0  0.40333 -0.73091  1.53757
## 6 - 1 == 0 -1.37000 -2.50424 -0.23576
## 3 - 2 == 0  0.03143 -1.38231  1.44517
## 4 - 2 == 0 -0.51500 -1.88227  0.85227
## 5 - 2 == 0 -0.18667 -1.57582  1.20249
## 6 - 2 == 0 -1.96000 -3.34915 -0.57085
## 4 - 3 == 0 -0.54643 -1.93868  0.84582
## 5 - 3 == 0 -0.21810 -1.63184  1.19565
## 6 - 3 == 0 -1.99143 -3.40517 -0.57769
## 5 - 4 == 0  0.32833 -1.03894  1.69561
## 6 - 4 == 0 -1.44500 -2.81227 -0.07773
## 6 - 5 == 0 -1.77333 -3.16249 -0.38418

6 Conclusion

We have found an extremely significant dependence (p-value = 0.0005621) between the mean egg length and bird type on the global 5% significance level.

The mean length of cuckoo’s eggs in nests of the Eurasian wern are smaller as compared to those from all other bird types in the dataset:

  • the meadow pipit (adjusted p-value = 0.00856, mean difference = -1.37 mm, 95% CI [-2.50323; -0.23677])
  • the tree pipit (adjusted p-value = 0.00112, mean difference = -1.96 mm, 95% CI [-3.34792; -0.57208])
  • the dunnock (adjusted p-value = 0.00115, mean difference = -1.99 mm, 95% CI [-3.40513; -0.57773])
  • the European robin (adjusted p-value = 0.03184, mean difference = -1.45 mm, 95% CI [-2.81223; -0.07777])
  • the white wagtail (adjusted p-value = 0.03184, mean difference = -1.77 mm, 95% CI [-3.16244; -0.38422])

For the other bird types, we have insufficient evidence to suggest differences in mean length of the cuckoo bird’s eggs.

LS0tCnRpdGxlOiAnVHV0b3JpYWwgNy4zOiBBTk9WQSBvbiB0aGUgY3Vja29vIGRhdGFzZXQnCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgY29kZV9kb3dubG9hZDogeWVzCiAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICBudW1iZXJfc2VjdGlvbnM6IHllcwogICAgdGhlbWU6IGNvc21vCiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKICBwZGZfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwotLS0KCkluIHRoaXMgc2hvcnQgdHV0b3JpYWwsIHdlIHBlcmZvcm0gYSBoeXBvdGhlc2lzIHRlc3Qgb24gdGhlCiJjdWNrb28iIGRhdGFzZXQuIAoKIyBUaGUgY3Vja29vIGRhdGFzZXQKClRoZSBjb21tb24gY3Vja29vIGRvZXMgbm90IGJ1aWxkIGl0cyBvd24gbmVzdDogaXQgcHJlZmVycwp0byBsYXkgaXRzIGVnZ3MgaW4gYW5vdGhlciBiaXJkcycgbmVzdC4gSXQgaXMga25vd24sIHNpbmNlIDE4OTIsCnRoYXQgdGhlIHR5cGUgb2YgY3Vja29vIGJpcmQgZWdncyBhcmUgZGlmZmVyZW50IGJldHdlZW4gZGlmZmVyZW50CmxvY2F0aW9ucy4gSW4gYSBzdHVkeSBmcm9tIDE5NDAsIGl0IHdhcyBzaG93biB0aGF0IGN1Y2tvb3MgcmV0dXJuCnRvIHRoZSBzYW1lIG5lc3RpbmcgYXJlYSBlYWNoIHllYXIsIGFuZCB0aGF0IHRoZXkgYWx3YXlzIHBpY2sKdGhlIHNhbWUgYmlyZCBzcGVjaWVzIHRvIGJlIGEgImZvc3RlciBwYXJlbnQiIGZvciB0aGVpciBlZ2dzLgoKT3ZlciB0aGUgeWVhcnMsIHRoaXMgaGFzIGxlYWQgdG8gdGhlIGRldmVsb3BtZW50IG9mIGdlb2dyYXBoaWNhbGx5CmRldGVybWluZWQgc3Vic3BlY2llcyBvZiBjdWNrb29zLiBUaGVzZSBzdWJzcGVjaWVzIGhhdmUgZXZvbHZlZCBpbgpzdWNoIGEgd2F5IHRoYXQgdGhlaXIgZWdncyBsb29rIGFzIHNpbWlsYXIgYXMgcG9zc2libGUgYXMgdGhvc2UKb2YgdGhlaXIgZm9zdGVyIHBhcmVudHMuCgpUaGUgY3Vja29vIGRhdGFzZXQgY29udGFpbnMgaW5mb3JtYXRpb24gb24gMTIwIEN1Y2tvbyBlZ2dzLApvYnRhaW5lZCBmcm9tIHJhbmRvbWx5IHNlbGVjdGVkICJmb3N0ZXIiIG5lc3RzLgpGb3IgdGhlc2UgZWdncywgcmVzZWFyY2hlcnMgaGF2ZSBtZWFzdXJlZCB0aGUgYGxlbmd0aGAgKGluIG1tKQphbmQgZXN0YWJsaXNoZWQgdGhlIGB0eXBlYCAoc3BlY2llcykgb2YgZm9zdGVyIHBhcmVudC4KVGhlIHR5cGUgY29sdW1uIGlzIGNvZGVkIGFzIGZvbGxvd3M6CgotIGB0eXBlPTFgOiBNZWFkb3cgcGlwaXQKLSBgdHlwZT0yYDogVHJlZSBwaXBpdAotIGB0eXBlPTNgOiBEdW5ub2NrCi0gYHR5cGU9NGA6IEV1cm9wZWFuIHJvYmluCi0gYHR5cGU9NWA6IFdoaXRlIHdhZ3RhaWwKLSBgdHlwZT02YDogRXVyYXNpYW4gd3JlbgoKIyBHb2FsCgpUaGUgcmVzZWFyY2hlcnMgd2FudCB0b3Rlc3QgaWYgdGhlIHR5cGUgb2YgZm9zdGVyIHBhcmVudApoYXMgYW4gZWZmZWN0IG9uIHRoZSBhdmVyYWdlIGxlbmd0aCBvZiB0aGUgY3Vja29vIGVnZ3MuCgpJbiB0aGVvcnksIHRoZXkgd2FudCB0byBzdHVkeSB0aGlzIGZvciBhbGwgc2l4IHNwZWNpZXMuClByZXZpb3VzbHksIHdlIGxvb2tlZCBhdCBhIHNpbmdsZSBwYWlyd2lzZSBjb21wYXJpc29uCmJldHdlZW4gdGhlIEV1cm9wZWFuIHJvYmluIGFuZCB0aGUgRXVyYXNpYW4gd3JlbiB3aXRoIGEKdC10ZXN0LiBIZXJlLCB3ZSB3aWxsIGFuYWx5c2UgYWxsIHR5cGVzIHNpbXVsdGFuZW91c2x5CndpdGggQU5PVkEuCgpMb2FkIHRoZSByZXF1aXJlZCBsaWJyYXJpZXMKCmBgYHtyLCBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKYGBgCgojIyBJbXBvcnQgdGhlIGRhdGEKCmBgYHtyLCBtZXNzYWdlPUZBTFNFfQpDdWNrb28gPC0gcmVhZF90c3YoImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9HVFBCL1BTTFMyMC9tYXN0ZXIvZGF0YS9DdWNrb28udHh0IikKaGVhZChDdWNrb28pCmBgYAoKIyBEYXRhIHRpZHlpbmcKCkl0IHNlZW1zIHRoYXQgdGhlIGB0cHllYCBjb2x1bW4gaXMgYSBgZG91YmxlYCByYXRoZXIgdGhhbiBhIGBmYWN0b3JgLiAKTGV0J3MgZml4IHRoaXM6CgpgYGB7cn0KQ3Vja29vIDwtIEN1Y2tvbyAlPiUKICBtdXRhdGUodHlwZSA9IGFzLmZhY3Rvcih0eXBlKSkKYGBgCgoKIyBEYXRhIGV4cGxvcmF0aW9uCgpIb3cgbWFueSBiaXJkcyBkbyB3ZSBoYXZlIGZvciBlYWNoIHR5cGU/CgpgYGB7cn0KQ3Vja29vICU+JQogIGNvdW50KHR5cGUpCmBgYAoKVmlzdWFsaXplIHRoZSBkYXRhCgpgYGB7cn0KQ3Vja29vICU+JQogIGdncGxvdChhZXMoeD10eXBlLHk9bGVuZ3RoLGZpbGw9dHlwZSkpICsKICB0aGVtZV9idygpICsKICBzY2FsZV9maWxsX2JyZXdlcihwYWxldHRlPSJSZEd5IikgKwogIGdlb21fYm94cGxvdCgpICsKICBnZW9tX2ppdHRlcih3aWR0aCA9IDAuMikgKwogIGdndGl0bGUoIkJveHBsb3Qgb2YgdGhlIGxlbmd0aCBvZiBlZ2dzIHBlciB0eXBlIikgKwogIHlsYWIoImxlbmd0aCAobW0pIikgKyAKICBzdGF0X3N1bW1hcnkoZnVuLnk9bWVhbiwgZ2VvbT0icG9pbnQiLCBzaGFwZT01LCBzaXplPTMsIGNvbG9yPSJibGFjayIsIGZpbGw9ImJsYWNrIikKYGBgCgojIyBDaGVjayB0aGUgYXNzdW1wdGlvbnMgZm9yIEFOT1ZBCgpUbyBzdHVkeSB3aGV0aGVyIG9yIG5vdCB0aGUgb2JzZXJ2ZWQgZGlmZmVyZW5jZXMgaW4gYXZlcmFnZSBlZ2cgbGVuZ3RoIApiZXR3ZWVuIHRoZSBkaWZmZXJlbnQgZm9zdGVyIGJpcmQgdHlwZXMgYXJlIHNpZ25pZmljYW50LCB3ZSBtYXkgcGVyZm9ybSAKYW4gQU5PVkEuCgpUaGUgbnVsbCBoeXBvdGhlc2lzIG9mIEFOT1ZBIHN0YXRlcyB0aGF0OgokSDAkOiBUaGUgbWVhbiBlZ2cgbGVuZ3RoIGlzIGVxdWFsIGJldHdlZW4gdGhlIGRpZmZlcmVudCBiaXJkIHRweWVzLgoKVGhlIGFsdGVybmF0aXZlIGh5cG90aGVzaXMgb2YgQU5PVkEgc3RhdGVzIHRoYXQ6CiRIQSQ6IFRoZSBtZWFuIGVnZyBsZW5ndGggZm9yIGF0IGxlYXN0IG9uZSBiaXJkIHR5cGUgaXMgZGlmZmVyZW50IApmcm9tIHRoZSBtZWFuIGVnZyBsZW5ndGggaW4gYXQgbGVhc3Qgb25lIG90aGVyIGJpcmQgdHlwZS4KCkJlZm9yZSB3ZSBtYXkgcHJvY2VlZCB3aXRoIHRoZSBhbmFseXNpcywgd2UgbXVzdCBtYWtlIHN1cmUgdGhhdCBhbGwKYXNzdW1wdGlvbnMgZm9yIEFOT1ZBIGFyZSBtZXQuIEFOT1ZBIGhhcyB0aHJlZSBhc3N1bXB0aW9uczoKCjEuIFRoZSBvYnNlcnZhdGlvbnMgYXJlIGluZGVwZW5kZW50IG9mIGVhY2ggb3RoZXIgKGluIGFsbCBncm91cHMpCjIuIFRoZSBkYXRhIChsZW5ndGgpIG11c3QgYmUgbm9ybWFsbHkgZGlzdHJpYnV0ZWQgKGluIGFsbCBncm91cHMpCjMuIFRoZSB2YXJpYWJpbGl0eSB3aXRoaW4gYWxsIGdyb3VwcyBpcyBzaW1pbGFyCgpUaGUgZmlyc3QgYXNzdW1wdGlvbiBpcyBtZXQsIGFzIHdlIG1heSBhc3N1bWUgdGhhdCB0aGVyZSBhcmUgbm8Kc3BlY2lmaWMgcGF0dGVybnMgb2YgY29ycmVsYXRpb24gYmV0d2VlbiB0aGUgcmFuZG9tbHkgc2VsZWN0ZWQgbmVzdHMuCgpUbyBjaGVjayB0aGUgbm9ybWFsaXR5IGFzc3VtcHRpb24sIHdlIHdpbGwgdXNlIFFRIHBsb3RzLgoKYGBge3J9CkN1Y2tvbyAlPiUgCiAgZ2dwbG90KGFlcyhzYW1wbGU9bGVuZ3RoKSkgKwogIGdlb21fcXEoKSArCiAgZ2VvbV9xcV9saW5lKCkgKyAKICBmYWNldF9ncmlkKH50eXBlKQpgYGAKClRoZXJlIHNlZW0gdG8gYmUgbm8gY2xlYXIgZGV2aWF0aW9ucyBmcm9tIG5vcm1hbGl0eS4KClRoZSB0aGlyZCBhc3N1bXB0aW9uIG9mIGVxdWFsIHZhcmlhbmNlcyBzZWVtcyB0byBiZSBtZXQgYmFzZWQgb24gdGhlCnZpc3VhbGl6YXRpb24gd2l0aCB0aGUgYm94cGxvdHMgKHNlZSBhYm92ZSkuCgpBcyBzdWNoLCB3ZSBtYXkgcHJvY2VlZCB3aXRoIHRoZSBBTk9WQSBhbmFseXNpcy4KCiMgRGF0YSBhbmFseXNpcwoKIyMgQU5PVkEKCmBgYHtyfQpmaXQgPC0gbG0obGVuZ3RofnR5cGUsIEN1Y2tvbykKZml0X2Fub3ZhIDwtIGFub3ZhKGZpdCkKZml0X2Fub3ZhCmBgYAoKVGhlIHAtdmFsdWUgb2YgdGhlIEFOT1ZBIGFuYWx5c2lzIGlzIGV4dHJlbWVseSBzaWduaWZpY2FudAoocC12YWx1ZSA9IGByIGZvcm1hdChmaXRfYW5vdmEkIlByKD5GKSJbMV0sZGlnaXRzPTQpYCksIApzbyB3ZSByZWplY3QgdGhlIG51bGwgaHlwb3RoZXNpcyB0aGF0IHRoZSBtZWFuIAplZ2cgbGVuZ3RoIGlzIGVxdWFsIGJldHdlZW4gdGhlIGRpZmZlcmVudCBiaXJkIHR5cGVzLgpXZSBjYW4gc2F5IHRoYXQgdGhlIG1lYW4gZWdnIGxlbmd0aCBpcyBzaWduaWZpY2FudGx5IGRpZmZlcmVudApiZXR3ZWVuIGF0IGxlYXN0IHR3byBiaXJkIHR5cGVzIG9uIHRoZSA1JSBzaWduaWZpY2FuY2UgbGV2ZWwuCgpCYXNlZCBvbiB0aGlzIGFuYWx5c2lzLCB3ZSBkbyBub3QgeWV0IGtub3cgYmV0d2VlbiB3aGljaCBwYXJ0aWN1bGFyCmJpcmQgdHlwZXMgdGhlcmUgaXMgYSBzaWduaWZpY2FudCBkaWZmZXJlbmNlLiBUbyBzdHVkeSB0aGlzLCB3ZSB3aWxsCnBlcmZyb20gdGhlIFR1Y2tleSBwb3N0LWhvYyBhbmFseXNpcy4KCiMjIFBvc3QtaG9jIGFuYWx5c2lzCgpXZSB3aWxsIHBlcmZvcm0gYSBwb3N0LWhvYyBhbmFseXNpcywgdG8gbG9vayBhdCB0aGUgZGlmZmVyZW5jZSAKaW4gZWdnIGxlbmd0aCBiZXR3ZWVuIGVhY2ggcGFpcndpc2UgY29tcGFyaXNvbiBvZiBiaXJkIHR5cGVzLgpJbXBvcnRhbnRseSwgd2l0aCB0aGlzIHN0cmF0ZWd5LCB0aGUgcC12YWx1ZXMgd2lsbCBiZSBjb3JyZWN0bHkKYWRqdXN0ZWQgZm9yIG11bHRpcGxlIHRlc3RpbmcuCgpUaGUgbnVsbCBoeXBvdGhlc2lzIGZvciBlYWNoIHBhaXJ3aXNlIHRlc3QgaXMgdGhhdCB0aGVyZSBpcyBubwpkaWZmZXJlbmNlIGluIHRoZSBtZWFuIGVnZyBsZW5ndGggYmV0d2VlbiBib3RoIGJpcmQgdHlwZXMuCgpUaGUgYWx0ZXJuYXRpdmUgaHlwb3RoZXNpcyBmb3IgZWFjaCBwYWlyd2lzZSB0ZXN0IHN0YXRlcyB0aGF0IHRoZXJlCmlzIGluZGVlZCBhIGRpZmZlcmVuY2UgaW4gdGhlIG1lYW4gZWdnIGxlbmd0aCBiZXR3ZWVuIGJvdGggYmlyZCB0eXBlcy4KCldlIHdpbGwgYWxzbyBjYWxjdWxhdGUgdGhlIGNvbmZpZGVuY2UgaW50ZXJ2YWwgb24gdGhlIG1lYW4gZGlmZmVyZW5jZXMuCgpgYGB7cixtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KG11bHRjb21wLCBxdWlldGx5ID0gVFJVRSkKbWNwIDwtIGdsaHQoZml0LCBsaW5mY3QgPSBtY3AodHlwZSA9ICJUdWtleSIpKQpzdW1tYXJ5KG1jcCkKY29uZmludChtY3ApCmBgYAoKIyBDb25jbHVzaW9uCgpXZSBoYXZlIGZvdW5kIGFuIGV4dHJlbWVseSBzaWduaWZpY2FudCBkZXBlbmRlbmNlIChwLXZhbHVlID0gYHIgZm9ybWF0KGZpdF9hbm92YSQiUHIoPkYpIlsxXSxkaWdpdHM9NClgKQpiZXR3ZWVuIHRoZSBtZWFuIGVnZyBsZW5ndGggYW5kIGJpcmQgdHlwZSBvbiB0aGUgZ2xvYmFsIAo1JSBzaWduaWZpY2FuY2UgbGV2ZWwuCgpUaGUgbWVhbiBsZW5ndGggb2YgY3Vja29vJ3MgZWdncyBpbiBuZXN0cyBvZiB0aGUgRXVyYXNpYW4Kd2VybiBhcmUgc21hbGxlciBhcyBjb21wYXJlZCB0byB0aG9zZSBmcm9tIGFsbCBvdGhlciBiaXJkCnR5cGVzIGluIHRoZSBkYXRhc2V0OgoKLSB0aGUgbWVhZG93IHBpcGl0IChhZGp1c3RlZCBwLXZhbHVlID0gMC4wMDg1NiwgbWVhbiBkaWZmZXJlbmNlID0gLTEuMzcgbW0sIDk1JSBDSSBbLTIuNTAzMjM7IC0wLjIzNjc3XSkKLSB0aGUgdHJlZSBwaXBpdCAoYWRqdXN0ZWQgcC12YWx1ZSA9IDAuMDAxMTIsIG1lYW4gZGlmZmVyZW5jZSA9IC0xLjk2IG1tLCA5NSUgQ0kgWy0zLjM0NzkyOyAtMC41NzIwOF0pCi0gdGhlIGR1bm5vY2sgKGFkanVzdGVkIHAtdmFsdWUgPSAwLjAwMTE1LCBtZWFuIGRpZmZlcmVuY2UgPSAtMS45OSBtbSwgOTUlIENJIFstMy40MDUxMzsgLTAuNTc3NzNdKQotIHRoZSBFdXJvcGVhbiByb2JpbiAoYWRqdXN0ZWQgcC12YWx1ZSA9IDAuMDMxODQsIG1lYW4gZGlmZmVyZW5jZSA9IC0xLjQ1IG1tLCA5NSUgQ0kgWy0yLjgxMjIzOyAtMC4wNzc3N10pCi0gdGhlIHdoaXRlIHdhZ3RhaWwgKGFkanVzdGVkIHAtdmFsdWUgPSAwLjAzMTg0LCBtZWFuIGRpZmZlcmVuY2UgPSAtMS43NyBtbSwgOTUlIENJIFstMy4xNjI0NDsgLTAuMzg0MjJdKQoKRm9yIHRoZSBvdGhlciBiaXJkIHR5cGVzLCB3ZSBoYXZlIGluc3VmZmljaWVudCBldmlkZW5jZSB0byBzdWdnZXN0CmRpZmZlcmVuY2VzIGluIG1lYW4gbGVuZ3RoIG9mIHRoZSBjdWNrb28gYmlyZCdzIGVnZ3MuCgoK