The poison dataset
In this experiment, 96 fish (dojofish, goldfish and zebrafish) were placed separately in a tank with two liters of water and a certain dose (in mg) of the poison EI-43,064. The resistance of the fish against the poison was measured as the amount of minutes the fish survived after being exposed to the poison (Surv_time
, in minutes). Additionally, the weight of each fish was measured.
Goal
Suppose that researchers are ,mainly interested in studying the effect of poison dose on the survival of fish. They know however that the weight can also impact the survival and might also change the effect of the poison dose.
In this tutorial session we will focus on Dojofish and we will model the survival time in function of the dose and the weight of the fish, and including an interaction between dose and weight.
Load libraries
# install.packages("GGally")
library(GGally)
library(car)
library(multcomp)
library(tidyverse)
theme_set(theme_bw())
Data exploration
Prior to the analysis, we should explore our data. To start our data exploration, we will make use of the ggpairs
function of the GGally
R package. This function will generate a visualization containing multiple panels, which display (1) univariate plots of the different variables in our dataset, (2) bivariate plots and (3) correlation coefficients between the different variables.
Based on these plots, we observe that:
- The survival time seems to be associated with dose and fish weight.
From the tutorial of H6 we have seen that the fish weights were not nicely uniform across the different poison dosages due to the randomisation.
poison %>%
filter(Species == "Dojofish") %>%
ggplot(aes(x = Dose, y = Weight)) +
geom_point() +
ggtitle("Association between dose and weight") +
theme_bw() +
stat_summary(
fun = mean, geom = "point",
col = "black", size = 4,
shape = 24, fill = "red"
)
Analysis with main effect and interaction for dose and weight
Model specification
\[
y_i=\beta_0+\beta_d x_d + \beta_g x_g +\beta_{d:g} x_d x_g+ \epsilon_i,
\]
with \(\epsilon_i \text{ i.i.d. } N(0,\sigma^2)\)
Assumptions
The model will again be fit to allow for assessing the model assumptions
# lm_Int <- lm(log2Surv_time ~ Dose+Weight+Dose:Weight, data = poison) # equivalent
lm_Int <- lm(log2Surv_time ~ Dose * Weight, data = poison) # * -> short notation
par(mfrow = c(2, 2))
plot(lm_Int)
The plots look very similar to those of the additive model from the previous exercise so we know that all assumptions are met.
Inference
We then inspect the results.
##
## Call:
## lm(formula = log2Surv_time ~ Dose * Weight, data = poison)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.62807 -0.34662 -0.05587 0.30656 0.79086
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.2851 2.4062 -0.118 0.906
## Dose -0.2217 1.5440 -0.144 0.887
## Weight 1.6061 1.1325 1.418 0.165
## Dose:Weight -0.3485 0.7242 -0.481 0.633
##
## Residual standard error: 0.416 on 35 degrees of freedom
## Multiple R-squared: 0.5173, Adjusted R-squared: 0.4759
## F-statistic: 12.5 on 3 and 35 DF, p-value: 1.036e-05
Interpretation of model parameters
We may interpret the estimate parameters as follows:
When we compare the log2 survival of fish with weight \(x_g\) that are exposed to a dose that differs with 1 mg/l, the expected log2 survival time will be \(\beta_d+\beta_{d:g}*x_g\) higher for the fish that were exposed to the highest dose.
When we compare the log2 survival of fish exposed to dose \(x_g\), but that have a weight that differs with 1 g, the expected log2 survival time will be \(\beta_d+\beta_{d:g}*x_g\) higher for fish with the highest weight.
The parameter \(\beta_{d:g}\) thus shows that the effect of dose on the log2 survival time is dependent on the weight of the fish, and, that the effect of weight on the log2 survival time dependents on the dose that was administered.
Inference
The effect of dose is now parameterized by two model parameters (\(\beta_d\) and \(\beta_{d:g}\)). We first evaluate an omnibus hypotheses that there is no effect of dose, i.e., no main effect nor an interaction effect. We can test this with an F-test that compares a full model (1) containing a main effect for dose, a main effect for weight and an interaction between dose and weight with a model (2) that only contains a main effect for weight (i.e. no effect for dose).
lmDojo_weight <- lm(log2Surv_time ~ Weight, data = poison)
anova(lmDojo_weight, lm_Int)
We observe an extremely significant (overall, or global) effect for dose on the log2 survival time of dojofish (p-value <<0.001)`.
Conventional approach
We already established that there is a significant overall effect of dose. Now, we will test if there is a significant interaction effect between dose and weight. Since we only have one interaction term in this model, this can be achieved in several ways:
- The
summary
function
- An F-test comparing models with and without the interaction effect
- An ANOVA table with type III sum of squares
##
## Call:
## lm(formula = log2Surv_time ~ Dose * Weight, data = poison)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.62807 -0.34662 -0.05587 0.30656 0.79086
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.2851 2.4062 -0.118 0.906
## Dose -0.2217 1.5440 -0.144 0.887
## Weight 1.6061 1.1325 1.418 0.165
## Dose:Weight -0.3485 0.7242 -0.481 0.633
##
## Residual standard error: 0.416 on 35 degrees of freedom
## Multiple R-squared: 0.5173, Adjusted R-squared: 0.4759
## F-statistic: 12.5 on 3 and 35 DF, p-value: 1.036e-05
lm_additive <- lm(log2Surv_time ~ Dose + Weight, data = poison) # 2
anova(lm_additive, lm_Int) # 2
Anova(lm_Int, type = "III") # 3
There is no significant interaction between dose and weight. As such, the effect of dose on survival is not signifiantly different between fish of different weight (p = 0.63).
The conventional approach is to remove the interaction effect from the model. As such, we are left with the additive linear regression model.
##
## Call:
## lm(formula = log2Surv_time ~ Dose + Weight, data = poison)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.59629 -0.33110 -0.06836 0.32507 0.83315
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.8294 0.6457 1.285 0.207119
## Dose -0.9590 0.1888 -5.081 1.17e-05 ***
## Weight 1.0783 0.2792 3.862 0.000451 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4116 on 36 degrees of freedom
## Multiple R-squared: 0.5141, Adjusted R-squared: 0.4871
## F-statistic: 19.04 on 2 and 36 DF, p-value: 2.282e-06
2^(coef(lm_additive)) # original scale
## (Intercept) Dose Weight
## 1.777004 0.514402 2.111484
## 2.5 % 97.5 %
## (Intercept) 0.7169728 4.4042734
## Dose 0.3945101 0.6707292
## Weight 1.4259918 3.1265018
Conclusion
The conclusion is exactly the same as for the additive model analysis above (obviously, as we are dealing with the same model):
The dose of the poison has an extremely significant effect on the survival time of dojofish (p-value << 0.001). The geometric average of the survival time for dojofish that are exposed to a poison dose that is 1mg/L larger is approximately halved, factor = \(2^{\beta_d}=\) 0.51) .
The effect of dose on survival is not significantly different between fish of different weight. (p = 0.63).
LS0tCnRpdGxlOiAiRXhlcmNpc2UgOC4yOiBOb24tYWRkaXRpdmUgbGluZWFyIG1vZGVsIG9uIHRoZSBwb2lzb24gZGF0YXNldCIKYXV0aG9yOiAiTGlldmVuIENsZW1lbnQsIEplcm9lbiBHaWxpcyBhbmQgTWlsYW4gTWFsZmFpdCIKZGF0ZTogInN0YXRPbWljcywgR2hlbnQgVW5pdmVyc2l0eSAoaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSIKLS0tCgojIFRoZSBwb2lzb24gZGF0YXNldAoKSW4gdGhpcyBleHBlcmltZW50LCA5NiBmaXNoIChkb2pvZmlzaCwgZ29sZGZpc2ggYW5kIHplYnJhZmlzaCkKd2VyZSBwbGFjZWQgc2VwYXJhdGVseSBpbiBhIHRhbmsgd2l0aCB0d28gbGl0ZXJzIG9mIHdhdGVyIGFuZAphIGNlcnRhaW4gZG9zZSAoaW4gbWcpIG9mIHRoZSBwb2lzb24gRUktNDMsMDY0LiBUaGUgcmVzaXN0YW5jZQpvZiB0aGUgZmlzaCBhZ2FpbnN0IHRoZSBwb2lzb24gd2FzIG1lYXN1cmVkIGFzIHRoZSBhbW91bnQgb2YKbWludXRlcyB0aGUgZmlzaCBzdXJ2aXZlZCBhZnRlciBiZWluZyBleHBvc2VkIHRvIHRoZSBwb2lzb24gKGBTdXJ2X3RpbWVgLCBpbgptaW51dGVzKS4gQWRkaXRpb25hbGx5LCB0aGUgd2VpZ2h0IG9mIGVhY2ggZmlzaCB3YXMgbWVhc3VyZWQuCgojIEdvYWwKClN1cHBvc2UgdGhhdCByZXNlYXJjaGVycyBhcmUgLG1haW5seSBpbnRlcmVzdGVkIGluIHN0dWR5aW5nIHRoZSBlZmZlY3Qgb2YKcG9pc29uIGRvc2Ugb24gdGhlIHN1cnZpdmFsIG9mIGZpc2guIFRoZXkga25vdyBob3dldmVyIHRoYXQgdGhlIHdlaWdodCBjYW4KYWxzbyBpbXBhY3QgdGhlIHN1cnZpdmFsIGFuZCBtaWdodCBhbHNvIGNoYW5nZSB0aGUgZWZmZWN0IG9mIHRoZSBwb2lzb24gZG9zZS4KCkluIHRoaXMgdHV0b3JpYWwgc2Vzc2lvbiB3ZSB3aWxsIGZvY3VzIG9uIERvam9maXNoIGFuZCB3ZSB3aWxsIG1vZGVsIHRoZQpzdXJ2aXZhbCB0aW1lIGluIGZ1bmN0aW9uIG9mIHRoZSBkb3NlIGFuZCB0aGUgd2VpZ2h0IG9mIHRoZSBmaXNoLAoqKmFuZCBpbmNsdWRpbmcgYW4gaW50ZXJhY3Rpb24gYmV0d2VlbiBkb3NlIGFuZCB3ZWlnaHQuKioKCgpMb2FkIGxpYnJhcmllcwoKYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CiMgaW5zdGFsbC5wYWNrYWdlcygiR0dhbGx5IikKbGlicmFyeShHR2FsbHkpCmxpYnJhcnkoY2FyKQpsaWJyYXJ5KG11bHRjb21wKQoKbGlicmFyeSh0aWR5dmVyc2UpCnRoZW1lX3NldCh0aGVtZV9idygpKQpgYGAKCiMgSW1wb3J0IHRoZSBkYXRhCgpgYGB7ciwgbWVzc2FnZT1GQUxTRX0KcG9pc29uIDwtIHJlYWRfY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vc3RhdE9taWNzL1BTTFNEYXRhL21haW4vcG9pc29uLmNzdiIpCmBgYAoKIyBEYXRhIHRpZHlpbmcKCldlIGNhbiBzZWUgYSBjb3VwbGUgb2YgdGhpbmdzIGluIHRoZSBkYXRhIHRoYXQgY2FuIGJlIGltcHJvdmVkOgoKMS4gQ2FwaXRhbGlzZSB0aGUgZmlzdCBjb2x1bW4gbmFtZQoKMi4gU2V0IHRoZSBTcGVjaWVzIGNvbHVtbiBhcyBhIGZhY3RvcgoKMy4gQ2hhbmdlIHRoZSBzcGVjaWVzIGZhY3RvciBsZXZlbHMgZnJvbSAwLCAxIGFuZCAyIHRvCkRvam9maXNoLCBHb2xkZmlzaCBhbmQgWmVicmFmaXNoLiAqSGludCo6IHVzZSB0aGUgYGZjdF9yZWNvZGVgIGZ1bmN0aW9uLgoKNC4gSW4gcHJldmlvdXMgYW5hbHlzaXMgb24gdGhpcyBkYXRhc2V0IChgU2ltcGxlIGxpbmVhciByZWdyZXNzaW9uIHNlc3Npb25gKSwgd2UKcGVyZm9ybWVkIGEgbG9nLXRyYW5zZm9ybWF0aW9uIG9uIHRoZSByZXNwb25zZSB2YXJpYWJsZSBgU3Vydl90aW1lYCB0byBtZWV0IHRoZQpub3JtYWxpdHkgYW5kIGhvbW9zY2VkYXN0aWNpdHkgYXNzdW1wdGlvbnMgb2YgdGhlIGxpbmVhciBtb2RlbC4gSGVyZSwgd2Ugd2lsbAppbW1lZGlhdGVseSB3b3JrIHdpdGggbG9nLXRyYW5zZm9ybWVkIHN1cnZpdmFsIHRpbWVzOyBzdG9yZSB0aGVzZSBpbiB0aGUgbmV3CnZhcmlhYmxlIGBsb2cyU3Vydl90aW1lYCBhbmQgcmVtb3ZlIHRoZSBub24tdHJhbnNmb3JtZWQgdmFsdWVzLgoKNS4gU3Vic2V0IHRoZSBkYXRhIHRvIG9ubHkgcmV0YWluICoqRG9qb2Zpc2gqKi4KCmBgYHtyfQpwb2lzb24gPC0gcG9pc29uICU+JQogIHJlbmFtZSgiU3BlY2llcyIgPSAic3BlY2llcyIpICU+JQogIG11dGF0ZShTcGVjaWVzID0gYXMuZmFjdG9yKFNwZWNpZXMpKSAlPiUKICBtdXRhdGUoU3BlY2llcyA9IGZjdF9yZWNvZGUoU3BlY2llcywKICAgIERvam9maXNoID0gIjAiLCBHb2xkZmlzaCA9ICIxIiwgWmVicmFmaXNoID0gIjIiCiAgKSkgJT4lCiAgbXV0YXRlKGxvZzJTdXJ2X3RpbWUgPSBsb2cyKFN1cnZfdGltZSkpICU+JQogIHNlbGVjdCgtU3Vydl90aW1lKSAlPiUKICBmaWx0ZXIoU3BlY2llcyA9PSAiRG9qb2Zpc2giKQoKcG9pc29uCmBgYAoKIyBEYXRhIGV4cGxvcmF0aW9uCgpQcmlvciB0byB0aGUgYW5hbHlzaXMsIHdlIHNob3VsZCBleHBsb3JlIG91ciBkYXRhLgpUbyBzdGFydCBvdXIgZGF0YSBleHBsb3JhdGlvbiwgd2Ugd2lsbCBtYWtlIHVzZSBvZiB0aGUgYGdncGFpcnNgIGZ1bmN0aW9uIG9mIHRoZQpgR0dhbGx5YCBSIHBhY2thZ2UuIFRoaXMgZnVuY3Rpb24gd2lsbCBnZW5lcmF0ZSBhIHZpc3VhbGl6YXRpb24gY29udGFpbmluZwptdWx0aXBsZSBwYW5lbHMsIHdoaWNoIGRpc3BsYXkgKDEpIHVuaXZhcmlhdGUgcGxvdHMgb2YgdGhlIGRpZmZlcmVudCB2YXJpYWJsZXMKaW4gb3VyIGRhdGFzZXQsICgyKSBiaXZhcmlhdGUgcGxvdHMgYW5kICgzKSBjb3JyZWxhdGlvbiBjb2VmZmljaWVudHMgYmV0d2Vlbgp0aGUgZGlmZmVyZW50IHZhcmlhYmxlcy4KCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCBmaWcud2lkdGg9MTJ9CiAgZ2dwYWlycyhwb2lzb24pCmBgYAoKQmFzZWQgb24gdGhlc2UgcGxvdHMsIHdlIG9ic2VydmUgdGhhdDoKCi0gVGhlIHN1cnZpdmFsIHRpbWUgc2VlbXMgdG8gYmUgYXNzb2NpYXRlZCB3aXRoIGRvc2UgYW5kIGZpc2ggd2VpZ2h0LgoKRnJvbSB0aGUgdHV0b3JpYWwgb2YgSDYgd2UgaGF2ZSBzZWVuIHRoYXQgdGhlIGZpc2ggd2VpZ2h0cyB3ZXJlIG5vdCBuaWNlbHkKdW5pZm9ybSBhY3Jvc3MgdGhlIGRpZmZlcmVudCBwb2lzb24gZG9zYWdlcyBkdWUgdG8gdGhlIHJhbmRvbWlzYXRpb24uCgoKYGBge3IsIG1lc3NhZ2U9RkFMU0V9CnBvaXNvbiAlPiUKICBmaWx0ZXIoU3BlY2llcyA9PSAiRG9qb2Zpc2giKSAlPiUKICBnZ3Bsb3QoYWVzKHggPSBEb3NlLCB5ID0gV2VpZ2h0KSkgKwogIGdlb21fcG9pbnQoKSArCiAgZ2d0aXRsZSgiQXNzb2NpYXRpb24gYmV0d2VlbiBkb3NlIGFuZCB3ZWlnaHQiKSArCiAgdGhlbWVfYncoKSArCiAgc3RhdF9zdW1tYXJ5KAogICAgZnVuID0gbWVhbiwgZ2VvbSA9ICJwb2ludCIsCiAgICBjb2wgPSAiYmxhY2siLCBzaXplID0gNCwKICAgIHNoYXBlID0gMjQsIGZpbGwgPSAicmVkIgogICkKYGBgCgojIEFuYWx5c2lzIHdpdGggbWFpbiBlZmZlY3QgYW5kIGludGVyYWN0aW9uIGZvciBkb3NlIGFuZCB3ZWlnaHQKCiMjIE1vZGVsIHNwZWNpZmljYXRpb24KCiQkCnlfaT1cYmV0YV8wK1xiZXRhX2QgeF9kICsgXGJldGFfZyB4X2cgK1xiZXRhX3tkOmd9IHhfZCB4X2crIFxlcHNpbG9uX2ksCiQkCgp3aXRoICRcZXBzaWxvbl9pIFx0ZXh0eyBpLmkuZC4gfSBOKDAsXHNpZ21hXjIpJAoKIyMgQXNzdW1wdGlvbnMKClRoZSBtb2RlbCB3aWxsIGFnYWluIGJlIGZpdCB0byBhbGxvdyBmb3IgYXNzZXNzaW5nIHRoZSBtb2RlbCBhc3N1bXB0aW9ucwoKYGBge3J9CiMgbG1fSW50IDwtIGxtKGxvZzJTdXJ2X3RpbWUgfiBEb3NlK1dlaWdodCtEb3NlOldlaWdodCwgZGF0YSA9IHBvaXNvbikgIyBlcXVpdmFsZW50CmxtX0ludCA8LSBsbShsb2cyU3Vydl90aW1lIH4gRG9zZSAqIFdlaWdodCwgZGF0YSA9IHBvaXNvbikgIyAqIC0+IHNob3J0IG5vdGF0aW9uCgpwYXIobWZyb3cgPSBjKDIsIDIpKQpwbG90KGxtX0ludCkKYGBgCgpUaGUgcGxvdHMgbG9vayB2ZXJ5IHNpbWlsYXIgdG8gdGhvc2Ugb2YgdGhlIGFkZGl0aXZlIG1vZGVsIGZyb20gdGhlIHByZXZpb3VzIGV4ZXJjaXNlIHNvIHdlIGtub3cgdGhhdCBhbGwgYXNzdW1wdGlvbnMgYXJlICoqbWV0KiouCgojIyBJbmZlcmVuY2UKCldlIHRoZW4gaW5zcGVjdCB0aGUgcmVzdWx0cy4KCmBgYHtyfQpzdW1tYXJ5KGxtX0ludCkKYGBgCgojIyBJbnRlcnByZXRhdGlvbiBvZiBtb2RlbCBwYXJhbWV0ZXJzCgpXZSBtYXkgaW50ZXJwcmV0IHRoZSBlc3RpbWF0ZSBwYXJhbWV0ZXJzIGFzIGZvbGxvd3M6CgotIFdoZW4gd2UgY29tcGFyZSB0aGUgbG9nMiBzdXJ2aXZhbCBvZiBmaXNoIHdpdGggd2VpZ2h0ICR4X2ckIHRoYXQgYXJlIGV4cG9zZWQgdG8gYSBkb3NlIHRoYXQgZGlmZmVycyB3aXRoIDEgbWcvbCwgdGhlIGV4cGVjdGVkIGxvZzIgc3Vydml2YWwgdGltZSB3aWxsIGJlCiRcYmV0YV9kK1xiZXRhX3tkOmd9KnhfZyQgaGlnaGVyIGZvciB0aGUgZmlzaCB0aGF0IHdlcmUgZXhwb3NlZCB0byB0aGUgaGlnaGVzdCBkb3NlLgoKLSBXaGVuIHdlIGNvbXBhcmUgdGhlIGxvZzIgc3Vydml2YWwgb2YgZmlzaCBleHBvc2VkIHRvIGRvc2UgJHhfZyQsIGJ1dCB0aGF0IGhhdmUgYSB3ZWlnaHQgdGhhdCBkaWZmZXJzIHdpdGggMSBnLCB0aGUgZXhwZWN0ZWQgbG9nMiBzdXJ2aXZhbCB0aW1lCndpbGwgYmUgICRcYmV0YV9kK1xiZXRhX3tkOmd9KnhfZyQgaGlnaGVyIGZvciBmaXNoIHdpdGggdGhlIGhpZ2hlc3Qgd2VpZ2h0LgoKVGhlIHBhcmFtZXRlciAkXGJldGFfe2Q6Z30kIHRodXMgc2hvd3MgdGhhdCB0aGUgZWZmZWN0IG9mIGRvc2Ugb24gdGhlIGxvZzIgc3Vydml2YWwgdGltZSBpcyBkZXBlbmRlbnQgb24gdGhlCndlaWdodCBvZiB0aGUgZmlzaCwgYW5kLCB0aGF0IHRoZSBlZmZlY3QKb2Ygd2VpZ2h0IG9uIHRoZSBsb2cyIHN1cnZpdmFsIHRpbWUgZGVwZW5kZW50cyBvbiB0aGUgZG9zZSB0aGF0IHdhcwphZG1pbmlzdGVyZWQuCgojIyBJbmZlcmVuY2UKClRoZSBlZmZlY3Qgb2YgZG9zZSBpcyBub3cgcGFyYW1ldGVyaXplZCBieSB0d28gbW9kZWwgcGFyYW1ldGVycyAoJFxiZXRhX2QkCmFuZCAkXGJldGFfe2Q6Z30kKS4gV2UgZmlyc3QgZXZhbHVhdGUgYW4gKm9tbmlidXMqIGh5cG90aGVzZXMgdGhhdCB0aGVyZSBpcwpubyBlZmZlY3Qgb2YgZG9zZSwgaS5lLiwgbm8gbWFpbiBlZmZlY3Qgbm9yIGFuIGludGVyYWN0aW9uIGVmZmVjdC4gV2UgY2FuIHRlc3QKdGhpcyB3aXRoIGFuIEYtdGVzdCB0aGF0IGNvbXBhcmVzIGEgKmZ1bGwqIG1vZGVsICgxKSBjb250YWluaW5nIGEgbWFpbiBlZmZlY3QKZm9yIGRvc2UsIGEgbWFpbiBlZmZlY3QgZm9yIHdlaWdodCBhbmQgYW4gaW50ZXJhY3Rpb24gYmV0d2VlbiBkb3NlIGFuZCB3ZWlnaHQKd2l0aCBhIG1vZGVsICgyKSB0aGF0IG9ubHkgY29udGFpbnMgYSBtYWluIGVmZmVjdCBmb3Igd2VpZ2h0IChpLmUuIG5vIGVmZmVjdApmb3IgZG9zZSkuCgpgYGB7cn0KbG1Eb2pvX3dlaWdodCA8LSBsbShsb2cyU3Vydl90aW1lIH4gV2VpZ2h0LCBkYXRhID0gcG9pc29uKQphbm92YShsbURvam9fd2VpZ2h0LCBsbV9JbnQpCmBgYAoKV2Ugb2JzZXJ2ZSBhbiBleHRyZW1lbHkgc2lnbmlmaWNhbnQgKG92ZXJhbGwsIG9yIGdsb2JhbCkgZWZmZWN0IGZvciBkb3NlIG9uCnRoZSBsb2cyIHN1cnZpdmFsIHRpbWUgb2YgZG9qb2Zpc2gKKHAtdmFsdWUgPDwwLjAwMSlgLgoKIyMgQ29udmVudGlvbmFsIGFwcHJvYWNoCgpXZSBhbHJlYWR5IGVzdGFibGlzaGVkIHRoYXQgdGhlcmUgaXMgYSBzaWduaWZpY2FudCBvdmVyYWxsIGVmZmVjdCBvZiBkb3NlLgpOb3csIHdlIHdpbGwgdGVzdCBpZiB0aGVyZSBpcyBhIHNpZ25pZmljYW50IGludGVyYWN0aW9uIGVmZmVjdCBiZXR3ZWVuIGRvc2UKYW5kIHdlaWdodC4gU2luY2Ugd2Ugb25seSBoYXZlIG9uZSBpbnRlcmFjdGlvbiB0ZXJtIGluIHRoaXMgbW9kZWwsIHRoaXMgY2FuCmJlIGFjaGlldmVkIGluIHNldmVyYWwgd2F5czoKCjEuIFRoZSBgc3VtbWFyeWAgZnVuY3Rpb24KMi4gQW4gRi10ZXN0IGNvbXBhcmluZyBtb2RlbHMgd2l0aCBhbmQgd2l0aG91dCB0aGUgaW50ZXJhY3Rpb24gZWZmZWN0CjMuIEFuIEFOT1ZBIHRhYmxlIHdpdGggdHlwZSBJSUkgc3VtIG9mIHNxdWFyZXMKCgpgYGB7cn0Kc3VtbWFyeShsbV9JbnQpICMgMQpsbV9hZGRpdGl2ZSA8LSBsbShsb2cyU3Vydl90aW1lIH4gRG9zZSArIFdlaWdodCwgZGF0YSA9IHBvaXNvbikgIyAyCmFub3ZhKGxtX2FkZGl0aXZlLCBsbV9JbnQpICMgMgpBbm92YShsbV9JbnQsIHR5cGUgPSAiSUlJIikgIyAzCmBgYAoKVGhlcmUgaXMgbm8gc2lnbmlmaWNhbnQgaW50ZXJhY3Rpb24gYmV0d2VlbiBkb3NlIGFuZCB3ZWlnaHQuIEFzIHN1Y2gsIHRoZQplZmZlY3Qgb2YgZG9zZSBvbiBzdXJ2aXZhbCBpcyBub3Qgc2lnbmlmaWFudGx5IGRpZmZlcmVudCBiZXR3ZWVuIGZpc2ggb2YKZGlmZmVyZW50IHdlaWdodCAocCA9IGByIGZvcm1hdChBbm92YShsbV9JbnQsdHlwZT0iSUlJIikkIlByKD5GKSJbNF0sIGRpZ2l0cyA9IDIpYCkuCgpUaGUgY29udmVudGlvbmFsIGFwcHJvYWNoIGlzIHRvIHJlbW92ZSB0aGUgaW50ZXJhY3Rpb24gZWZmZWN0IGZyb20gdGhlCm1vZGVsLiBBcyBzdWNoLCB3ZSBhcmUgbGVmdCB3aXRoIHRoZSBhZGRpdGl2ZSBsaW5lYXIgcmVncmVzc2lvbiBtb2RlbC4KCmBgYHtyfQpzdW1tYXJ5KGxtX2FkZGl0aXZlKQpgYGAKCmBgYHtyfQoyXihjb2VmKGxtX2FkZGl0aXZlKSkgIyBvcmlnaW5hbCBzY2FsZQoyXihjb25maW50KGxtX2FkZGl0aXZlKSkKYGBgCgojIyMgQ29uY2x1c2lvbgoKVGhlIGNvbmNsdXNpb24gaXMgZXhhY3RseSB0aGUgc2FtZSBhcyBmb3IgdGhlIGFkZGl0aXZlIG1vZGVsIGFuYWx5c2lzIGFib3ZlCihvYnZpb3VzbHksIGFzIHdlIGFyZSBkZWFsaW5nIHdpdGggdGhlIHNhbWUgbW9kZWwpOgoKVGhlIGRvc2Ugb2YgdGhlIHBvaXNvbiBoYXMgYW4gZXh0cmVtZWx5IHNpZ25pZmljYW50IGVmZmVjdCBvbiB0aGUgc3Vydml2YWwgdGltZQpvZiBkb2pvZmlzaCAocC12YWx1ZSA8PCAwLjAwMSkuIFRoZSBnZW9tZXRyaWMgYXZlcmFnZSBvZiB0aGUgc3Vydml2YWwgdGltZSBmb3IKZG9qb2Zpc2ggdGhhdCBhcmUgZXhwb3NlZCB0byBhIHBvaXNvbiBkb3NlIHRoYXQgaXMgMW1nL0wgbGFyZ2VyIGlzIGFwcHJveGltYXRlbHkKaGFsdmVkLApmYWN0b3IgPSAkMl57XGJldGFfZH09JCBgciBmb3JtYXQoMl4obG1fYWRkaXRpdmUkY29lZlsiRG9zZSJdKSxkaWdpdHM9MilgKSAuCgoKVGhlIGVmZmVjdCBvZiBkb3NlIG9uIHN1cnZpdmFsIGlzIG5vdCBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBiZXR3ZWVuIGZpc2ggb2YKZGlmZmVyZW50IHdlaWdodC4gKHAgPSBgciBmb3JtYXQoc3VtbWFyeShsbV9JbnQpJGNvZWZmaWNpZW50c1s0LDRdLCBkaWdpdHM9MilgKS4KCgojIFJlbWFyawoKTm90ZSwgdGhhdCB3ZSBoYXZlIGhhdmUgdG8gdGVzdCBpZiB0aGUgaW50ZXJhY3Rpb24gaXMgc2lnbmlmaWNhbnQgYmVmb3JlIHVzaW5nCnRoZSBhZGRpdGl2ZSBtb2RlbC4K