Creative Commons License


1 Need for a good control

  • A good control group is crucial.

  • To assess the effect of an intervention, we need to compare a test and control group.

  • This is often not possible in a pretest/post-test design: e.g. effect before and after administering a drug without the use of a placebo group.

  • Groups in an observational study are often not comparable: advanced statistical methods are required to draw causal conclusions.

  • Double blinding

  • We have to be aware of confounding!

  • Randomized studies: random assignment of subjects in the study to the different treatment arms \(\rightarrow\) comparable groups.


2 Randomization

  • Randomization completely at random (no systematic allocation).

2.1 Simple Randomization

  • Can lead to differences in the number of experimental units in each treatment arm

  • in 5% of the cases we might observe an imbalance of

    • of at least 60:40 in a study with 100 subjects, and
    • of at least 531:469 in a study with 1000 subjects.
  • This imbalance is not problematic, but causes a loss in precision.


2.2 Balanced Randomization

  • Equal numbers of each treatment are assigned to a block of 2 or 4 patients.

      1. AB, (2) BA
      1. AABB, (2) ABAB, (3) ABBA, (4) BABA, (5) BAAB, (6) BBAA
  • Balanced Randomization ensures \(\pm\) the same number of people in the control and the treatment arm of the experiment.

  • Does not make that we have an equal number of males with and without the treatment, etc.

  • In small studies, it is possible that the groups are unbalanced in other characteristics (e.g. gender, race, age …)

  • This is not problematic because it occurs at random, but, again it causes a loss in precision.


2.3 Stratified randomization

  • The imbalance according to for instance gender can be avoided using stratified Randomization: balanced randomization per stratum

Stratified Randomization


3 Sample size

  • The sample size and the design are crucial.

  • The larger the sample size, the more precise the results.

4 Bad design example

  • dm: diabetic medium, nd: non diabetic medium, co: control

  • 4 bio-reps, 2 techreps/biorep

  • dm: diabetic medium, nd: non diabetic medium, co: control

  • 4 bio-reps, 2 techreps/biorep, 2 plates A & B

  • Treatment and plate almost entirely confounded


5 Wrap-up

  • Sample size is very important.

  • To assess the effect of a treatment, we should compare comparable and representative groups of subjects with and without the treatment (a good control!).

  • In observational studies, the researcher cannot choose the treatment. It was the patient or their MD who had chosen it

  • In experimental studies, the researcher assigns the treatment.

  • Confounding can be avoided via randomization.

  • We can also correct for confounding in the statistical analysis for the confounders that have been registered.

LS0tCnRpdGxlOiAiMy4gU29tZSBjb25jZXB0cyBvbiBleHBlcmltZW50YWwgZGVzaWduIgphdXRob3I6ICJMaWV2ZW4gQ2xlbWVudCIKZGF0ZTogInN0YXRPbWljcywgR2hlbnQgVW5pdmVyc2l0eSAoaHR0cHM6Ly9zdGF0b21pY3MuZ2l0aHViLmlvKSIKb3V0cHV0OgogICAgaHRtbF9kb2N1bWVudDoKICAgICAgY29kZV9kb3dubG9hZDogdHJ1ZQogICAgICB0aGVtZTogY29zbW8KICAgICAgdG9jOiB0cnVlCiAgICAgIHRvY19mbG9hdDogdHJ1ZQogICAgICBoaWdobGlnaHQ6IHRhbmdvCiAgICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQotLS0KCjxhIHJlbD0ibGljZW5zZSIgaHJlZj0iaHR0cHM6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LW5jLXNhLzQuMCI+PGltZyBhbHQ9IkNyZWF0aXZlIENvbW1vbnMgTGljZW5zZSIgc3R5bGU9ImJvcmRlci13aWR0aDowIiBzcmM9Imh0dHBzOi8vaS5jcmVhdGl2ZWNvbW1vbnMub3JnL2wvYnktbmMtc2EvNC4wLzg4eDMxLnBuZyIgLz48L2E+CgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGluY2x1ZGUgPSBUUlVFLCBjb21tZW50ID0gTkEsIGVjaG8gPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICAgbWVzc2FnZSA9IEZBTFNFLCB3YXJuaW5nID0gRkFMU0UpCmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KE5IQU5FUykKYGBgCgpgYGB7ciBwb3AyU2FtcDJQb3AsIG91dC53aWR0aD0nODAlJyxmaWcuYXNwPS44LCBmaWcuYWxpZ249J2NlbnRlcicsZWNobz1GQUxTRX0KaWYgKCJwaSIlaW4lbHMoKSkgcm0oInBpIikKa29wdm9ldGVyPC1mdW5jdGlvbih4LHksYW5nbGU9MCxsPS4yLGNleC5kb3Q9LjUscGNoPTE5LGNvbD0iYmxhY2siKQp7CmFuZ2xlPWFuZ2xlLzE4MCpwaQpwb2ludHMoeCx5LGNleD1jZXguZG90LHBjaD1wY2gsY29sPWNvbCkKbGluZXMoYyh4LHgrbCpjb3MoLXBpLzIrYW5nbGUpKSxjKHkseStsKnNpbigtcGkvMithbmdsZSkpLGNvbD1jb2wpCmxpbmVzKGMoeCtsLzIqY29zKC1waS8yK2FuZ2xlKSx4K2wvMipjb3MoLXBpLzIrYW5nbGUpK2wvNCpjb3MoYW5nbGUpKSxjKHkrbC8yKnNpbigtcGkvMithbmdsZSkseStsLzIqc2luKC1waS8yK2FuZ2xlKStsLzQqc2luKGFuZ2xlKSksY29sPWNvbCkKbGluZXMoYyh4K2wvMipjb3MoLXBpLzIrYW5nbGUpLHgrbC8yKmNvcygtcGkvMithbmdsZSkrbC80KmNvcyhwaSthbmdsZSkpLGMoeStsLzIqc2luKC1waS8yK2FuZ2xlKSx5K2wvMipzaW4oLXBpLzIrYW5nbGUpK2wvNCpzaW4ocGkrYW5nbGUpKSxjb2w9Y29sKQpsaW5lcyhjKHgrbCpjb3MoLXBpLzIrYW5nbGUpLHgrbCpjb3MoLXBpLzIrYW5nbGUpK2wvMipjb3MoLXBpLzIrcGkvNCthbmdsZSkpLGMoeStsKnNpbigtcGkvMithbmdsZSkseStsKnNpbigtcGkvMithbmdsZSkrbC8yKnNpbigtcGkvMitwaS80K2FuZ2xlKSksY29sPWNvbCkKbGluZXMoYyh4K2wqY29zKC1waS8yK2FuZ2xlKSx4K2wqY29zKC1waS8yK2FuZ2xlKStsLzIqY29zKC1waS8yLXBpLzQrYW5nbGUpKSxjKHkrbCpzaW4oLXBpLzIrYW5nbGUpLHkrbCpzaW4oLXBpLzIrYW5nbGUpK2wvMipzaW4oLXBpLzItcGkvNCthbmdsZSkpLGNvbD1jb2wpCn0KCnBhcihtYXI9YygwLDAsMCwwKSxtYWk9YygwLDAsMCwwKSkKcGxvdCgwLDAseGxhYj0iIix5bGFiPSIiLHhsaW09YygwLDEwKSx5bGltPWMoMCwxMCksY29sPTAseGF4dD0ibm9uZSIseWF4dD0ibm9uZSIsYXhlcz1GQUxTRSkKcmVjdCgwLDYsMTAsMTAsYm9yZGVyPSJyZWQiLGx3ZD0yKQp0ZXh0KC41LDgsInBvcHVsYXRpb24iLHNydD05MCxjb2w9InJlZCIsY2V4PTIpCnN5bWJvbHMgKDMsIDgsIGNpcmNsZXM9MS41LCBjb2w9InJlZCIsYWRkPVRSVUUsZmc9InJlZCIsaW5jaGVzPUZBTFNFLGx3ZD0yKQpzZXQuc2VlZCgzMzApCmdyaWQ9c2VxKDAsMS4zLC4wMSkKCmZvciAoaSBpbiAxOjUwKQp7CglhbmdsZTE9cnVuaWYobj0xLG1pbj0wLG1heD0zNjApCglhbmdsZTI9cnVuaWYobj0xLG1pbj0wLG1heD0zNjApCglyYWRpdXM9c2FtcGxlKGdyaWQscHJvYj1ncmlkXjIqcGkvc3VtKGdyaWReMipwaSksc2l6ZT0xKQoJa29wdm9ldGVyKDMrcmFkaXVzKmNvcyhhbmdsZTEvMTgwKnBpKSw4K3JhZGl1cypzaW4oYW5nbGUxLzE4MCpwaSksYW5nbGU9YW5nbGUyKQp9CnRleHQoNy41LDgsIkNob2xlc3Rlcm9sIGluIHBvcHVsYXRpb24iLGNvbD0icmVkIixjZXg9MS4yKQoKcmVjdCgwLDAsMTAsNCxib3JkZXI9ImJsdWUiLGx3ZD0yKQp0ZXh0KC41LDIsInNhbXBsZSIsc3J0PTkwLGNvbD0iYmx1ZSIsY2V4PTIpCnN5bWJvbHMgKDMsIDIsIGNpcmNsZXM9MS41LCBjb2w9InJlZCIsYWRkPVRSVUUsZmc9ImJsdWUiLGluY2hlcz1GQUxTRSxsd2Q9MikKZm9yIChpIGluIDA6MikKCWZvciAoaiBpbiAwOjQpCnsKCglrb3B2b2V0ZXIoMi4xK2oqKDMuOS0yLjEpLzQsMS4xK2kpCn0KdGV4dCg3LjUsMiwiQ2hvbGVzdGVyb2wgaW4gc2FtcGxlIixjb2w9ImJsdWUiLGNleD0xLjIpCgphcnJvd3MoMyw1LjksMyw0LjEsY29sPSJibGFjayIsbHdkPTMpCmFycm93cyg3LDQuMSw3LDUuOSxjb2w9ImJsYWNrIixsd2Q9MykKdGV4dCgxLjUsNSwiRVhQLiBERVNJR04gKDEpIixjb2w9ImJsYWNrIixjZXg9MS4yKQp0ZXh0KDguNSw1LCJFU1RJTUFUSU9OICZcbklORkVSRU5DRSAoMykiLGNvbD0iYmxhY2siLGNleD0xLjIpCnRleHQoNy41LC41LCJEQVRBIEVYUExPUkFUSU9OICZcbkRFU0NSSVBUSVZFIFNUQVRJU1RJQ1MgKDIpIixjb2w9ImJsYWNrIixjZXg9MS4yKQpgYGAKCi0tLQoKIyBOZWVkIGZvciBhIGdvb2QgY29udHJvbAoKLSBBIGdvb2QgY29udHJvbCBncm91cCBpcyBjcnVjaWFsLgoKLSBUbyBhc3Nlc3MgdGhlIGVmZmVjdCBvZiBhbiBpbnRlcnZlbnRpb24sIHdlIG5lZWQgdG8gY29tcGFyZSBhIHRlc3QgYW5kIGNvbnRyb2wgZ3JvdXAuCgotIFRoaXMgaXMgb2Z0ZW4gbm90IHBvc3NpYmxlIGluIGEgcHJldGVzdC9wb3N0LXRlc3QgZGVzaWduOiBlLmcuIGVmZmVjdCBiZWZvcmUgYW5kIGFmdGVyIGFkbWluaXN0ZXJpbmcgYSBkcnVnIHdpdGhvdXQgdGhlIHVzZSBvZiBhIHBsYWNlYm8gZ3JvdXAuCgotIEdyb3VwcyBpbiBhbiBvYnNlcnZhdGlvbmFsIHN0dWR5IGFyZSBvZnRlbiBub3QgY29tcGFyYWJsZTogYWR2YW5jZWQgc3RhdGlzdGljYWwgbWV0aG9kcyBhcmUgcmVxdWlyZWQgdG8gZHJhdyBjYXVzYWwgY29uY2x1c2lvbnMuIAoKLSBEb3VibGUgYmxpbmRpbmcKCi0gV2UgaGF2ZSB0byBiZSBhd2FyZSBvZiBjb25mb3VuZGluZyEKCi0gUmFuZG9taXplZCBzdHVkaWVzOiByYW5kb20gYXNzaWdubWVudCBvZiBzdWJqZWN0cyBpbiB0aGUgc3R1ZHkgdG8gdGhlIGRpZmZlcmVudCB0cmVhdG1lbnQgYXJtcyAkXHJpZ2h0YXJyb3ckIGNvbXBhcmFibGUgZ3JvdXBzLiAKCi0tLQoKIyBSYW5kb21pemF0aW9uCgotIFJhbmRvbWl6YXRpb24gY29tcGxldGVseSBhdCByYW5kb20gKG5vIHN5c3RlbWF0aWMgYWxsb2NhdGlvbikuCgojIyBTaW1wbGUgUmFuZG9taXphdGlvbgoKLSBDYW4gbGVhZCB0byBkaWZmZXJlbmNlcyBpbiB0aGUgbnVtYmVyIG9mIGV4cGVyaW1lbnRhbCB1bml0cyBpbiBlYWNoIHRyZWF0bWVudCBhcm0gCgotIGluIDUlIG9mIHRoZSBjYXNlcyB3ZSBtaWdodCBvYnNlcnZlIGFuIGltYmFsYW5jZSBvZiAKICAgIC0gb2YgYXQgbGVhc3QgNjA6NDAgaW4gYSBzdHVkeSB3aXRoIDEwMCBzdWJqZWN0cywgYW5kIAogICAgLSBvZiBhdCBsZWFzdCA1MzE6NDY5IGluIGEgc3R1ZHkgd2l0aCAxMDAwIHN1YmplY3RzLgoKLSBUaGlzIGltYmFsYW5jZSBpcyBub3QgcHJvYmxlbWF0aWMsIGJ1dCBjYXVzZXMgYSBsb3NzIGluIHByZWNpc2lvbi4gCgotLS0KCiMjIEJhbGFuY2VkIFJhbmRvbWl6YXRpb24KCi0gRXF1YWwgbnVtYmVycyBvZiBlYWNoIHRyZWF0bWVudCBhcmUgYXNzaWduZWQgdG8gYSBibG9jayBvZiAyIG9yIDQgcGF0aWVudHMuIAogICAgLSAoMSkgQUIsICgyKSBCQQogICAgLSAoMSkgQUFCQiwgKDIpIEFCQUIsICgzKSBBQkJBLCAoNCkgQkFCQSwgKDUpIEJBQUIsICg2KSBCQkFBCgotIEJhbGFuY2VkIFJhbmRvbWl6YXRpb24gZW5zdXJlcyAkXHBtJCB0aGUgc2FtZSBudW1iZXIgb2YgcGVvcGxlIGluIHRoZSBjb250cm9sIGFuZCB0aGUgdHJlYXRtZW50IGFybSBvZiB0aGUgZXhwZXJpbWVudC4gCgotIERvZXMgbm90IG1ha2UgdGhhdCB3ZSBoYXZlIGFuIGVxdWFsIG51bWJlciBvZiBtYWxlcyB3aXRoIGFuZCB3aXRob3V0IHRoZSB0cmVhdG1lbnQsIGV0Yy4gIAoKLSBJbiBzbWFsbCBzdHVkaWVzLCBpdCBpcyBwb3NzaWJsZSB0aGF0IHRoZSBncm91cHMgYXJlIHVuYmFsYW5jZWQgaW4gb3RoZXIgY2hhcmFjdGVyaXN0aWNzIChlLmcuIGdlbmRlciwgcmFjZSwgYWdlIC4uLikKCi0gVGhpcyBpcyBub3QgcHJvYmxlbWF0aWMgYmVjYXVzZSBpdCBvY2N1cnMgYXQgcmFuZG9tLCBidXQsIGFnYWluIGl0IGNhdXNlcyBhIGxvc3MgaW4gcHJlY2lzaW9uLiAKCi0tLQoKIyMgU3RyYXRpZmllZCByYW5kb21pemF0aW9uCgotIFRoZSBpbWJhbGFuY2UgYWNjb3JkaW5nIHRvIGZvciBpbnN0YW5jZSBnZW5kZXIgY2FuIGJlIGF2b2lkZWQgdXNpbmcgc3RyYXRpZmllZCBSYW5kb21pemF0aW9uOiBiYWxhbmNlZCByYW5kb21pemF0aW9uIHBlciBzdHJhdHVtCgohW1N0cmF0aWZpZWQgUmFuZG9taXphdGlvbl0oaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL0dUUEIvUFNMUzIwL2doLXBhZ2VzL2Fzc2V0cy9maWdzL3N0cmF0aWZpY2F0aW9uLnBuZyl7IHdpZHRoPTUwJSB9CgotLS0KCiMgU2FtcGxlIHNpemUgCgotIFRoZSBzYW1wbGUgc2l6ZSBhbmQgdGhlIGRlc2lnbiBhcmUgY3J1Y2lhbC4KCi0gVGhlIGxhcmdlciB0aGUgc2FtcGxlIHNpemUsIHRoZSBtb3JlIHByZWNpc2UgdGhlIHJlc3VsdHMuCgoKIyBCYWQgZGVzaWduIGV4YW1wbGUKCi0gZG06IGRpYWJldGljIG1lZGl1bSwgbmQ6IG5vbiBkaWFiZXRpYyBtZWRpdW0sIGNvOiBjb250cm9sCi0gNCBiaW8tcmVwcywgMiB0ZWNocmVwcy9iaW9yZXAKIVtdKGh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9HVFBCL1BTTFMyMC9naC1wYWdlcy9hc3NldHMvZmlncy9xcGNyQmFkRGVzaWduMS5wbmcpeyB3aWR0aD0xMDAlIH0KCi0gZG06IGRpYWJldGljIG1lZGl1bSwgbmQ6IG5vbiBkaWFiZXRpYyBtZWRpdW0sIGNvOiBjb250cm9sCi0gNCBiaW8tcmVwcywgMiB0ZWNocmVwcy9iaW9yZXAsIDIgcGxhdGVzIEEgJiBCCi0gVHJlYXRtZW50IGFuZCBwbGF0ZSBhbG1vc3QgZW50aXJlbHkgY29uZm91bmRlZCAKCiFbXShodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vR1RQQi9QU0xTMjAvZ2gtcGFnZXMvYXNzZXRzL2ZpZ3MvcXBjckJhZERlc2lnbjIucG5nKXsgd2lkdGg9MTAwJSB9CgotLS0KCgojIFdyYXAtdXAKCi0gU2FtcGxlIHNpemUgaXMgdmVyeSBpbXBvcnRhbnQuCgotIFRvIGFzc2VzcyB0aGUgZWZmZWN0IG9mIGEgdHJlYXRtZW50LCB3ZSBzaG91bGQgY29tcGFyZSBjb21wYXJhYmxlIGFuZCByZXByZXNlbnRhdGl2ZSBncm91cHMgb2Ygc3ViamVjdHMgd2l0aCBhbmQgd2l0aG91dCB0aGUgdHJlYXRtZW50IChhIGdvb2QgY29udHJvbCEpLiAKCi0gSW4gb2JzZXJ2YXRpb25hbCBzdHVkaWVzLCB0aGUgcmVzZWFyY2hlciBjYW5ub3QgY2hvb3NlIHRoZSB0cmVhdG1lbnQuIEl0IHdhcyB0aGUgcGF0aWVudCBvciB0aGVpciBNRCB3aG8gaGFkIGNob3NlbiBpdCAKCi0gSW4gZXhwZXJpbWVudGFsIHN0dWRpZXMsIHRoZSByZXNlYXJjaGVyIGFzc2lnbnMgdGhlIHRyZWF0bWVudC4KCi0gQ29uZm91bmRpbmcgY2FuIGJlIGF2b2lkZWQgdmlhIHJhbmRvbWl6YXRpb24uCgotIFdlIGNhbiBhbHNvIGNvcnJlY3QgZm9yIGNvbmZvdW5kaW5nIGluIHRoZSBzdGF0aXN0aWNhbCBhbmFseXNpcyBmb3IgdGhlIGNvbmZvdW5kZXJzIHRoYXQgaGF2ZSBiZWVuIHJlZ2lzdGVyZWQuIAoK