+ - 0:00:00
Notes for current slide
Notes for next slide

3 simple insights

on experiments in the social sciences

Merlin Schaeffer

2023-04-21

1 / 7

Allais' classic experiment on decision theory

Source: Wikipedia

2 / 7

Allais' classic experiment on decision theory

Source: Wikipedia

  • 45% of the world's leading decision theorists were inconsistent in 1952!
3 / 7

Allais' classic experiment on decision theory

Source: Wikipedia

  • 45% of the world's leading decision theorists were inconsistent in 1952!

Insight 1

Experiment = artificial environment/setup to test a theory about how the (social) world works.

3 / 7

Randomized Controlled Trials Experiment that tests causal claims

  • If we randomly divide subjects into treatment and control groups: They come from the same underlying population.

    Similar on average, in every way,
    including their Y0Y0 !

    E[Y0i|D=1]=E[Y0i|D=0]E[Y0i|D=1]=E[Y0i|D=0]!
4 / 7

Randomized Controlled Trials Experiment that tests causal claims

  • If we randomly divide subjects into treatment and control groups: They come from the same underlying population.

    Similar on average, in every way,
    including their Y0Y0 !

    E[Y0i|D=1]=E[Y0i|D=0]E[Y0i|D=1]=E[Y0i|D=0]!

E[Y1i|D=1]E[Y0i|D=0]=E[Y0i+κ|D=1]E[Y0i|D=0],=κ+E[Y0i|D=1]E[Y0i|D=0]0(if randomization has worked),=κ.The average causal effect

4 / 7

2) Healthcare chauvinism A conjoint survey experiment with Mikkel

  • (2 X 2 X 2 X 2 =) 16 full factorial design.
    • Power analysis for β=0.05: n=2,500 with 3 choices per respondent.
    • Power ~1 for AMCE &
      0.7 for AMCIE.

5 / 7

Healthcare chauvinism

6 / 7

Healthcare chauvinism

Insight 2

Many RCTs don't really test a causal change induced by X, but rather elicit latent opinions.

6 / 7

Insight 3

RCTs which demonstrate how X changes people's opinions and/or behavior are pretty rare!

7 / 7

Allais' classic experiment on decision theory

Source: Wikipedia

2 / 7
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
oTile View: Overview of Slides
Esc Back to slideshow