class: center, middle, inverse, title-slide .title[ # 3 simple insights ] .subtitle[ ## on experiments in the social sciences ] .author[ ### Merlin Schaeffer ] .date[ ### 2023-04-21 ] --- class: clear # Allais' classic experiment on decision theory .push-left[ <img src="media/Allain_1.png" width="100%" style="display: block; margin: auto;" /> .center[.backgrnote[*Source*: [Wikipedia](https://en.wikipedia.org/wiki/Allais_paradox)]] ] --- class: clear # Allais' classic experiment on decision theory .push-left[ <img src="media/Allain_2.png" width="100%" style="display: block; margin: auto;" /> .center[.backgrnote[*Source*: [Wikipedia](https://en.wikipedia.org/wiki/Allais_paradox)]] - 45% of the world's leading decision theorists were inconsistent in 1952! ] -- .push-right[ .content-box-red[ .center[ **Insight 1** Experiment = artificial environment/setup to test a theory about how the (social) world works. ]]] --- class: clear # Randomized Controlled Trials .font50[Experiment that tests *causal* claims] .push-left[ <img src="./media/randomization2.png" width="60%" style="display: block; margin: auto;" /> - If we *randomly* divide subjects into treatment and control groups: They come from the same underlying population. <br> <br> `\(\rightarrow\)` Similar .alert[on average], *in every way*,<br> **including their `\(Y_{0}\)` **! <br> <br> `\(\rightarrow E[Y_{0i}|D=1] = E[Y_{0i}|D=0]\)`! ] -- .push-right[ `$$\begin{equation}\begin{split} E & [Y_{1i}|D=1] - E[Y_{0i}|D=0] \\ \\ & = E[\color{red}{Y_{0i} + \kappa} |D=1] - E[Y_{0i}|D=0], \\ \\ &= \color{red}{\kappa} + \underbrace{E[Y_{0i} |D=1] - E[Y_{0i}|D=0]}_{\underbrace{0}_{\text{(if randomization has worked)}}}, \\ \\ & = \underbrace{\color{red}{\kappa}.}_{\text{The average causal effect}} \end{split}\end{equation}$$` ] --- class: clear # 2) Healthcare chauvinism .font50[A conjoint survey experiment with Mikkel] .left-column[ - (2 X 2 X 2 X 2 =) 16 full factorial design. + Power analysis for `\(\beta = 0.05\)`: `\(n = 2,500\)` with 3 choices per respondent. + Power ~1 for AMCE & <br> 0.7 for AMCIE. ] .right-column[ <img src="media/Paired_Vignette.png" width="100%" style="display: block; margin: auto;" /> ] --- # Healthcare chauvinism .right-column[ <img src="media/Figure_1.svg" width="100%" style="display: block; margin: auto;" /> ] -- .left-column[ .content-box-red[ .center[ **Insight 2** Many RCTs don't really test a causal change induced by `\(X\)`, but rather elicit latent opinions. ]]] --- class: middle center .content-box-red[ .center[**Insight 3** RCTs which demonstrate how `\(X\)` changes people's opinions and/or behavior are pretty rare! ]]