class: center middle main-title section-title-3 # In-person<br>session 5 .class-info[ **September 20, 2021** .light[PMAP 8521: Program evaluation<br> Andrew Young School of Policy Studies ] ] --- name: outline class: title title-inv-8 # Plan for today -- .box-5.medium.sp-after-half[Colliders] -- .box-2.medium.sp-after-half[Potential outcomes vs. do() notation] -- .box-4.medium.sp-after-half[do-calculus, adjustment, and CATEs] -- .box-6.medium.sp-after-half[Logic models, DAGs, and measurement] --- layout: false name: colliders class: center middle section-title section-title-5 animated fadeIn # Colliders --- layout: true class: middle --- .box-5.less-medium.sp-after[How do I know which of these is which?] .center[ <figure> <img src="img/05-class/dag-associations.png" alt="DAG associations" title="DAG associations" width="100%"> </figure> ] --- class: middle .box-5.medium[But what does that mean,<br>"opening a backdoor path"?] .box-5.medium[How does statistical association<br>get passed through paths?] --- class: middle .pull-left[ <figure> <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%"> </figure> ] .pull-right[ <img src="05-class_files/figure-html/confounding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" /> ] --- class: middle .pull-left[ <figure> <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%"> </figure> ] .pull-right[ <img src="05-class_files/figure-html/mediating-dag-alone-1.png" width="100%" style="display: block; margin: auto;" /> ] --- class: middle .pull-left[ <figure> <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%"> </figure> ] .pull-right[ <img src="05-class_files/figure-html/colliding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" /> ] --- layout: true class: middle --- d-separation --- .box-5.medium[How can you be sure<br>you include everything in a DAG?] .box-5.medium[Is there a rule of thumb<br>for the number of nodes?] --- .box-5.large[How exactly do colliders<br>mess up your results?] .box-5.medium[It looks like you can<br>still get the effect of X on Y] --- .center[ <figure> <img src="img/04-class/facebook.png" alt="Facebook collider" title="Facebook collider" width="55%"> </figure> ] ??? <https://www.businessinsider.com/facebook-sent-incomplete-misinformation-data-flawed-researchers-2021-9> <https://www.nytimes.com/live/2020/2020-election-misinformation-distortions#facebook-sent-flawed-data-to-misinformation-researchers> --- layout: false class: title title-5 # Does niceness improve appearance? .pull-left[ <img src="05-class_files/figure-html/unnamed-chunk-1-1.png" width="100%" style="display: block; margin: auto;" /> ] -- .pull-right[ <img src="05-class_files/figure-html/unnamed-chunk-2-1.png" width="100%" style="display: block; margin: auto;" /> ] --- class: title title-5 # Collider distorts the true effect! .pull-left[ <img src="05-class_files/figure-html/unnamed-chunk-3-1.png" width="100%" style="display: block; margin: auto;" /> ] -- .pull-right[ <img src="05-class_files/figure-html/unnamed-chunk-4-1.png" width="100%" style="display: block; margin: auto;" /> ] --- .box-5.large[Effect of race on police use of force<br>using administrative data] --- .box-5.medium[Effect of race on police use of force<br>using administrative data] .pull-left[ <figure> <img src="img/05-class/klm-dag.png" alt="Use of force" title="Use of force" width="100%"> </figure> ] .pull-right[ <figure> <img src="img/05-class/klm.png" alt="Use of force" title="Use of force" width="100%"> </figure> ] --- layout: false name: po-do class: center middle section-title section-title-2 animated fadeIn # Potential outcomes<br>vs. do() notation --- layout: true class: title title-2 --- `$$\operatorname{E}(\cdot), \mathbf{E}(\cdot), \mathbb{E}(\cdot) \quad \text{vs.}\quad \operatorname{P}(\cdot)$$` --- # Causal effects with potential outcomes $$ `\begin{aligned} & \textbf{Potential outcomes notation:} \\ \delta\ =& {\textstyle \frac{1}{n} \sum_{i=1}^n} Y_i (1) - Y_i (0) \\ & \\ & \text{or alternatively with } \textbf{E} \\ \delta\ =& \textbf{E} [Y_i (1) - Y_i (0)] \\ \end{aligned}` $$ --- # Causal effects with do() $$ `\begin{aligned} & \textbf{Pearl notation:} \\ \delta\ =& \textbf{E}[Y_i \mid \operatorname{do}(X = 1) - Y_i \mid \operatorname{do}(X = 0)] \\ & \\ & \text{or more simply} \\ \delta\ =& \textbf{E} [Y_i \mid \operatorname{do}(X)] \\ \end{aligned}` $$ --- layout: false class: middle --- $$ \delta \quad = \quad \textbf{E} [Y_i \mid \operatorname{do}(X)] \quad = \quad \textbf{E} [Y_i (1) - Y_i (0)] $$ --- .box-2.medium[We can't see this] `$$\textbf{E} [Y_i \mid \operatorname{do}(X)] \quad \text{or} \quad \textbf{E} [Y_i (1) - Y_i (0)]$$` .box-2.medium[So we find the average causal effect (ACE)] $$ \hat{\delta} = \textbf{E} [Y_i \mid X = 1] - \textbf{E} [Y_i \mid X = 0] $$ --- .center[ <figure> <img src="img/05-class/cor-not-cause.png" alt="Correlation is not causation" title="Correlation is not causation" width="100%"> </figure> ] --- layout: false name: po-do class: center middle section-title section-title-4 animated fadeIn # do-calculus,<br>adjustment, and CATEs --- layout: true class: middle --- .box-4.medium[ How can you create a DAG/transform the expression and do all the backdoor adjustments to determine causality if you can't possibly think of all variables that may impact your outcome variable? ] --- .box-4.medium[Where can we learn more about *do*-calculus?] .box-inv-4[[Here!](https://www.andrewheiss.com/blog/2021/09/07/do-calculus-backdoors/)] .center[ <figure> <img src="img/05-class/do-calculus-math.png" alt="Do-calculus" title="Do-calculus" width="70%"> </figure> ] --- .box-4.medium[Adjusting for backdoor confounding] .center[ <figure> <img src="img/05-class/backdoor.png" alt="Backdoor adjustment" title="Backdoor adjustment" width="100%"> </figure> ] --- .box-4.medium[Adjusting for frontdoor confounding] --- .box-4.medium[When you're making groups for CATE, how do you decide what groups to put people in?] --- layout: false name: logic-dag class: center middle section-title section-title-6 animated fadeIn # Logic models, DAGs,<br>and measurement --- layout: true class: middle --- .box-6.large[What's the difference between<br>logic models and DAGs?] .box-inv-6[Can't I just remake my logic model in Dagitty and be done?] --- layout: false class: title title-6 # DAGs vs. Logic models .box-6.large[DAGs are a *statistical* tool] .box-inv-6[Describe a data-generating process<br>and isolate/identify relationships] -- .box-6.large.sp-before[Logic models are a *managerial* tool] .box-inv-6[Oversee the inner workings of a program and its theory] --- class: middle .pull-left[ <figure> <img src="img/04-class/greenspace-eater.png" alt="Green space in Berkeley" title="Green space in Berkeley" width="100%"> </figure> ] .pull-right[ <figure> <img src="img/04-class/greenspace-conversation.png" alt="Covid green spaces" title="Covid green spaces" width="100%"> </figure> ] ??? <https://theconversation.com/how-cities-can-add-accessible-green-space-in-a-post-coronavirus-world-139194> <https://sf.eater.com/2020/5/14/21258980/berkeley-coronavirus-covid-19-jesse-arreguin-street-closures>