Class 4: Logic models and DAGs

# In-person<br>session 4

**September 13, 2021**

]

---

# Plan for today

.box-2.medium.sp-after-half[How to read a regression table]

.box-4.medium.sp-after-half[Miscellaneous R things]

.box-1.medium.sp-after-half[Logic models]

.box-5.medium.sp-after-half[DAGs]

---

layout: false
name: regression-table
class: center middle section-title section-title-5 animated fadeIn

# How to read<br>a regression table

---

<table style="font-size:0.5em;border-bottom: 0; width: auto !important; margin-left: auto; margin-right: auto;" class="table">
 <thead>
  <tr>
   <th style="text-align:left;">   </th>
   <th style="text-align:center;"> Model 1 </th>
   <th style="text-align:center;"> Model 2 </th>
   <th style="text-align:center;"> Model 3 </th>
   <th style="text-align:center;"> Model 4 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:center;"> 362.307 </td>
   <td style="text-align:center;"> -5780.831*** </td>
   <td style="text-align:center;"> -5736.897*** </td>
   <td style="text-align:center;"> -5433.534*** </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;"> (283.345) </td>
   <td style="text-align:center;"> (305.815) </td>
   <td style="text-align:center;"> (307.959) </td>
   <td style="text-align:center;"> (286.558) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_length_mm </td>
   <td style="text-align:center;"> 87.415*** </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 6.047 </td>
   <td style="text-align:center;"> -5.201 </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;"> (6.402) </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> (5.180) </td>
   <td style="text-align:center;"> (4.860) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> flipper_length_mm </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 49.686*** </td>
   <td style="text-align:center;"> 48.145*** </td>
   <td style="text-align:center;"> 48.209*** </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> (1.518) </td>
   <td style="text-align:center;"> (2.011) </td>
   <td style="text-align:center;"> (1.841) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> sexmale </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 358.631*** </td>
  </tr>
  <tr>
   <td style="text-align:left;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px"> (41.572) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Num.Obs. </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 333 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 </td>
   <td style="text-align:center;"> 0.354 </td>
   <td style="text-align:center;"> 0.759 </td>
   <td style="text-align:center;"> 0.760 </td>
   <td style="text-align:center;"> 0.807 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 Adj. </td>
   <td style="text-align:center;"> 0.352 </td>
   <td style="text-align:center;"> 0.758 </td>
   <td style="text-align:center;"> 0.759 </td>
   <td style="text-align:center;"> 0.805 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> AIC </td>
   <td style="text-align:center;"> 5400.0 </td>
   <td style="text-align:center;"> 5062.9 </td>
   <td style="text-align:center;"> 5063.5 </td>
   <td style="text-align:center;"> 4863.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> BIC </td>
   <td style="text-align:center;"> 5411.5 </td>
   <td style="text-align:center;"> 5074.4 </td>
   <td style="text-align:center;"> 5078.8 </td>
   <td style="text-align:center;"> 4882.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Log.Lik. </td>
   <td style="text-align:center;"> -2696.987 </td>
   <td style="text-align:center;"> -2528.427 </td>
   <td style="text-align:center;"> -2527.741 </td>
   <td style="text-align:center;"> -2426.664 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> F </td>
   <td style="text-align:center;"> 186.443 </td>
   <td style="text-align:center;"> 1070.745 </td>
   <td style="text-align:center;"> 536.626 </td>
   <td style="text-align:center;"> 457.118 </td>
  </tr>
</tbody>
<tfoot><tr><td style="padding: 0; " colspan="100%">
<sup></sup> + p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</td></tr></tfoot>
</table>

---

<table style="NAborder-bottom: 0; width: auto !important; margin-left: auto; margin-right: auto;" class="table">
 <thead>
  <tr>
   <th style="text-align:left;">   </th>
   <th style="text-align:center;"> Model 1 </th>
   <th style="text-align:center;"> Model 2 </th>
   <th style="text-align:center;"> Model 3 </th>
   <th style="text-align:center;"> Model 4 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> (Intercept) </td>
   <td style="text-align:center;"> 362.307 </td>
   <td style="text-align:center;"> -5780.831*** </td>
   <td style="text-align:center;"> -5736.897*** </td>
   <td style="text-align:center;"> -5433.534*** </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;"> (283.345) </td>
   <td style="text-align:center;"> (305.815) </td>
   <td style="text-align:center;"> (307.959) </td>
   <td style="text-align:center;"> (286.558) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> bill_length_mm </td>
   <td style="text-align:center;"> 87.415*** </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 6.047 </td>
   <td style="text-align:center;"> -5.201 </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;"> (6.402) </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> (5.180) </td>
   <td style="text-align:center;"> (4.860) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> flipper_length_mm </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 49.686*** </td>
   <td style="text-align:center;"> 48.145*** </td>
   <td style="text-align:center;"> 48.209*** </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> (1.518) </td>
   <td style="text-align:center;"> (2.011) </td>
   <td style="text-align:center;"> (1.841) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> sexmale </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> 358.631*** </td>
  </tr>
  <tr>
   <td style="text-align:left;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px">  </td>
   <td style="text-align:center;box-shadow: 0px 1px"> (41.572) </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Num.Obs. </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 342 </td>
   <td style="text-align:center;"> 333 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 </td>
   <td style="text-align:center;"> 0.354 </td>
   <td style="text-align:center;"> 0.759 </td>
   <td style="text-align:center;"> 0.760 </td>
   <td style="text-align:center;"> 0.807 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> R2 Adj. </td>
   <td style="text-align:center;"> 0.352 </td>
   <td style="text-align:center;"> 0.758 </td>
   <td style="text-align:center;"> 0.759 </td>
   <td style="text-align:center;"> 0.805 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> AIC </td>
   <td style="text-align:center;"> 5400.0 </td>
   <td style="text-align:center;"> 5062.9 </td>
   <td style="text-align:center;"> 5063.5 </td>
   <td style="text-align:center;"> 4863.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> BIC </td>
   <td style="text-align:center;"> 5411.5 </td>
   <td style="text-align:center;"> 5074.4 </td>
   <td style="text-align:center;"> 5078.8 </td>
   <td style="text-align:center;"> 4882.4 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Log.Lik. </td>
   <td style="text-align:center;"> -2696.987 </td>
   <td style="text-align:center;"> -2528.427 </td>
   <td style="text-align:center;"> -2527.741 </td>
   <td style="text-align:center;"> -2426.664 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> F </td>
   <td style="text-align:center;"> 186.443 </td>
   <td style="text-align:center;"> 1070.745 </td>
   <td style="text-align:center;"> 536.626 </td>
   <td style="text-align:center;"> 457.118 </td>
  </tr>
</tbody>
<tfoot><tr><td style="padding: 0; " colspan="100%">
<sup></sup> + p &lt; 0.1, * p &lt; 0.05, ** p &lt; 0.01, *** p &lt; 0.001</td></tr></tfoot>
</table>

---

.box-5.large[See full documentation and<br>examples for `modelsummary()` [here](https://vincentarelbundock.github.io/modelsummary/articles/modelsummary.html)]

---

layout: false
name: r-stuff
class: center middle section-title section-title-4 animated fadeIn

# R stuff

---

---

.box-4.large[Make nicer tables when<br>knitting with `kable()`]

.box-inv-4[(Or even fancier tables with [kableExtra](https://cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html)!)]

---

.box-4.large[Navigating larger R Markdown files]

---

.box-4.large[Figure resizing]

---

.box-4.large[It's okay to remove placeholder text!]

---

.box-4.large[R vs. Python]

---

layout: false
name: logic-models
class: center middle section-title section-title-1 animated fadeIn

# Logic models

---

---

.box-1.large[Logic models as managerial tools]

---

.box-1.large[Inputs vs. Activities vs.<br>Outputs vs. Outcomes]

---

.box-1.large[Impact theory vs. logic model]

---

# Impact theory

.center[
<figure>
  <img src="img/03/psd-impact-theory-arrow.png" alt="PSD impact theory" title="PSD impact theory" width="100%">
</figure>
]

---

.center[
<figure>
  <img src="img/01/psd-logic-model.png" alt="PSD logic model" title="PSD logic model" width="70%">
</figure>
]

---

# MPA/MPP at GSU

.pull-left[
<figure>
  <img src="img/03/mpp-mission@4x.png" alt="MPP mission" title="MPP mission" width="130%">
</figure>
]

.pull-right[
<figure>
  <img src="img/03/mpa-mission@4x.png" alt="MPA mission" title="MPA mission" width="100%">
</figure>
]

???

Inputs:

- Students
- Curriculum
- Money (grants, tuition)
- AYSPS itself
- Faculty, staff
- Technology (iCollege, PAWS)
- Infrastructure
- Transportation
- State regulations
- National accreditation standards
- Faculty research

Activities:

- Classes
- Group work
- Exams
- Internships
- Job fairs
- Commencement
- Studying
- Tutoring
- Office hours
- Advising
- Attendance

Outputs:

- Assignments and projects
- Grades
- Degree
- Jobs
- Awards
- Network
- Debt

Outcomes:

- Leadership
- AYS national ranking
- Public service motivation
- Critical thinking
- Efficient workforce
- Income
- Better society

---

---

.box-1.medium[Isn't it best to always<br>have an articulated theory?]

.box-1.medium[Should implicit theory and articulated theory<br>be the same thing in most cases?]

---

.box-1.medium[How much does this evaluation stuff cost?]

.box-1.medium[Can you do scaled-down versions<br>of these evaluations?]

---

.box-1.medium[What if a program exists already<br>and doesn't have a logic model?]

.box-1.medium[What if a program exists already and doesn't have baseline data (or any data!)?]

---

.box-1.large[What about stakeholders, politicians,<br>and other decision makers?]

---

.box-1.medium[What should you do if you find that your theory of change (or logic model in general) is wrong in the middle of the program? Is it ethical to stop or readjust?]

---

---

# Outcomes and programs

.box-1.medium[Outcome variable]

.box-inv-1.sp-after-half.SMALL[Thing you're measuring]

.box-1.medium[Outcome change]

.box-inv-1.sp-after-half.SMALL[∆ in thing you're measuring over time]

.box-1.medium[Program effect]

.box-inv-1.SMALL[∆ in thing you're measuring over time *because of* the program]

---

# Outcomes and programs

---

layout: false
name: dags
class: center middle section-title section-title-5 animated fadeIn

# DAGs

---

---

.box-5.medium[You keep saying that causal inference<br>lets you "legally" make causal claims.]

.box-5.medium[Are there actual legal consequences<br>if you make a causal claim without<br>specific language?]

---

.box-5.large[Causal thinking is necessary—<br>even for descriptive work!]

---

]

???

Necessity of causal thinking: Mention the McElreath tweet on birth certificate introduction and death ages: <https://twitter.com/rlmcelreath/status/1427564280744976384>

<https://www.biorxiv.org/content/10.1101/704080v2>

---

.box-5.less-medium["Every time I get a haircut, I become more mature!"]

---

.box-5.less-medium["Every time I get a haircut, I become more mature!"]

---

.box-5.less-medium[Getting older opens a backdoor path]

---

.box-5.less-medium[How do I know which of these is which?]

.box-inv-5.small[Common cause]
]

.pull-middle-3.center[
.box-5.medium[Causation]

.box-inv-5.small[Mediation]
]

.box-inv-5.small[Selection /<br>endogeneity]
]

---

.box-5.medium[But what does that mean,<br>"opening a backdoor path"?]

.box-5.medium[How does statistical association<br>get passed through paths?]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/confounding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/mediating-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/colliding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

---

.box-5.large[How do we actually<br>adjust for these things?]

---

.box-5.medium[How can you be sure<br>you include everything in a DAG?]

.box-5.medium[Is there a rule of thumb<br>for the number of nodes?]

---

.box-5.large[How exactly do colliders<br>mess up your results?]

.box-5.medium[It looks like you can<br>still get the effect of X on Y]

---

---

.center[
<figure>
  <img src="img/04-class/facebook.png" alt="Facebook collider" title="Facebook collider" width="55%">
</figure>
]

???

<https://www.businessinsider.com/facebook-sent-incomplete-misinformation-data-flawed-researchers-2021-9>

<https://www.nytimes.com/live/2020/2020-election-misinformation-distortions#facebook-sent-flawed-data-to-misinformation-researchers>

---

---

.box-5.medium[Why do DAGs have to be acyclic?]

.box-5.medium[What if there really is reverse causation?]

---

.box-5.large[What's the difference between<br>logic models and DAGs?]

---

# DAGs vs. Logic models

.box-5.large[DAGs are a *statistical* tool]

.box-5.large.sp-before[Logic models are a *managerial* tool]

---

.pull-left[
<figure>
  <img src="img/04-class/greenspace-eater.png" alt="Green space in Berkeley" title="Green space in Berkeley" width="100%">
</figure>
]

.pull-right[
<figure>
  <img src="img/04-class/greenspace-conversation.png" alt="Covid green spaces" title="Covid green spaces" width="100%">
</figure>
]

???

<https://theconversation.com/how-cities-can-add-accessible-green-space-in-a-post-coronavirus-world-139194>

<https://sf.eater.com/2020/5/14/21258980/berkeley-coronavirus-covid-19-jesse-arreguin-street-closures>