Measurement and DAGs

# Measurement and DAGs

**Session 4**

]

---

# Plan for today

.box-4.medium.sp-after-half[Abstraction, stretching, and validity]

.box-6.medium.sp-after-half[Causal models]

.box-5.medium[Paths, doors, and adjustment]

---

name: abstraction
class: center middle section-title section-title-4 animated fadeIn

# Abstraction, stretching, and validity

---

---

# Indicators

&nbsp;

---

&nbsp;

.box-4.large.sp-after[How do you measure abstract outcomes?]

.box-inv-4.large[Move up the ladder of abstraction.]

---

.center[
<figure>
 <img src="img/04/monty-python-witch.jpg" alt="Witch scene from Monty Python and the Holy Grail" title="Witch scene from Monty Python and the Holy Grail" width="80%">
</figure>
]

???

https://www.youtube.com/watch?v=yp_l5ntikaU

---

???

Here's a dataset of witches.

What are the common features across witches in general?

Are these criteria useful if certain cases don’t fit like Sabrina?

What are the most important elements here? Doing magic + being female – that’s good, probably – covers Hermione and HP world

---

---

# Conceptual stretching

.center[
<figure>
 <img src="img/04/stretching.jpg" alt="Magical women who aren't witches" title="Magical women who aren't witches" width="100%">
</figure>
]

???

But now the concept is stretched too far – this includes all magical female creatures, like trolls and goblins

Too much conceptual stretching

---

# Ladder of abstraction for witches

.box-inv-4.small[Mammal]

.box-inv-4.small[Enmagicked]

.box-4.tiny[Trolls, elves, gods/goddesses]

.box-inv-4.small[Female]

.box-4.tiny[Arwen, Winky, Athena]

.box-inv-4.small[Human]

.box-4.tiny[Salem witch trials]
]

.box-4.tiny[Hermione Granger, Sabrina Spellman]
]
]

.box-4.tiny[Elphaba, Halloween decorations]
]

---

# Connection to theory

]

.pull-right[
<figure>
 <img src="img/04/trials-2.jpg" alt="Salem witch trials" title="Salem witch trials" width="100%">
</figure>
]

???

Accusations typically targeted successful women; women who worked with medicine; women who posed a threat to the patriarchy. In real life, it was a misogynistic tool for maintaining social order.

What went wrong? The theory was wrong, yes, but also the indicators weren’t connected to reality

---

# Outcomes and programs

.box-4.medium[Outcome variable]

.box-inv-4.sp-after-half.SMALL[Thing you're measuring]

.box-4.medium[Outcome change]

.box-inv-4.sp-after-half.SMALL[∆ in thing you're measuring over time]

.box-4.medium[Program effect]

.box-inv-4.SMALL[∆ in thing you're measuring over time *because of* the program]

---

# Outcomes and programs

---

# Connecting measurment to programs

.box-inv-4.medium.sp-after-half[Measurable definition of program effect]

.box-inv-4.medium.sp-after-half[Ideal measurement]

.box-inv-4.medium.sp-after-half[Feasible measurement]

.box-inv-4.medium[Connection to real world]

---

layout: false
name: causal-models
class: center middle section-title section-title-6 animated fadeIn

# Causal models

---

---

# Types of data

.box-inv-6.medium[Experimental]

]

.box-inv-6.medium[Observational]

]

.box-inv-6.medium.sp-before[Which kind lets you prove causation?]

---

# Causation with observational data

.box-inv-6.medium.sp-after[Can you prove causation with observational data?]

.box-inv-6.medium[Why is it so controversial to use observational data?]

---

---

---

# The causal revolution

.pull-left-narrow[
<figure>
 <img src="img/01/book-of-why.jpg" alt="The Book of Why" title="The Book of Why" width="100%">
</figure>
]

.pull-right-wide.center[
<figure>
 <img src="img/01/judea-pearl.jpg" alt="Judea Pearl" title="Judea Pearl" width="90%">
</figure>
]

---

# Causal diagrams

.box-inv-6.medium[Directed acyclic graphs (DAGs)]

.box-6.SMALL[**Directed**: Each node has an arrow that points to another node]

.box-6.SMALL[**Acyclic**: You can't cycle back to a node (and arrows only have one direction)]

.box-6.SMALL[**Graph**: It's… um… a graph]

]

]

---

# Causal diagrams

.box-inv-6.medium[Directed acyclic graphs (DAGs)]

.box-6.SMALL[Graphical model of the process that generates the data]

.box-6.SMALL[Maps your philosophical model]

.box-6.SMALL[Fancy math ("*do*-calculus") tells you what to control for to isolate and identify causation]

]

![](04-slides_files/figure-html/simple-dag-1.png)

]

---

# Acyclicalness

.box-6.sp-after[**This isn't acyclic!** Wealth ↔ Power]

---

# How to draw a DAG

.box-inv-6.medium[What is the causal effect of an additional year of education on earnings?]

---

# 1. List variables

.box-6.sp-after[Education (treatment) → Earnings (outcome)]

.center.float-left.sp-after-half[
.box-inv-6[Location]&ensp;.box-inv-6[Ability]&ensp;.box-inv-6[Demographics]
]

.center.float-left.sp-after-half[
.box-inv-6[Socioeonomic status]&ensp;.box-inv-6[Year of birth]
]

.center.float-left[
.box-inv-6[Compulsory schooling laws]&ensp;.box-inv-6[Job connections]
]

---

# 2. Simplify

.box-6.sp-after[Education (treatment) → Earnings (outcome)]

.center.float-left.sp-after-half[
.box-inv-6[Location]&ensp;.box-inv-3[Ability]&ensp;.box-inv-3[Demographics]
]

.center.float-left.sp-after-half[
.box-inv-3[Socioeonomic status]&ensp;.box-inv-6[Year of birth]
]

.center.float-left[
.box-inv-6[Compulsory schooling laws]&ensp;.box-inv-6[Job connections]
]

---

# 3. Draw arrows

]

]

---

# 3. Draw arrows

.box-inv-6[Background, year of birth, location, job connections, and school requirements all cause education]

]

]

---

# 3. Draw arrows

]

]

---

# 3. Draw arrows

]

]

---

# 3. Draw arrows

.box-inv-6[Location and background are probably related, but neither causes the other. Something unobservable (U1) does that.]

]

]

---

# Let the computer do this!

.box-inv-6.large.sp-after-half[dagitty.net]

.box-inv-6.large[**ggdag** package in R]

---

layout: false
name: paths-doors-adjustment
class: center middle section-title section-title-5 animated fadeIn

# Paths, doors, and adjustment

---

---

# Causal identification

]

![](04-slides_files/figure-html/edu-earn-full-1.png)

]

---

# Causal identification

.box-inv-5.medium[A causal effect is *identified* if the association between treatment and outcome is propertly stripped and isolated]

---

# Paths and associations

.box-inv-5.medium[Arrows in a DAG transmit associations]

.box-inv-5.medium[You can redirect and control those paths by "adjusting" or "conditioning"]

---

# Three types of associations

.box-inv-5.small[Common cause]
]

.pull-middle-3.center[
.box-5.medium[Causation]

.box-inv-5.small[Mediation]
]

.box-inv-5.small[Selection / endogeneity]
]

---

# Confounding

.pull-left-wide[
<img src="04-slides_files/figure-html/confounding-dag-big-1.png" width="100%" style="display: block; margin: auto;" />
]

.box-inv-5.medium[But **Z** causes both **X** and **Y**]

.box-inv-5.medium[**Z** * confounds* the **X** → **Y** association]
]

---

# Paths

.box-inv-5.medium[**Z** is a *backdoor*]
]

---

# *d*-connection

.pull-right-narrow[
.box-inv-5[**X** and **Y** are "*d*-connected" because associations can pass through **Z**]

---

# Effect of money on elections

.box-inv-5.medium[What are the paths between **money** and **win margin**?]

.pull-left[
<img src="04-slides_files/figure-html/money-elections-1.png" width="100%" style="display: block; margin: auto;" />
]

---

# Closing doors

.pull-left[
<img src="04-slides_files/figure-html/confounding-dag-adjusted-1.png" width="100%" style="display: block; margin: auto;" />
]

---

# Closing doors

.pull-left[
.box-inv-5.small[Find the part of campaign money that is explained by quality, remove it. This is the residual part of money.]

.box-inv-5.small[Find the part of win margin that is explained by quality, remove it. This is the residual part of win margin.]

.box-inv-5.small[Find the relationship between the residual part of money and residual part of win margin. **This is the causal effect**.]
]

.pull-right[
<img src="04-slides_files/figure-html/money-elections-adjusted-1.png" width="100%" style="display: block; margin: auto;" />
]

---

# Closing doors

---

# How to adjust

.box-inv-5.medium[Include term in regression]

$$
`\begin{aligned}
\text{Win margin} =& \beta_0 + \beta_1 \text{Campaign money} +\\
& \beta_2 \text{Candidate quality} + \varepsilon
\end{aligned}`
$$

.center.float-left[
.box-inv-5.medium[Matching]&ensp;.box-inv-5.medium[Stratifying]

.box-inv-5.medium.sp-before-half[Inverse probability weighting]
]

---

# *d*-separation

.pull-right[
.box-inv-5.medium[If we control for **Z**, **X** and **Y** are now "*d*-separated" and the association is isolated!]

]

---

# Closing backdoors

.pull-left-narrow[
.box-inv-5[Block all backdoor paths to identify the main pathway you care about]
]

---

# All paths

.pull-left.left[
.box-5.smaller[Education → Earnings]

.box-5.smaller[Education → Job connections → Earnings]

.box-7.smaller[Education ← Background → Earnings]

.box-7.smaller[Education ← Background ← U1 → Location → Earnings]

.box-7.smaller[Education ← Location → Earnings]

.box-7.smaller[Education ← Location ← U1 → Background → Earnings]

.box-7.smaller[Education ← Year → Earnings]
]

---

# All paths

.pull-left-narrow[
.box-inv-5[Adjust for **Location**, **Background** and **Year** to isolate the **Education → Earnings** causal effect]
]

.pull-right-wide[
<img src="04-slides_files/figure-html/edu-earn-adjust-1.png" width="100%" style="display: block; margin: auto;" />
]

---

# Let the computer do this!

.box-inv-5.large.sp-after-half[dagitty.net]

.box-inv-5.large[The **ggdag** and **dagitty** packages in R]

---

# How do you know if this is right?

.pull-left-narrow[
.box-inv-5[You can test the implications of the model to see if they're right in your data]

$$
X \perp Y\ |\ Z
$$

.box-5.small[X is independent of Y, given Z]
]

.pull-right-wide[
<figure>
 <img src="img/04/dagitty-implications.png" alt="Dagitty implications" title="Dagitty implications" width="100%">
</figure>
]

---

# Causation

.pull-left-wide[
<img src="04-slides_files/figure-html/causation-dag-big-1.png" width="100%" style="display: block; margin: auto;" />
]

.box-inv-5.medium[**X** causes **Z** which causes **Y**]

.box-5.medium[Should you control for **Z**?]
]

---

# Causation

.box-inv-5.medium[No!]

---

# Causation and overcontrolling

---

# Colliders

.pull-left-wide[
<img src="04-slides_files/figure-html/collider-dag-big-1.png" width="100%" style="display: block; margin: auto;" />
]

.box-inv-5.medium[**Y** causes **Z**]

.box-5.medium[Should you control for **Z**?]
]

---

# Programming and social skills

.box-inv-5.medium[Do programming skills reduce social skills?]

.pull-left[
<img src="04-slides_files/figure-html/programming-social-skills-1.png" width="100%" style="display: block; margin: auto;" />
]

.pull-right[
.box-5[You go to a tech company and conduct a survey. You find a negative relationship! Is it real?]
]

---

# Programming and social skills

.box-inv-5.medium[Do programming skills reduce social skills?]

---

---

---

# Colliders and selection bias

---

# Three types of associations

![](04-slides_files/figure-html/confounding-dag-1.png)

.box-inv-5.small[Common cause]
]

![](04-slides_files/figure-html/mediation-dag-1.png)

.box-inv-5.small[Mediation]
]

![](04-slides_files/figure-html/collision-dag-1.png)

.box-inv-5.small[Selection / endogeneity]
]