Large-N Observational Research Designs

Justin Murphy

Why do some states fail?

With this type of question, all we can do is select and compare observations

Hypothesis: Autocracies are more likely to fail than democracies.

 

1. Obtain a large sample of democracy and autocracies over multiple years each

 

2. Count the number of state failures in each

 

3. Compare the means

Simple Large-N Observational Research Design

What are some shortcomings?

1. Probably omitted variable bias

2. Maybe the difference is just random noise?

To mitigate omitted variable Bias

  1. Identify key omitted variable
  2. Measure it for all country-years
  3. Calculate it's unique correlation with state failure
  4. See if there is still unique correlation of democracy and state failure
  • When you do this with many cases using statistics, it's Large-N Observational Research
  • When you do this with a few cases "manually," it's called Small-N Observational Research

How to know if it is Random Noise?

  1. Calculate the standard error.
  2. "Standard error" is equal to standard deviation divided by the square root of the total number of units in the group.
  3. Check whether the error bars overlap.
  4. If they don't, it's a statistically significant difference (not reducible to random noise).