Discussion about this post

User's avatar
Joe Kim's avatar

Thanks for your response, Chris. I think we are perfectly aligned. The problem is that many will view different research methods as "good" vs. "bad". RCTs have (appropriately) held an esteemed place in the hierarchy of evidence but I think this has also led to an underappreciation of it's limitations and misunderstandings on how observational approaches (including natural or quasi experimental methods) can valuably contribute to increasing our knowledge base.

Expand full comment
Joe Kim's avatar

I read your NY Times Op-Ed piece with interest. Although I generally agree with your point on the value of natural experiments to potentially increase the internal validity of population-based research studies, your piece gives the impression that observational research methods are uniformly problematic. I think it is important to highlight that the validity of causal inferences from different study designs are on a spectrum. Even randomized controlled trials are subject to selection, information, and confounding biases, especially over follow-up. There are numerous examples where observational studies have provided important and valid insights on intervention/exposure/treatment effects, especially when methodologically framed as a target trial emulation (e.g., COVID-19 vaccine studies). I would also emphasize that the use of natural experiments in populations is a form of observational study, assuming the latter refers to situations where the intervention/exposure/treatment was not manipulated for the purpose of answering a research question. These issues speak to the importance of understanding the principles that underlie making causal inferences from data and appropriately deploying the tools at our disposal to make these inferences as valid as possible.

Expand full comment
1 more comment...

No posts