• Sofía Bahena, Ed.D. • IDRA Newsletter • June-July 2016
It is common knowledge that correlation does not imply causation. Mark Wilson (2014) humorously illustrates this point in a series of graphs depicting near-perfect relationships, such as the one between the divorce rate in Maine and per capita consumption of margarine in the United States (r=0.99) (see more at Tyler Vigen, n.d.). By even the most conservative of standards, this correlation would be deemed statistically significant; however, one would not argue that eating more margarine causes divorce. Yet, researchers sometimes make similar conclusions that imply causal relationships when in fact they are only correlational (such as in the word gap premise discussed on Page 3).
Confirmation bias may partially explain why we are inclined to confuse correlation with causation. Psychologists have written extensively about this widespread tendency to interpret relationships in a way that aligns with our preexisting beliefs (Nickerson, 1998). Because we are all vulnerable to this bias, it is important for both producers and consumers of research to be aware of confirmation bias and how to avoid it. For example, we can do the following.
Explore our own lens. The term reflexivity refers to an introspective process in which researchers are “attentive to and conscious of the cultural, political, social, linguistics and ideological origins of one’s own perspective” (Patton, 2002, p. 65). Though reflexivity is a process generally practiced in qualitative research, it is useful for quantitative work as well.
The way that statistical analyses are conducted and interpreted are just as informed by the quantitative researchers’ lens as is the work of an ethnographer. The same rationale follows for how readers respond to any given study’s findings.
Begin with a clear theoretical framework. In critiquing the perceived “language gap,” Dudley-Marling & Lucas (2009) warn against elevating method over theory, a process that ignores our innate biases and perspectives. They emphasize that “data collected by physical and social scientists only have meaning in the context of some theoretical framework” (p. 366). For this reason, it is important to draw from relevant research and begin with a strong theoretical foundation.
Theory is important in identifying the key research questions to ask, measures to collect, and hypothesizing relationships between variables of interest (Murnane & Willett, 2011). Likewise, when we read research, we must identify the theoretical foundation the authors are building on, testing or complicating.
Contextualize findings or conclusions. Though we may not be able to avoid confirmation bias completely, we can at least contextualize our findings or conclusions within our own lens and a broader theory. By simply acknowledging the ways in which our own perspectives influence our work and interpretations, we can better understand the relationships at hand and potentially discover new insights.
Causal inferences are justified, not by the strength of relationships, but by the design of the research study. How have the researchers been able to address alternative explanations or threats to validity? After all, “there’s no such thing as a philosophy-free science; there is only science whose philosophical baggage is taken on board without examination” (Dennet, 1995, as cited in Dudley-Marling & Lucas, 2009, p. 21).
Dudley-Marling, C., & K. Lucas. “Pathologizing the Language and Culture of Poor Children,” Language Arts (2009). 86(5), 362-370.
Murnane, R.J., & J.B. Willett. Methods Matter: Improving Causal Inference in Educational and Social Science Research (New York: Oxford University Press, 2011).
Nickerson, R.S. “Confirmation Bias: A ubiquitous Phenomenon in Many Guises,” Review of General Psychology (1998). 2(2), 175-220.
Patton, M.Q. Qualitative Research and Evaluation Methods, third edition (Thousand Oaks, Calif.: Sage Publications, (2002).
Vigen, T. Spurious Correlations, web page (Spurious Media LLC, no date).
Wilson, M. Hilarious graphs Prove that Correlation Isn’t Causation, Fast Company web page (Co.Design, May 13, 2014).
Sofía Bahena, Ed.D., is an IDRA senior education associate. Comments and questions may be directed to her via email at
[©2016, IDRA. This article originally appeared in the June-July 2016 IDRA Newsletter by the Intercultural Development Research Association. Permission to reproduce this article is granted provided the article is reprinted in its entirety and proper credit is given to IDRA and the author.]