Researching the Research: How to Understand and Evaluate Psychological Studies

Given my work as a clinical psychologist, I try my best to stay up on the latest research across several areas, so that I can help parents make well-informed decisions regarding their children’s treatment. Recently, I was tuned in to NPR and was intrigued to hear a Dutch researcher being interviewed with regard to a study about the role of diet in Attention Deficit/Hyperactivity Disorder in children. I turned up the volume and listened carefully, hoping to hear something that would prove interesting and useful for my clients. The statement that the researcher went on to say astounded me. In no uncertain terms, the researcher concluded that a majority of children with ADHD symptoms were exhibiting those symptoms as a result of dietary sensitivities. I waited for the typical caveats that researchers are obligated to provide regarding limitations of their results. I waited for the interviewer to ask some pointed questions regarding the nature of the research that led to these conclusions. However, neither was forthcoming. ( If you are curious, you can read the transcript of the interview on NPR – the study was discussed on All Things Considered on March 12, 2011. A response evaluating the conclusions to this study, put out by CHADD, can be found here:www.chaddleadershipblog.blogspot.com/2011/03/elimination-diets-for-adhd-not-ready.html

Given the age we live in, parents have access to an astounding amount of information, which is both extremely helpful and also potentially enormously overwhelming. It can be very difficult to sort through claims that are made and posted on the internet or via other news media regarding a particular disorder and/or treatment. Here are a few guidelines to help parents and other concerned individuals to be informed consumers of psychological research:

1. How was the sample selected for the study? How representative was this sample of the population that was intended as the focus of research? For example, studying children who have AD/HD – Combined Type may not yield results that are applicable to children with other forms of AD/HD.
2. How were variables defined and measured? For example, if the researchers report “improvement” in symptoms, what constituted improvement and where did this information come from? Studies that rely on reports from individuals that know that the individuals are participating in a treatment study, and that know what type of treatment was given, are considered less reliable. These reports may be inadvertently biased by beliefs that the treatment is working. Studies that include reports from raters that are “blind” to the conditions of the study are more reliable.
3. Who was compared in the study? Was there an experimental group and a control group? Was there a “placebo control”? Often individuals participating in a treatment study improve by virtue of being involved in a study, and not due to the treatment itself. Thus, it is very important to compare a treatment to other comparable types of interventions, not just to no treatment.

When in doubt, consult nationally known organizations for the latest information regarding a given treatment. Examples of reliable websites include the American Psychological Association website (www.apa.org), the NIMH website (www.nih.nimh.gov) and for AD/HD, the CHADD website (www. chadd.org ).

Kathleen Boykin McElhaney, Ph.D
Licensed Clinical Psychologist