NIH Study Suggests Measurement Bias in Common Child Behavior Assessment Tool

FOR IMMEDIATE RELEASE

Researchers identify less biased questions that could reliably capture childhood behavior problems

Scores from a commonly used measure of behavior problems in young children may be skewed depending on the primary language, education, and sex of the caregiver who fills out the survey, as well as the child’s age and race, according to new research from the NIH’s Environmental influences on Child Health Outcomes (ECHO) Program.

ECHO Cohort researchers analyzed data from caregivers who filled out the widely used Child Behavior Checklist (CBCL) 1.5–5 on behalf of 9,087 young children ages 18 to 71 months from 26 ECHO research sites across the United States. The caregiver-reported survey evaluates a range of behavior problems in young children.

"Understanding children's behavior can help us identify potential issues in their development and mental health down the road,” said Shuting Zheng, PhD of the University of California, San Francisco. “To do this effectively, we need reliable ways to measure their behavior equivalently for children from diverse backgrounds."

The study found that how caregivers respond to the questions on the measure was biased the most by the language used to complete the survey, followed by factors such as the caregiver's education level and sex and the child's age and race. These biases persisted after accounting for mental health disparities between these groups. Researchers noted that some information could get lost in translation and people from different cultures could understand child behavior problems differently. The gender of the person answering the questions and their level of education also had an impact and researchers pointed to parents’ expectations about how children should develop as possible influences in how they answered the survey questions.

ECHO Cohort researchers were then able to identify some CBCL questions less affected by measurement biases introduced by sociodemographic factors but which still reliably captured childhood behavior problems. The subset of questions that showed little bias across survey-taker demographics still captures a wide range of behaviors across internalizing (e.g., emotionally reactive, withdrawn) and externalizing problems (aggressive behaviors and attention problems).

Going forward, ECHO Cohort researchers plan to apply these questions, evaluating their reliability in identifying children with clinically significant behavior problems.

“Finding questions with less measurement bias across socio-demographic groups helps researchers and clinicians measure behavior problems in different groups of kids of varying ages and family backgrounds more accurately,” Dr. Zheng said.

Dr. Zheng and Somer Bishop, PhD of the University of California, San Francisco, and Maxwell Mansolf, PhD of Northwestern University led this collaborative research that was published in the Journal of Child Psychology & Psychiatry Advances.

Zheng, S. et al. Measurement Bias in Caregiver-Report of Early Childhood Behavior Problems across Demographic Factors in an ECHO-wide Diverse Sample. Journal of Child Psychology & Psychiatry Advances. DOI: 10.1002/jcv2.12198.

###

About ECHO:

Launched in 2016, the Environmental influences on Child Health Outcomes (ECHO) Program is a research program in the Office of the Director at the NIH with the mission to enhance the health of children for generations to come. ECHO investigators study the effects of a broad range of early environmental influences on child health and development. For more information, visit echochildren.org.

About the NIH: NIH, the nation’s medical research agency, includes 27 Institutes and Centers and is a component of the U.S. Department of Health and Human Services. NIH is the primary federal agency conducting and supporting basic, clinical, and translational medical research, and is investigating the causes, treatments, and cures for both common and rare diseases. For more information, visit.

Media Contacts

For information or to request an interview, contact Rebekah Yeager,  rebekah.yeager@nih.gov.

If you are not a member of the media, but have a general inquiry, please contact  NIHKidsandEnvironment@od.nih.gov.

Connect With Us

echo connectorCheck out the recent issues of our bi-monthly newsletter, the ECHO Connector, for program news and the latest research findings.

To receive the ECHO Connector through email, subscribe here.

Follow @ECHOChildHealth for the latest ECHO Program updates on Twitter.

ECHO Researchers Propose Eye-Tracking Technology Can Provide a Non-Biased, Cross-Cultural Tool for Assessing Attention Patterns in Young Children

Collaborative ECHO research led by Sara Nozadi, PhD, of the University of New Mexico and Andréa Aguiar, PhD of the University of Illinois Urbana-Champaign suggests that eye-tracking technology can be used as a robust measure across diverse populations and settings to assess attention patterns linked to socially withdrawn behavior in preschool children. Their research, titled “Cross-Cultural Applicability of Eye-Tracking in Assessing Attention to Emotional Faces in Preschool-Aged Children” is published in the Emotion Journal.

Humans tend to give more attention to emotional information than neutral information. Past research indicates departures from this pattern typically occur in children with socially withdrawn behaviors. However, most of this research has been very limited and only conducted in controlled settings among primarily White children from urban areas.

In this study, ECHO researchers wanted to determine the extent to which two eye-tracking–based measures could be used cross-culturally to assess attention biases and how these biases might relate to socially withdrawn behaviors in children.

The researchers leveraged ECHO’s large sample size and diverse population to compare information from children in two ECHO cohorts that were very different demographically, geographically, racially and culturally. They analyzed data from 125 children aged 3.5 to 5.5 years from the Navajo Birth Cohort Study (NBCS) and 70 children aged 3.8 to 4.0 years from the Illinois Kids Development Study (IKIDS). NBCS is a cohort of indigenous children who live in rural Tribal lands across New Mexico, Utah, and Arizona and tend to have relatively low socioeconomic status (SES). In contrast, children from IKIDS are primarily Non-Hispanic White and high-SES, living around the University of Illinois at Urbana-Champaign campus in central Illinois. Children in the NBCS historically have not been included in developmental studies, including those investigating attention biases, whereas the IKIDS sample closely resembles the children assessed in most of the previously published eye-tracking studies of attention bias toward emotional stimuli.

Researchers had both groups of children complete two eye-tracking tasks that measured their attention to photos of human faces with positive, negative, and neutral expressions. Indigenous children were assessed in different locations in the Navajo territory, whereas IKIDS children came to a research laboratory in the University campus. Mothers also reported on their child’s socially withdrawn behaviors using the Child Behavioral Checklist (CBCL)—a standardized survey commonly used in research and clinical settings to evaluate risk for behavioral and emotional problems in children.

Attention patterns were similar across the two samples of children irrespective of differences in their cultures, demographics, or assessment settings. Overall, children paid more attention to emotional faces compared to neutral faces, and they were particularly quick to detect angry faces—because this expression may signify a threat. These findings replicate results of previous research with both adults and children. Indigenous children with socially withdrawn behaviors were more likely than other children in the study to avoid emotional faces—a response that has been observed in children with higher levels of socially withdrawn behaviors.

“This work shows that eye tracking can be used to measure patterns of attention towards emotional facial cues—both adaptive patterns and those that underlie socially withdrawn behaviors—in young children,” said Dr. Nozadi. “Because eye tracking technology does not rely on limited clinical observations or parents’ reports, it can be used as a more objective and less biased measure across various settings and communities.”

Moving forward, the researchers will look at how the responses they see in these young children change with age and determine whether eye tracking continues to be an unbiased tool for assessing attention throughout development.

Read the research summary.

ECHO Researchers Create a Common Measure of Adult Depression

Courtney K. Blackwell, PhD
Courtney K. Blackwell, PhD

More than 264 million people experience depression worldwide. In the United States alone, more than 17 million adults experience depression each year. And women, especially those who are currently or recently pregnant, are at higher risk for a depressive episode. As such, regular depression screenings for women during and after pregnancy is a recommended practice, and doctors may want to continue screening women throughout their lives. Consistent measurement of depression throughout a woman’s lifetime can be difficult because the measurement tools used around pregnancy are scored differently from the common measures of adult depression. Linking these measurement tools together will help doctors track the changes in depression across a woman’s lifetime.

To address this research gap, Courtney K. Blackwell, PhD, and her ECHO colleagues conducted a study aimed at linking the Edinburgh Postnatal Depression Scale (EPDS)—a measure of depression in women during or after pregnancy—with the Patient-Reported Outcomes Measurement Information System Depression (PROMIS-D), a general measure of adult depression.

This study included approximately 1,200 mothers from two ECHO sites in the Northern Plains of the US who each completed the EPDS and PROMIS-D surveys. All data were collected using an online survey. Most of the participants were white, but the sample also included American Indian/Alaska Natives. On average, the participants were 35 years old, and about half had a 4-year college degree or higher.

While the EPDS includes more specific measures of postpartum depression symptoms, the study found that the EPDS and PROMIS-D gave depression scores that were highly correlated for women around pregnancy. Both surveys asked questions about increased depressed mood and decreased interest and engagement in daily activities. The researchers were able to convert EPDS scores to PROMIS-D scores for consistent tracking of depression in women throughout adulthood. The converted scores held for different ages, ethnicities, races, and study sites.

“To know about someone’s depression across time, it is important to have a way to use data from different measures that may have been used at different timepoints,” said Blackwell. “This study provides a way to convert scores from a measure used during and after pregnancy to a measure that is used at any time in adulthood.”

The conversion table may also be useful to ECHO researchers who want to standardize depression measurements from multiple cohort sites. Additional studies may examine how different depression measures can be converted using similar methods.

The conversion table is available on prosettastone.org.

 

Access the research summary.

Richard Gershon: Advances in Pediatric Measurement Science: Implications in ECHO

return to discovery home

Advances in Pediatric Measurement Science: Implications in ECHO

Speaker:

Richard Gershon

Northwestern University

 

 

 

 

 

*If you would like CME credits for attending this presentation, please email Katherine.Simos@duke.edu*

About the Speaker:
Dr. Gershon is the Vice Chair for Research in Medical Social Sciences at Northwestern University’s Feinberg School of Medicine. Dr. Gershon has had a robust career in Patient Report Outcomes (PRO) and Performance Outcomes as he has acted as the principal investigator for the NIH Toolbox for the Assessment of Neurological Function and Behavior and the NIH Roadmap Patient – Reporting Outcomes Measurement Information System (PROMIS) Technical Center. He has served as PI, MPI or co-investigator for over 70 NIH, DoD, and Foundation awards.

Topic Areas: PRO Core, Person-reported outcomes, Patient-reported outcomes

Date: Wednesday, January 8 11, 1 to 2pm