13 - Response Rates and Response Bias: 20 Years of Pediatrician Surveys

Poster Number: 13
Publication Number: 13.414

Elizabeth A. Gottschlich, American Academy of Pediatrics, Itasca, IL, United States; Mary Pat Frintner, American Academy of Pediatrics, Itasca, IL, United States; William H. Burr, American Academy of Pediatrics, Itasca, IL, United States; Tylar Kist, American Academy of Pediatrics, Naperville, IL, United States; Chloe Somberg, American Academy of Pediatrics, Itasca, IL, United States; William Cull, American Academy of Pediatrics, Itasca, IL, United States

Background: Response rates to surveys have been declining the last several decades, including among physicians, which raises questions about the representativeness of survey results.

Objective: Track pediatrician survey response rates from 2000-2019, measure whether the likelihood of responding to surveys differed by age, gender, and geography, and test whether these indicators of response bias changed with response rates.

Design/Methods: We examined 68 surveys of pediatricians conducted by the American Academy of Pediatrics (AAP), 2000-2019. 48 surveys targeted non-retired US post-resident AAP members and 20 targeted US graduating pediatric residents. Select demographic characteristics for the full eligible survey samples, including nonrespondents, were available in the AAP administrative database. Mean age, percent female, and percent in the Northeast (NE) region were compared for each survey’s respondent group and the full sample. We defined response bias for a given characteristic as the absolute difference between the respondent groups’ value and the full sample’s value. A bias value of 0 reflects no response bias for that indicator.Linear regression examined whether survey response rates have changed over time. To test if survey response varied by age, gender, and region, one-sample t-tests compared the bias value for each of the 3 characteristics to 0. Linear regression also tested the relationship of response rates with response bias.

Results: Across survey years, response rates ranged from a high of 72% (2000) to a low of 47% (2019), with a mean of 56%. Rates declined significantly across years (β=-.57, p< .001); Figure 1.The mean age of the full samples was 43.0 years and 43.1 years for respondent groups. Mean portion female was 59.8% in the full samples and 62.2% for respondents. On average, the full samples and the respondents had about one quarter in the NE (24.2% and 24.1%, respectively). One-sample t-tests showed survey response varied significantly by each of the 3 characteristics (all p< .001). However, the average response bias values were substantively small; Table.In linear regression, age bias was associated with lower response rates (β=-.36, p< .01; Figure 2); gender and region biases were not. However, even for surveys with the lowest response rates, the age bias never exceeded 1.2 years.Conclusion(s): Pediatrician survey response rates have continued declining over the past 20 years. Lower rates were linked to greater response bias for age but not gender or region. Even at the lower end of response rates in the 40-50% range, we found minimal bias for the available indicators.
Figure 1. Trends in Pediatricians’ Response to Surveys by YearNote: Each circle represents 1 survey of pediatricians. Survey response rates significantly decreased between 2000 and 2019.
Table 1. Descriptive Statistics for Age, Gender, and Region Response Bias Values