Patient experience metrics have become increasingly important in evaluations of health care organizations and physician performance. Although such measures have been touted as a way to make objective comparisons of performance, they are subject to many of the same biases as other survey instruments, including gender bias.

A total of 320 surveys were conducted between February and October 2020. Surveys included vignettes describing different scenarios, and respondents were asked to rate the vignette physician in each scenario on 1 of 3 themes: listening, time, or courtesy. Three vignettes per theme were used. Half of the surveys used a male physician and half used a female physician. Using tests of difference, we compared the ratings of male and female vignette physicians. We also used a statistical technique known as anchoring vignettes to show how respondents’ ratings of vignette physicians related to their ratings of their own physicians.

In all 9 vignette scenarios, the male vignette physician was rated more highly than the female vignette physician. These differences were statistically significant in 2 of 9 scenarios. Male vignette physicians were given more top-box ratings than female vignette physicians. Anchoring vignettes showed a statistically nonsignificant association between vignette ratings and ratings of respondents’ own physicians.

Our findings revealed a pattern of higher ratings of male vignette physicians when compared to female vignette physicians, which may translate to ratings of patients’ own physicians. These findings suggest that current methods to evaluate patients’ experiences with their own physicians may disadvantage female physicians.