Survey data about non-binary people: a critical tool to inform discussions about policy, resource allocations, and other issues that affect them

by Pisana Ferrari – cApStAn Ambassador to the Global Village

“International Non-Binary People’s Day” is celebrated ever year on the 14th of July since 2012 and is aimed at raising awareness around the issues faced by people who do not identify as male or female. The issue is gaining increasing traction in the current debate around the theme of Diversity, Equity and Inclusion (DEI). A number of prominent public figures are opening up about their gender identity (and pronouns they wish to be addressed by), and many organisations worldwide are working towards the use of more inclusive language. The UN, WHO, the International Labour Organization, European Parliament and European Commission, and a number of professional associations, universities, companies (the latest to date being Ford and GM, in July 2021), major news agencies and publications have introduced internal guidelines for the gender-neutral use of language. Things are moving also in terms of including gender identity questions in surveys. According to the UCLA William’s Institute GenIUSS (Gender Identity in U.S. Surveillance) questions related to gender identity “are a step towards making transgender and other gender minority people visible and countable in nationwide surveys” and that “the resulting data could provide transgender and other gender minority people with a critical tool to guide local and national discussions about policy, resource allocations, and other issues that affect them.” What are the best terms to use when asking about gender identity in surveys? A number of researchers, including the GenIUSS group, are testing a two-step method which envisages asking two separate questions: “sex assigned at birth” and “current gender identity”. Regarding the subsets for gender identity many include an “other” option (see also our article on “Non-binary” measures of sex/gender in surveys”). We found the approach used by Pew Research on this issue very interesting and, in particular, the decision in future surveys to limit most of the Center’s U.S. surveys to the single gender question.

A survey on how to best ask for gender identity

In June and July 2020, Pew Research conducted an online opt-in survey to test two alternative gender question options in English and Spanish. Of the 5,903 people taking the survey 4,931 responded in English and 972 responded in Spanish. Respondents were randomly assigned to receive one of three sets of gender questions after receiving other demographic questions. One group of respondents (n=1,654) received the standard Pew Research Center question (“Are you male or female?”). Another group (n=2,611) received a question about sex, followed by a question about gender: 1. “Were you born male or female?” 2. “Do you describe yourself as a man, a woman, or in some other way?” The phrase “some other way” also included a text box for respondent specification. To minimize the number of adults who might incorrectly be classified as transgender by accidentally chose the wrong answer they asked a third follow-up question of those whose responses to these two questions did not match: 3. “Just to confirm, you were born [male/female] and now describe yourself [as a man/as a woman/in some other way]. Is that correct?” If respondents said their answers were not recorded correctly, they were given the opportunity to re-answer both questions. A final group (n=1,638) received the same questions as the second group, but in reverse order (gender first). This group was also asked to confirm if their answers to the two questions didn’t match. At the end of the survey Pew asked all respondents whether they thought their questions were biased (and, if so, why), followed by two questions about their views on transgender rights and offering a third gender option on forms and online profiles.

The new gender question is more inclusive and improves data accuracy

Regardless of the way in which Pew asked respondents their sex/gender, they observed consistent distributions among men and women, suggesting little effect of the question wording for most people. However, among English-speaking adults who received the new gender questions, 1.5% selected the third, non-binary response option or chose a gender inconsistent with their sex, suggesting that the inclusion of the third option had the intended effect on being more inclusive and improving data accuracy. Also, providing a third gender option didn’t appear to be off-putting.Fewer than 1% opted not to answer the gender question and the revised gender question also didn’t cause more people to stop taking the survey before completing it.

Asking about sex in addition to gender isn’t necessary

Based on comments collected at the end of their experiment, Pew found that fewer than 1% of respondents mentioned the sex and gender questions in any way. Among those who did, 30% expressed frustration over the presence of multiple questions or concern about the use of the term “born” in the question on sex. Pew says that the inclusion of a question about respondent’s sex was not necessary for most of the Center’s research, which focuses predominantly on gender. They do construct weights based on population estimates by sex but the correlation between gender and sex suggests that gender can reliably be used as a proxy for sex for this purpose. Another important consideration is respondents’ time: “If we don’t need to ask a question, we won’t.”