What we’ve learned from seven years of working to make RC 50% women, trans, and non-binary

In April of 2012, we announced our goal to make RC 50% women. Seven years later, we are close to reaching an improved version of this goal: 48% of new Recursers in 2019 so far identify as women, trans, or non-binary¹. This post is a summary of what we’ve tried, learned, and accomplished over the past seven years, as well as our overall strategy and why we choose to prioritize this work.

Outline

• Our strategy
  1. Get as strong and diverse a range of applicants as possible
  2. Minimize bias and evaluate everyone on the same admissions criteria
  3. Build and nurture an environment where as many kinds of people as possible can thrive
• Results and parting thoughts
• What about race, age, and other demographic factors?

Our strategy and why gender and other kinds of diversity are essential to RC’s success

Note: While the general approach outlined here can help increase many kinds of diversity, this post is primarily about gender diversity.

We believe nearly every aspect of RC gets better when RC becomes more diverse. The core experience of attending a batch and being part of our community, as well as our business model (recruiting), are all functions of the number, quality, and diversity of the people involved. This is what we mean when we say RC is community-driven. My cofounders and I have experienced RC across 60 batches: some with significant gender, racial, age, and other forms of diversity, and others with very little diversity. We believe firmly that the former are a better experience for everyone.

Additionally, we want RC to be a place where everyone can focus their time and energy on becoming better programmers, not worrying that they don’t belong or feeling like they have to represent their entire race or gender because they’re in the extreme minority. We focus on diversity so Recursers can focus on programming.

One set of things we want all members of our community to share are our admissions criteria. We want everyone here to be smart, pleasant, intellectually curious, self-directed, and to enjoy programming and want to get dramatically better at it.

The strategy I describe in this post informs nearly everything RC does, from how we approach marketing to how we structure our retreat and community. It boils down to: 1) Get as strong and diverse a pool of applicants as possible, 2) minimize bias and evaluate everyone on the same admissions criteria, and 3) build an environment where as many kinds of people as possible can thrive.

Most companies focus (if at all) on the first part of this process, and do little for the second and third parts. Getting a diverse range of applicants is essential, but it’s pointless if you then put them through a biased or ad hoc selection process, only to end up in an environment where only some groups can flourish or even want to remain.

1. Get as strong and diverse a range of applicants as possible

Since we take active, deliberate steps not to discriminate based on demographics during our admissions process, the makeup of our retreat and community is almost perfectly reflective of our applicant pool. If 5% of RC applicants identify as Black and 30% identify as women, then we can confidently expect the percentages of people in our upcoming batches who identify as Black and as women to be about 5% and 30%, respectively. As such, a core part of our strategy is biasing the top of our funnel — that is, getting more women and people of color to apply.

Here are some of the things we’ve tried.

We give money to women and people from other traditionally underrepresented groups. In April of 2012, we partnered with Etsy to fund living expense grants for women who can’t afford to attend RC otherwise. While RC is 100% free for all participants, living in New York is expensive. In 2014, we expanded our grants program to include people who identify as Black, Latina/o, Native American, and Pacific Islander.

By 2015, it had become increasingly hard to convince companies to contribute to our grants program, and we decided to begin funding grants ourselves, directly from RC’s operating budget. We have now disbursed over $1.5M in grants.

Grants have had a bigger impact on RC’s gender diversity than anything else we’ve tried. Continuing to provide grants to applicants who can’t afford to attend RC without them has been challenging but possible because we choose to prioritize it. For four months last year, my cofounders and I chose to take 60% pay cuts rather than cut our budget for grants. We were able to do this not just because we cared but because we knew that cutting grant funding would make our core product — the experience of attending a batch of RC — worse, and decrease our longterm odds of success.

We try to write clearly and frequently about our values. Writing is one of our organizational strengths, and in an otherwise unscalable business, writing lets us reach people around the world with a fixed investment of time and zero marginal costs. Next to the value of putting your money where your mouth is, this is the most important thing we learned from our partnership with Etsy. After starting our grants program, we saw an increase in applicants who identified as women and who did not need or request a grant. They said they applied because of the work we were doing and the clear way we explained it.

We’ve tried to critically self-assess the messages we are implicitly and explicitly putting out into the world, and we’re willing to make small and big changes. For example, we changed our company’s name four years ago in part because our old name (“Hacker School”) had unproductive connotations.

We’ve offered merit-based Fellowships. Late last year, we offered merit-based Fellowships of up to $10,000 to women, trans, and non-binary people working on open source projects, research, and art. Our announcement got a tremendous response, and our Winter 2 batch ended up being nearly 40% women, trans, and non-binary people, compared to the less than 20% we projected before announcing our Fellowships. The batch was also larger than any of our other recent batches, which is good for both our business and the experience of the people in the batch. (For more, see Alicia’s recent blog post.)

We’ve worked to make RC an experience people love and want to tell other people about. The number one way applicants hear about RC is from an alum who recommended it to them. The number two way is a recommendation from a friend or colleague who didn’t go to RC. Most encouragingly, applicants who identify as women, trans, or non-binary are 33% more likely to have heard about RC via word-of-mouth than cis-male applicants. Because of this we have experimented with a number of ways for our community to help spread the word about RC, but none have come even close to the impact that “just” having an experience people love and want to tell their friends about has.

Informational chats with prospective applicants. For the past several years, I’ve had a standing offer to have an informational chat with any woman or person of color that a Recurser thinks would make a good addition to our community. The chats are usually short and I answer questions about RC and our admissions process. I’ve done dozens of these now, but I have not tracked them. However, a number of the people I’ve met with have gone on to apply and attend RC. Speaking with prospective applicants has also been a good source of feedback, since it’s given us insight into parts of our admissions process we didn’t or don’t explain clearly enough on our website.

Not everything we’ve tried has worked. For instance, we’ve experimented with personal outreach to individual women and people of color. We did this in an ad hoc fashion for several years, and then last year tried a more systematic approach. Our team found over 100 women and people of color who we thought would make promising applicants, and sent personalized emails encouraging them to consider applying (while these were cold emails, we personally found, vetted, and emailed the people we contacted to ensure we weren’t being spammy). We tracked the (considerable) time we spent on this, and several months later, looked at how many of the people we’d emailed had applied and been admitted. We determined that the results weren’t worth the effort required and have stopped doing this.

Similarly, a few years ago one of my cofounders and I visited a dozen college campuses and gave talks about RC to their Women in Computer Science clubs. We met some great people (a few of whom have since become Recursers!) but we found that, at our size, the benefits of this weren’t enough to warrant the significant amount of time and money we put into traveling around the country.

Another example of an unsuccessful experiment is RC Start, a program we ran for most of 2016. We offered free, one-on-one mentorship to new programmers by matching them with RC alums for three sessions of programming help and advice. One of our hypotheses and reasons for trying RC Start was that we thought the applicant pool would have a higher percentage of women than the applicant pool for our retreat. Unfortunately, this turned out to not be true. Additionally, we realized that the program was a strategic mistake. My cofounder Dave explained this in detail in Why we stopped RC Start.

Though it’s too soon to say whether they’ll be successful or not, there are also a number of new things we’re trying. One of those things is targeted advertising. Before last year, we had spent effectively no money on advertising. Currently, we are experimenting with ads directed exclusively at female-identified Facebook and Instagram users. There are two things that make this particularly challenging. First, coming to RC is a major life decision, and so there’s commonly a long time between when people hear about us and when they apply (in fact, the majority of people who come to RC heard about it more than a year before they applied). Second, we’ve chosen not to use some of Facebook’s more powerful but troublesome features, like Custom Audiences (which would require that we upload personal information to Facebook) and Pixel (which would send all of our site traffic to Facebook).² We don’t know yet whether or not we’ll be able to get new qualified applicants cost effectively through ads.

We’re also working to develop ongoing relationships with CS departments and Women in CS groups. This started on a small scale organically as people came to RC, had great experiences, and spread the word to their schools and departments. Recently, we’ve begun a more concerted effort to establish ongoing relationships with groups at a number of universities around the US.

We’re also investing for the long term. Last year we launched Joy of Computing, a site that features technical work by members of the RC community. We designed it to give the public a way to see the incredible diversity of delightful work Recursers do. Our long term goal is to build Joy of Computing into a new way for people to hear and learn about RC. We don’t expect this to have a meaningful impact on our applications for at least a year, and probably longer.

2. Minimize bias and evaluate everyone on the same admissions criteria

We have spent many years refining our admissions process and eliminating bias wherever we can. Here are some of the most effective things we’ve done so far.

We use pseudonyms and hide demographic information. In 2014, we updated our admissions review software to replace people’s names with pseudonyms (e.g., “Keyboarding Animal” or “Temperature Jeans” instead of “José Smith” or “Kimberly Lin”) and to hide demographic information during the first part of our application review. While far from perfect, this helps us shape our initial assessment of applicants without knowing their demographics.

We have explicit admissions criteria. This is one of the most basic things organizations can do to reduce bias in any admissions or interview process, and most companies do it half-heartedly or not at all. If you don’t have a clear idea (written down and understood by everyone involved in the process) of exactly what you are trying to select for, how can you fairly and consistently evaluate candidates, and how can you know if your process is filtering for what you want it to? We publish our admissions criteria on our website.

We document and explain our process. Once you have criteria by which you’re trying to evaluate applicants, you need to create and document a process for evaluating those criteria. You must also be specific about what precisely you mean by your criteria. For instance, if you say you’re hiring for “go-getters,” what does that actually mean to you, and does it mean the same thing to your colleagues? In addition to sharing our admissions criteria publicly, we have many thousands of words of internal documentation of the what, how, and why of our process for our interviewers to draw upon.

We train our interviewers. Many companies will say hiring is one of the most important things they do, and then ask their employees to just start doing it. You want your process to be as consistent as possible across interviewers so that regardless of if an applicant speaks with interviewer A, B, or C, the outcome is the same.

We prioritize facts over opinions. When we write notes and grades for our admissions interviews, we think it’s more valuable to write about facts than opinions, or even conclusions. Consider the difference between the notes “the applicant can’t program at all” and “the applicant asked to use Google to look up how to write a for loop. After 10 minutes, they were unable to write a syntactically correct loop in Python.” The ideal interview notes would include a complete transcript of the interview, since you can always reconstruct an opinion from the facts, but you usually can’t reconstruct the facts from an opinion. We also take the context of a person’s experience and past educational opportunities into account when making decisions. For instance, while RC welcomes and admits people with a wide range of programming ability, we expect different things in our interviews from people who have been Staff Engineers than from people who are only six months into teaching themselves to program.

To collect interview notes, we’ve written custom software and have a standard format, which includes space for each criterion to includes the facts the interviewer sees as relevant, as well as radio buttons for whether they got a positive signal, negative signal, or no signal for that criterion.

We record interviews for quality control and training. In 2017, we began to record video of our interviews (with applicants’ consent, of course). These videos have been particularly helpful when training new interviewers. For instance, before conducting or even observing any interviews, prospective RC interviewers watch recordings of past interviews and then write up notes and grades as though they had conducted the interviews themselves. Then, with RC faculty, they talk through and compare their assessments and decisions to the ones made by the actual interviewers.

We have ongoing support and a process for giving interviewers feedback. Our interviewers meet regularly to discuss challenges, ask questions, and give feedback based on their recent interviews. We also regularly audit all interviewers’ notes and grades and provide feedback on how the notes are written, as well as their admissions decisions.

3. Build and nurture an environment where as many kinds of people as possible can thrive

There’s no sense building a pipeline if it’s leaky, or a funnel if it only leads to a cesspool. No marketing or admissions efforts matter if you have a toxic environment that hampers people and drives them out.

Here are some of the things that have helped us foster a healthy environment.

We have explicit social rules. We have a set of four lightweight social rules that help make RC a friendly, productive place to program and grow. These include No well-actually’s and No feigning surprise. Our social rules are intended to give people a shared framework and an easy mechanism for addressing small annoyances before they fester and become big problems.

We have a code of conduct, a system for reporting violations, and a response protocol. Where our social rules are meant to help with annoying behavior that most of us do from time to time, our code of conduct is meant for serious infractions that might warrant removal from our community. Our code of conduct is more than just words, we also have an established process for how people can report issues, and how we will handle them when they do.

We are intentional about how we welcome people and make them part of our community. We want people to feel safe and secure at RC so they can focus as much of their energy as possible on their own growth and work. Having a genuine community requires that people know the other people around them, and that everyone shares some fundamental values and purpose.

We start working towards this before people even begin RC. We run RC Gateway, which gives admitted Recursers the option to request an introduction to an existing member of our community based off their interests and background (e.g., someone might ask to speak with an alum who came to RC after leaving the industry to raise a family, or an alum who had worked as a web developer but focused on machine learning at RC).

The first day of RC is unlike every other day. While nearly all of RC is unstructured and Recursers decide for themselves how they want to spend their time, the first day of a batch includes a welcome breakfast, tours, an introductory ceremony, and a skit about our social rules. This is because our goals for the first day are different than for other days. When people first come to RC, we want to prioritize helping them get acquainted with our physical space, meet others in their batch, and set a healthy tone for the remainder of their time here.

We continue to make RC more accessible. In addition to attending RC for six or 12-week batches, people can now come for just one week and become an alum and lifelong member of the community. This is a change we made in early 2018, with the goal of making RC more accessible to people who can’t attend for a full six or 12-weeks. We hoped this would make it easier for parents to attend RC, and hypothesized the applicant pool might skew more female than the pool for our longer retreats. The latter turned out to be false (men and women apply to our different length retreats at effectively the same rates) but anecdotally the applicant pool has skewed older and included more parents. (We don’t have data here because we don’t track age or parental status in our admissions process.)

We have also changed some of our policies and updated our space to make RC more family-friendly. We have added a lactation and wellness room, and updated our guest policy to allow parents to bring their children with them to RC when childcare falls through.

Results and parting thoughts

Before we set our goal to make RC gender balanced in early 2012, only 5% of Recursers were women, trans, or non-binary. Today, 34% of our community of over 1,400 people are. Even more promising: Of the nearly 150 people who have already joined or who have confirmed for an upcoming batch this year, 48% identify as women, trans, or non-binary. As promising as this is, we’ve seen these numbers fluctuate. We know it will take continuous investment and work to have any chance of consistently achieving a gender-balanced environment at RC.

We have been able to prioritize this work over an extended period of time because we have built a team, culture, and company aligned with it. Caring, while important, can never be enough if your institutional priorities and incentives are working against your good intentions. The things you actually prioritize will always win in the long run. If you are serious about prioritizing diversity and inclusion at your company, you must make it inseparable from your product and overall success, and recognize that prioritizing anything necessarily implies de-prioritizing other things.

What about race, age, and other demographic factors?

Currently, 6% of our community identifies as Black, 6% identifies as Latina or Latino, and 0.2% identify as Native American. This is more racially diverse than the tech industry at large — for example, just 1.5% of the employees Google classifies as “tech” employees identify as Black.³ But Google’s is a low bar to compare ourselves to.

While we have made some progress in the past year — 7% of new Recursers so far in 2019 are Black, compared to 5% in 2018 — RC is still overwhelmingly white and Asian. We know that this will not change unless we invest the same amount of time and money in making RC more racially diverse as we have in making it more gender balanced.

We expect the challenges we’ll face to vary among different demographic groups. For example, Black applicants are slightly less likely to have heard about RC via word-of-mouth than our applicant pool as a whole, while women, trans, and non-binary applicants are much more likely to have heard about us from friends. We assume this is reflective of the small number and percentage of Black people currently in the RC community, and the systematic racism and racial segregation that persists in the US.

We do not have demographic data for age. Anecdotally and by observation, we know the majority of people who come to RC are in their 20s and 30s, but have ranged in age from 16 to early 60s. We believe RC would be a better experience for everyone if we had more older and more younger people in our community.

1. Our initial goal in 2012 was to make RC 50% “women.” Over the years, as we’ve learned and thought more deeply about gender diversity at RC, we’ve refined this goal to “make RC 50% people who identify as women, trans, or non-binary.”↩

2. This might seem odd or even hypocritical since we do use Google Analytics. We agree that sending all our traffic to Google is just as bad as sending it to Facebook. We plan to migrate to our own internal analytics eventually, and in the interim, thought it was better to not make the same mistake twice.↩

3. According to Google’s 2018 report.↩