For the last several years, I have been making myself available for free, first-come-first-served meetings that I call office hours. They’re 30 minute 1:1 virtual meetings, guided by the participant, on topics ranging from career advice to applied behavioral science.  And they’re motivated by the belief that when we require introductions or other forms of social proof to gate access to our time, we replicate the existing systemic biases inherent in those social systems.

In 2021, we released our first Diversity Report, a concentrated effort to make sure that this system is in fact serving a broad range of underrepresented people.  In our 2022 edition, we provide updates on our commitments from last year, refreshed data, and make our commitments for next year.

I use the plural repeatedly throughout this report.  That’s because making office hours happen is a team effort; even if I’m the one actually showing up, there is a tremendous amount of work from a number of people to make sure that we follow up on action items, share our learnings, and prepare this report every year.  In particular, Melanie Perera from Oceans and Zsanelle Lalani are instrumental in connecting job seekers with opportunities, sending out materials, and preparing the Mentor Minutes for social media. I am grateful for all they do and hope you take a moment to celebrate them.

Before looking at the results, a few quick notes on methodology.  To gather the data, we set up a Google Forms survey and then used Zapier to automatically email participants with a link after each meeting, along with an end-of-year followup reminding them of the survey.  In addition to asking for qualitative feedback to help us improve, we asked basic demographic questions about age, gender identity, sexual orientation, ethnicity, etc.  No questions were required, all were multiple choice with options presented in random order, with “Other” and “Prefer not to say” options included.

For 2022, given immense changes in my personal life (including moving across the country and my co-parent’s cancer diagnosis), my ability to do office hours consistently was somewhat curtailed – we went from ~800 meetings last year to more like ~550 meetings this year. We received 150 survey responses, giving us a response rate of ~27%.

That is roughly the same as last year and generally quite high.  Typical survey response rates are less than 5%, so we can make some reasonable assumptions that this data is representative of the larger population of participants. But you could always make an argument that some segments are more likely to respond, so take it all with a grain of salt.

On to the 2022 data!  Each section has a two paragraph format: first data, then interpretation.  There will be a separate section at the end for commentary and 2023 commitments. I’ve included the change from the 2021 numbers in parentheses so you can also see trends. I’m open to questions and feedback on the analysis, as well as suggestions on what commitments you’d like me to make; just shoot me an email. 

Age

The mean age in respondents was 34 (+0) and the median was 33 (+2).  The best comparison is probably the median age of the US working population, which is 42, so overall we’re skewed a little younger.  However, the standard deviation was around 9 (+0), with participants ranging from 17 to 69, so there was a good bit of variability.

I see this as an improvement from last year.  Not only did the range of ages go up, but so did the average and median age.  Why is that a success?  Because younger people have greater access to formal mentorship programs than do older people, a general upward trend in age means we’re serving the underserved.

Gender

Among respondents, 56% (+1) identified as women, 39% (-3) identified as men, and 5% (+2) identified as non-binary/genderqueer. This is a fairly large overrepresentation of women, who generally are less likely to participate in the workforce.  For non-binary people, the estimate jumped from 1% to 5% this year which is in line with our number; I expect this is a more accurate number for the population rather than a genuine increase in gender identification.

This is a mild improvement from last year. It isn’t that I don’t think men are deserving of help (especially since they may be underrepresented for other reasons) but since my office hours are an attempt to democratize access and systemic sexism is an issue, this is a positive trend.

Sexual Orientation

78% (+3) of respondents identified as heterosexual, with 22% (+7) identifying as some form of LGBQ.  This is significantly different than the base rate of 93% and 7%, respectively.

I am very publicly liberal and work in and around fields that are generally more liberal (tech/design/etc.), so these numbers aren’t entirely atypical in my larger community. That said, there are still significant biases present in our often heteronormative culture, so I’m generally happy to see overrepresentation here.

Race and Ethnicity

53% (+13) of respondents identified as White (base rate 77%), 7% (-8) as Black or African American (base rate 13%), 19% (-11) as Asian (base rate 6%), 1% (+1) Native American (base rate 2%) and 20% (+5) as More Than One Ethnicity (base rate 2%).  In addition, 12% (-1) identified as Hispanic or Latino/a/x (base rate 18%), with Mexican, Mexican American, or Chicano/a/x as the largest group.

This is obviously disappointing; a 13 point jump in White is a lot, even if it is still significantly below the baseline. This increase, however, is correlated with an increase in meetings with people based outside the United States; 58% of international meetings were White, compared to just 47% of US-based meetings.

Other

20% (-3) of respondents are first-generation Americans (base rate 14%), while 23% (+5) are first-generation college graduates (base rate 35%).  27% (-13) view themselves as underrepresented in their field, while 9% are living in poverty and 5% identify as disabled. 41% (+3) did not add any additional tagging. 74% are currently living in the United States, while 26% live in one of eighteen other countries with Canada, Germany, and the UK being the most represented.

Commentary and Commitments

In 2022, it was difficult for me to react to these findings because the only data I had to compare them with was the base rate. In 2023, I have last year’s data as a benchmark and generally come away with mixed feelings.

On the positive side, age, gender, and sexual orientation all showed year-over-year increases in representation.  While the gains were modest, they trended in the right direction and over a much larger sample size, giving us increasing confidence that we are serving those who need it most.

The biggest disappointment is obviously the increase in White-only participants.  Even with the growth in international audience, the purpose of my office hours is to reduce systemic bias and I simply can’t do that if I’m not meeting with those who face biases related to race and ethnicity.  While the rate of straight, cis White men who didn’t identify with any underrepresented categories remained the same this year at 13%, this is a place where I expect to see year-over-year improvements and so no change simply isn’t good enough.  It is on me to take action in 2023 to change this trajectory.

We made two important commitments in our 2021 report, both of which we were able to honor.  As promised, we began releasing clips from our office hours across multiple social media platforms and will continue to do that into 2023.  We also exceeded our $5K committed budget and spent closer to $12K supporting the needs of individuals who had challenges this year; while we have not yet set a budget for 2023, we will continue to push forward on that front.

There were a number of other operational changes to office hours this year, across the technology, processes, and team available.  For example, we introduced a tracker that allowed us to more easily view the applications of those who were looking for work and make referrals where appropriate; we assisted in 61 job searches in 2022.

I continue to believe that public, open office hours on a first-come, first-served basis can be a lever for reducing some forms of systemic bias.  Office hours are not just an opportunity for mentorship but a chance to deploy resources against real needs, whether that is using social capital to make a referral or financial capital to provide pilot funding and even essentials like food and interview clothing.

For 2023, we’re going to concentrate on the scalability of our impact by addressing two key shortcomings:

• Reusable content.  Reviewing our Vowel recordings from this year, it is clear that I’m spending a tremendous amount of time answering very similar questions. We’re working on everything from written guides and short videos to a chatbot trained on the office hour transcripts to make sure that we can serve more people.  I don’t intend to spend less time doing office hours but hopefully these solutions allow us to concentrate that time on increasingly personal questions and specific situations.

• Increased tooling.  While the ability to distribute Vowel recordings and shared notes has been a tremendous help to individual participants, process improvements like the job tracker have allowed us to more quickly provide asynchronous help in the moments when people need it most.  Rather than confining our impact to the office hours, we’re going to intensify our tooling, ranging from lightweight spreadsheets to custom web-based calculators, in order to create new scaffolding for people to help themselves.

I fundamentally believe that transparency helps drive accountability and accountability allows for autonomy.  My hope is to be able to offer an updated diversity report yearly for as long as I am able to continue doing office hours at this pace and with this team.  As I mentioned earlier, I’m open to questions and feedback on the analysis, as well as suggestions on what commitments you’d like me to make; just shoot me an email.

Side Note: When I released the diversity report for 2021, I wrote “Sometimes, doing the right thing feels absolutely ridiculous”. In August of 2022, I left my global executive role at CapGemini because they refused to release basic diversity data. Putting myself out of work during a recession feels like one of those ridiculous things and I am acutely aware of the privilege that allowed me to do it.  But accountability matters.  Working toward change from within a company is important but when business leaders refuse outright to even consider change, it is time to go.  So I’m going to keep releasing these reports yearly and I hope CapGemini and others consider doing the same.

I believe that behavioral science, correctly applied, can change the world. But, as with any emerging discipline, there is a period of self-definition in which people fight (with varying amounts of actual animosity) about who can claim what title and where the borders of the field are. 

Personally, I’ve largely been uninterested in the debate about who can and cannot call themselves a behavioral scientist (though to be clear, as a non-PhD, it benefits me not to start drawing lines). But that’s different from what it actually means to be doing behavioral science; as the name of the field suggests, it is the behaviors that should concern us. So I have become increasingly interested in how we might break down the various components of behavioral science into smaller units of work that could be credibly offered independently, while firmly maintaining the integrity of the behavioral science process as a whole.

To begin, let’s be clear that I am actually talking about applied behavioral science, which is explicitly concerned with changing behavior. This is distinct from academic behavioral sciences (like social psychology, behavioral economics, etc.), which further our understanding of the basic principles that underlie human behavior. That doesn’t mean academic folks don’t care about change or that applied folks don’t care about knowledge, just that each prioritizes one over the other. In my case, as an applied behavioral scientist, that means that while I still sometimes publish peer-reviewed papers, my primary work is changing the behaviors of populations.

My simple definition of applied behavioral science has always been “behavior as an outcome, science as a process,” which has the benefit of being easy to explain to people without exposure to the discipline and sounding pithy when you say it in a presentation. But if you’re trying to buy behavioral science services, or understand how you might begin to build them internally, that definition isn’t terribly useful. 

To make it more practical, I propose a four-stage model below that balances an understanding that each part is essential with the need to break it down into units of work that can be spread across internal teams and external vendors when necessary. But be warned: each handoff increases the potential for loss, particularly when there is an incomplete understanding of the adjoining stages. A tightly integrated process managed by people who understand the end-to-end process will always have the greatest likelihood of creating meaningful behavior change; that we can name the parts should not detract from the need for a whole.

• Strategy: the defining of a desired behavioral outcome, with population, motivation, limitations, behavior, and measurement all clearly demarcated. Plain version: figuring out what “works” and “worth doing” mean in behavioral terms by collaborating with stakeholders.
• Insights: the discovery of observations about the pressures that create current behaviors, both quantitative and qualitative. Plain version: figure out why people would want to do the behavior and why they aren’t already by talking to them individually and observing their behavior at scale.
• Design: the design of proposed interventions, based on behavioral insights, that may create the pre-defined behavioral outcome. Plain version: design products, processes, etc. to make the behavior more likely.
• Evaluation: the piloting (often but not always using randomized controlled trials) of behavioral interventions to evaluate to what extent they modify the existing rates of the pre-defined behavioral outcomes. Plain version: figure out whether the products, processes, etc. actually change the pressure, make the behavior more likely, and do so at a magnitude that is attractive for scaling.
• Behavioral Science: combining all four of those processes. Plain version: behavior as an outcome, science as a process.

Step 1: Behavioral Strategy

Because the process is linear and each step requires that the previous step was done (although not necessarily by the same person), we need to start by defining the behavioral outcome we want to achieve. In the latest version of the Intervention Design Process (or IDP; the applied system I propose in my book), we do that using a behavioral statement: When [population] wants to [motivation], and they [limitations], they will [behavior] (as measured by [data]). Arriving at that statement is deceptively hard work and requires running a disciplined process with stakeholders to define each of those variables. But done correctly, it paints a picture of the world we want to create when our interventions are working. 

Given that the process prioritizes what we want the result to be rather than the interventions that actually create the result, my proposed term is behavioral strategy. While it doesn’t have to include a cost/benefit ratio that defines how much an intervention can cost relative to the impact that it has, certainly knowing this can shape the rest of the process and allows stakeholders to more clearly understand the actual stakes. 

Inside a company, both Strategy and Product teams try to answer this question regularly, although they often express it in imprecise, non-behavioral terms that create misalignment later. Externally, a strategy firm like McKinsey could likely spin up a unit that did this work in a reasonable way but like internal teams, they currently tend not to focus specifically on behaviors and don’t offer this as a service today.

Step 2: Behavioral Insights

The next step in the IDP is understanding the distance from the world we want by understanding the pressures that create the world of behavior we have today. Insights can be both quantitative and qualitative, so collectively I propose behavioral insights and then splitting as needed: qualitative behavioral insights and quantitative behavioral insights, since there are specialists that concentrate on one approach or the other. 

Existing user researchers and data scientists frequently do this work (Spotify has Quantitative User Research, for example), and as long as they’re doing the work with an explicit emphasis on generating insights to change behavior, these teams could slot in here. If you wanted to buy it as a service, IPSOS’ behavioral science team seems to do behavioral insights as a specialized form of market research that focuses on behavior and other agencies may be able to provide insights if specifically pointed toward behavioral outcomes.

Step 3: Behavioral Design

Having mapped the behavior we want and understanding why it doesn’t yet occur, in the IDP we next get into pressure mapping and intervention design. There are lots of ways to create behavioral interventions that don’t use pressure mapping, like design thinking, but ultimately we are always trying to generate proposed interventions that may change behavior. I say “proposed” and “may” because while we have supporting evidence (because the design process is based on the behavioral insights we defined above), we haven’t actually tested whether the interventions create the behavior. 

Design and Product departments do this within companies today, although often lacking the behavioral focus, so it seems appropriate to call this behavioral design. And an agency like Fjord could potentially do this externally, so long as they are given an articulated behavior outcome and the relevant behavioral insights (neither of which they are likely to create themselves).

Step 4: Behavioral Evaluation

Finally, we have the evaluation of the proposed interventions, to see to what degree they actually create the outcome articulated in the behavioral strategy. While this is called impact evaluation in the non-profit world, behavioral testing builds on the more widely understood experimentation that is used in most for-profit companies. The theoretical gold standard is a randomized controlled trial, in which participants exposed to the intervention are compared against a control group, but that may not always be feasible; remember, in applied behavioral science, we only need to be as right as the cost/benefit ratio dictates. In a perfect world, doing this process also results in the observation of additional behavioral insights (because trying to change a system often reveals underlying truths about it) but I don’t think we should try to make this a specific requirement of this process. 

Very few companies actually run rigorous pilots today, although it does happen in some Product and Data Science organizations (and Marketing loves non-theory-driven RCTs in the form of A/B tests), so this is probably the largest potential growth area for behavioral science as a whole. In the non-profit sector (where impact evaluation is sometimes built into grants), an agency like Social Impact will do an RCT on your interventions for you, if you’re careful to make sure they translate “impact” in behavioral terms.

Combined: Behavioral Science

To me, behavioral science requires the combination of all of the above. If you can’t define a behavioral outcome (AKA don’t do behavioral strategy), then you miss out on the whole point of “behavioral” in this discussion; you can’t run a scientific process if you can’t measure what works, and you don’t know what “works” means if you don’t define it. 

Similarly, you could run pilots and measure behavioral outcomes but without behavioral insights, that’s not science: your interventions aren’t necessarily designed based on replicable understandings (my favorite example of this is Marissa Mayer testing 41 shades of blue at Google; because there was no theory behind the iterations, you could only know what worked in that limited moment but not why) and so if they don’t work, you’re not actually any closer to something that does. It is only when all four processes come together that you truly get to both of the words in the term behavioral science…and neatly arrive back at “behavior as an outcome, science as a process.”

Some people who currently offer behavioral science services are going to hate this taxonomy, because it threatens their identity, both personally and professionally. And I understand that feeling: removing ambiguity can feel like a loss, when clarity reveals you’re only covering some of the territory. And not offering some services isn’t always by choice; for example, I’ve often heard consultants complain that they can’t sell behavioral impact evaluation to clients because they already “know” it will work after the behavioral design phase.

But the purpose of this guide is arriving at a shared understanding of applied behavioral science and its components, and part of that is recognizing that no one piece of the field is better than any other. There is no shame in only doing part of it, as long as we clearly explain the other parts and push the importance of doing the full process. By creating areas of intersection and smooth handoffs, we can better allow for specialization and move the world incrementally closer to behavior as an outcome, science as a process. And that’s work worth doing, in any form.

Side Note: My belief in this model is why I’ve decided to join frog as the Executive Director of Behavioral Science. My role is two-fold: help my fellow frogs apply behavioral strategy, insights, design, and impact evaluation in their projects and help our clients build their own applied behavioral science capabilities. While I’ve worked hard to evangelize the field broadly in my previous roles (including writing Start At The End, which was as close to a handbook as I can get, and doing 30+ talks a year), ultimately my career to date has been about creating a long series of behavioral interventions that accomplished internal business goals. In contrast, at frog I’ll be focusing specifically on behavioral science as a process, both internally and externally. As we see more senior behavioral scientists within large companies, we have the opportunity to leverage existing cross-disciplinary expertise to further support that work. And frog, particularly as part of CapGemini Invent, is the right place to do that. The agencies I mention in the examples can all learn to do parts of the behavioral science process. But because they typically do only their siloed step, they think of their stage’s deliverable in isolation. At frog, because we can and have done the full cycle, we know each step is just a milestone, so we can take a more holistic view and plan our work to naturally connect to the next necessary step. And through our Org Activation practice, we can teach organizations alongside projects to help them grow their own capabilities. Behavioral science doesn’t belong to frog – it belongs to everyone. And it is with that belief firmly in mind that we look forward to growing this discipline together.

Recently, I accepted a new job offer.  And I was excited, as so many folks are when they find meaningful work.  It felt like such a great gig: my entire reporting line would be women!  It would be global in scope!  It would be spreading behavioral science among Fortune 500 CEOs!  I turned down three competing offers, told my family, and started working on plans for the first few months.

Then the employment agreement arrived.

It had the standard bevy of non-competes and non-solicits, which are problematic in their own right but still a standard that hasn’t been broadly challenged (although the FTC is working on it).  And yes, they wanted to own my IP, which has its own thorny definitional challenges when your job is uncovering and appplying underlying truths about human behavior (“Who owns science?” can turn into a very long conversation).  But those issues were navigable and in twenty years of working, despite all my apprehensions, I’d never actually had any problems arise.

Yet the packet also referenced agreeing to policies, like a code of conduct, that weren’t attached.  So I asked the executive recruiter if they could send them over, because one of the things drilled into me by my father was to never sign a contract I didn’t understand (FBFP, or “fucked by fine print”, was a common condition in rural Oregon).

“I’m sorry, our policy is not to share our policies outside of the company and since you don’t work here yet, we can’t send those to you.”

Ruh roh.

That stance isn’t actually all that uncommon, as the standard of many enterprise companies is to be closed by default, because they believe it minimizes risk.  In this case, risk that they’ll get bad press or lawsuits around something in the policies.  Or risk that candidates will read the policies and refuse to join, rather than reading them once they’re already dependent on the paycheck and cognitive dissonance changes their mind (literally).

And I get it, I really do.  Even though I believe employment policies should be open by default because public scrutiny is actually the best protection against catastrophic risk, I recognize it is something upon which reasonable people could disagree.

But closed by default isn’t just about legal risk; it is also about creating inclusive workplaces.  In the case of my offer, it turns out that joining would require me to stop tweeting (except to retweet officially sanctioned company propaganda…I mean, uh, very valuable thought leadership pieces) and blogging (because I might leak the secret recipe to behavioral science on an unapproved channel; its “behavior as an outcome, science as a process”, just FYI).

That was enough for me to turn down the offer, because I’m privileged enough to have other options and money to fall back on if I didn’t.  My new employer also asked me to agree to their code of conduct as part of my employment but when I asked to read it, they simply sent it over.  And just like my previous boss, Vivek Garipalli at Clover Health (“I don’t care if other companies literally follow you around the office every day; they can’t execute like we can.”), my new boss believes that public science is a good thing: he likened it to football, where playing in public helps the sport evolve.

But not everyone gets a happy ending.  Sometimes, “my poverty, but not my will, consents.”  And as a white guy, it is a) likely that the code of conduct I never got to read is built around my cultural norms and b) I probably wouldn’t even get in that much trouble for violating it, just because of how accountability works in corporate America.  But that is dramatically different for many, many people.  Often those with the fewest options are also those the environment is least inclusive of. 

And yes, it is possible that, tweeting and blogging aside, the policies I never ready were very sensible.  But closed by default means we can’t know and in the absence of evidence of the contrary, we should assume the norm.  And unfortunately, as evidenced by the preponderance of white males at the top of the social pyramid, racism and sexism and other forms of bias are the norm.

If you think that a bad code of conduct sounds far fetched, remember that Google is just now acknowledging that people have the legal right to talk about their salaries.  And we had to pass laws so that black people could wear their hair naturally at work (and they aren’t even universally adopted yet) because 80% of black women felt pressured to change their hair style in order to fit in.  Does the code of conduct mention salary data or hair?  No idea, because you can’t read the policy until after you join and I didn’t.  But again, until we specifically know that something is an exception, we should assume the norm.

In the end, closed by default policies minimize some risks but create others.  You lose talent that won’t sign in the blind or refuses to work in an environment that doesn’t value their ability to have a public opinion.  You decrease diversity by preventing the formation of an open, inclusive environment.  And given the preponderance of evidence that open environments are associated with profitability (Satya Nadella’s tenure at Microsoft is an excellent case study), closed by default creates very real profit risk.

We need to be default open.  To share our code of conduct, when people ask.  To publish research and processes, because execution is really the only moat.  And if the sharing sparks controversy or others iterate on it, we can use that feedback to build something better.  Because we can’t have it both ways, preaching innovation through failure but hiding out of fear of our own failures.  Truth will out; out yourself.

Side Note:  There are a host of industries that profit on ambiguity by trading on information asymmetry, either directly or indirectly.  But because of the endowment effect, asymmetry is hard to perpetuate because once we have access to information, it instantly becomes more valuable because we psychologically “own” it. That is why workplaces will always drift toward more liberal policies; once you’ve had access to more relaxed rules around social media, spending, dress code, etc, the price of giving that up will simply get higher and higher and higher. Thus shifting to a liberal workplace early is actually a competitive advantage, even if Jamie Diamond doesn’t see it yet.