Recently, I was talking to a company that helps employers gather feedback from their in-the-field workforce in order to boost employee retention. And while they have a ton of customers fielding surveys, the product folks were concerned that managers don’t actually seem to be looking at the feedback dashboard.
My suggestion was to use a Max Headroom: take a few customers and do away with the dashboard entirely, instead having one of the product managers meet with them weekly to deliver the feedback. That way, the SaaS company could eliminate the dashboard itself as a variable and focus on the core behavior of managers responding to employees.
The tactic is named after a character from the 80s played by an actor in prosthetics but billed as computer-generated; the goal is to have a human frontend with a computing backend. And it is similar to another testing methodology, the Mechanical Turk, where the frontend is computing but the backend is actually human.
When I suggest this to folks, the objection is usually that humans aren’t scalable. Which isn’t entirely true; for many SaaS companies, a bit of quick math shows that you can have a CS person take quite a lot of meetings and still be profitable.
But it also doesn’t matter. The point of a Max Headroom isn’t to find a scalable business model; it is to reduce the inhibiting pressures of a behavior to as near zero as possible. A meeting to go through the dashboard together doesn’t quite accomplish that; there are still plenty of inhibiting pressures like time, mental energy, etc. But it is meaningfully easier than trying to interpret and action a dashboard on your own.
The much larger, unspoken concern is: what if that doesn’t work? What if, even with human-delivered feedback, managers are still unwilling to do what it takes to retain employees?
And the sad answer might be that it is time to pivot. If you try a Max Headroom and your target audience still doesn’t do the behavior, even with inhibiting pressure reduced in unsustainable ways, then you may need to address your underlying behavioral statement.
There are exceptions where adding a human increases friction, like calling a taxi company instead of using the Uber app. But that is more typical in consumer products and less in SaaS, where value is usually produced by removing the cost of having a human do something.
Max Headroom tests are underutilized in product creation, because of the bias toward seemingly-scalable software as the solution to everything. But in an AI era, where we need to better decouple backends and frontends, they are more important than ever. Because if a human reading your AI’s script isn’t useful, your AI probably isn’t either.
https://mattwallaert.com/wp-content/uploads/2018/12/victory.jpg1200900matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2025-01-09 14:23:232025-01-09 14:23:23Add a human to the market-product equation
Movies will have you believe that all fights happen in bars. But to me, few places feel quite so ready to break into a spontaneous brawl as the line to get on an airplane.
And sure, there is sometimes utility to boarding the plane early. When my son was young, it was helpful to have an extra few minutes to get him settled. And you’re guaranteed to have overhead bin space if you need it.
But most bags do make it onboard, the seats are assigned, and we’re all leaving and arriving at the same time. The value of being in Group C instead of Group D is marginal at best.
So then why do people crowd?
Some of it is about social standing. In a million ways, humans pay to demonstrate our privilege and that is unlikely to end anytime soon.
But there is another factor at play: people like to feel like they are getting their money’s worth. And airlines have clearly connected boarding early to value. Early boarding is associated with Executive Diamond 100K Premier customers, higher-priced ticket types, and even explicit charges if you fly Spirit or Frontier.
So if you’re sitting in the waiting area and they call your group but you don’t board, you feel like you’re giving up value that you’ve already paid for. And humans are extremely sensitive to that form of loss.
This is, of course, a deliberate strategy by the airlines – controlling who gets to go first costs them nothing, yet creates the illusion of value. But there are plenty of times when “use it or lose it” is an unintentional consequence of designs that creates unnecessary feelings of missed value.
All-you-can-eat subscriptions like Netflix are an easy example. By putting so many options on the front page, they maximize the chances you’ll find something to watch. But they also highlight all the things you might want to watch (and more importantly, are paying for) but will never actually have time to enjoy.
Ditto for multi-function devices. There are a plethora of tools in my garage that can absolutely do more than I am capable of (I’m looking at you, compound sliding miter saw) and every time I use them for my basic needs, I’m keenly aware that I’m somehow missing out.
In a world where products compete on value, it often feels like the optimal is simply stuffing in as many sources of value as possible. And it is great that Netflix has many options and my miter saw can do many things.
The key to balancing the gain of more features and the loss of not using them is to control how visible those features are. Netflix can make fewer recommendations. The miter saw can hide the more advanced controls. As applied behavioral scientists, we can sometimes create greater perceived value simply by being more direct with the highest value features and deemphasizing the visibility of those that most people will never get value from.
https://mattwallaert.com/wp-content/uploads/2018/12/eagle-eye.jpg1200900matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2025-01-09 14:18:402025-01-09 14:18:40The illusion of value in product visibility
A few years ago, I was talking to an entrepreneur working on an easy way to share STD test results and he said something that stuck with me: HSV is a dumb reason to break up with someone, because about 50% of the adult population in the US has it, and so it is 50/50 that even if you dump this person, you’ll have to dump the next person too.
To me, this is a very clever reframe. It doesn’t try to debate anything about the morality of breaking up with someone or minimize HSV itself. Instead, it just focuses on the probability of an event occurring in the next instance.
This is on my mind this week because of a conversation with a mentee about her immigration status. She’s looking to change jobs, so she’s been applying and getting interviews, but doesn’t get selected to continue because she only has 3 years left on her OPT visa.
So I invited her to do one of my favorite activities: a reverse roleplay. She got to be the interviewer, I got to be her. And when she asked me the question about my immigration status, I used a similar reframe:
“The average tenure of someone at my level in a tech company is less than three years, so if you pass me over because of my OPT, it is just as likely that the person you choose to hire will actually leave in the same amount of time. So why not consider whether I’m the best person for the job instead of focusing on how long I’ll do it?”
Her job dropped.
Note that I’m not arguing the fairness of immigration status being a form of legal discrimination. And I’m not trying to convince the interviewer that I’m so much better than the next candidate that they should overlook the deficiency. Instead, I’m arguing it isn’t a deficiency in the first place because it is true for the majority of the population.
This works for all sorts of perceived shortcomings in an interview. For example, take a job description I read once that had both “Ten years of applied behavioral science experience” and “Ten years of insurance experience” as requirements – a combination that literally no one had at the time.
Rather than arguing that whichever you don’t have isn’t important or that you can make up for it in other ways, you can just rely on probability. By pointing out the non-overlap of those experience sets, you can level the playing field by arguing that everyone is going to have to learn something in order to do the job and so it might as well be you.
As the joke says, you don’t have to be faster than the bear, you just have to be faster than the other person they’re chasing. By pointing out equivalencies, you can reset the frame on yourself as a candidate to minimize shortcomings. You’ll still have to highlight your strengths, but it can sometimes eliminate at least some forms of bias.
https://mattwallaert.com/wp-content/uploads/2018/12/the-guide.jpg1200900matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2025-01-09 14:14:372025-01-09 14:14:37Preparing for an interview? Reframe your shortcomings using probability
Over the weekend, the former CFO of Oura Daniel Welch posted about the success of narrowing the product focus to a subset of women’s health. And one of the comments, from Diana Torgersen, caught my eye:
“Wonderful – but still wondering how Women’s health is a “narrow focus”? By focusing on women you’re focusing on 51-52% of the population so it seems it’s not that narrow, merely the other half of the population.”
The reason it perked me up is because a) that math is wrong when you’re talking about product narrowing and b) it is a mistake that tons of product people make, resulting in insufficient levels of focus.
Take Fitbit as a representative of the default mainstream wearable device (you could insert Apple, Samsung, or dozens of others in here if you want). Torgersen is right: they generally try to target 100% of the market, with an implicit focus on men (because the patriarchy sucks).
But they do more than that: they try to target 100% of the market across multiple features, including sleep, nutrition, exercise, stress, etc. That is 100% x 5 categories, or 500 product points.
If Oura had taken that approach, they would simply have halved the population but kept the same features; a Fitbit Fem that exclusively focused on women would still have 50% x 5, or 250 product points.
But Oura didn’t just narrow the population: they also narrowed the features by focusing on just fertility, pregnancy, and menopause. That’s 50% x 3, or 150 product points. And in reality, because fertility, pregnancy, and menopause are non-overlapping, it is actually just 50 product points: 50%/3 (each member of the population is in only one of the three states) x 3.
I suspect that Torgersen was really just trying to use a rhetorical question to make the point that focusing on women isn’t narrow and more people should see women as an important market; a true statement, for sure.
But it inadvertently also made a point about how people conflate population and features. When seeking product focus, many product folks will narrow population OR features, instead of population AND features. And the “and” is where all the magic happens.
Now to be fair, Welch’s post was really about expansion; Oura had already done 100 products points of everyone x sleep. But you could argue they started with 50 product points, by doing men x sleep; Welcome acknowledges that in the beginning, it was seen as a very masculine device. They could (and perhaps should) have started with women x sleep, but would still be 50 product points. And that was with a $2.3m seed.
Every product starts by changing one behavior for one person. Over time, you’ll remove Limitations and add Populations and Motivations as you expand. But start with one.
https://mattwallaert.com/wp-content/uploads/2018/12/itsaparty.jpg9001200matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2025-01-09 14:05:182025-01-09 14:15:37The myth of narrowing focus in product success
There is a routine that plays out in the workplace time and time again. I ask HOW you’re doing, you tell me WHAT is happening, and then we proceed as if those two things are exactly the same.
We don’t say “I’m sad because XYZ.” We just say XYZ, as if any reasonable person should know how we will feel in reaction to that specific combination of events. But just because I know what is happening around you doesn’t mean I know how you feel; humans are variation machines and what feels good to me doesn’t always feel good to you.
So a few years ago, I started explicitly trying to draw out the use of emotion words. First by moving away from coded wording; instead of asking how you’re DOING, I ask how you’re FEELING, even though we typically tend to think of those as the same thing. And reactions vary in a typical 2×2: not/noticing, not/emotional.
Some people don’t seem to notice and don’t change their wording. And that needs to be alright; inclusivity doesn’t mean that we demand emotionality from people who aren’t comfortable using that framing in the workplace. Ditto for noticing and not changing; if I offer the prompt and they choose to take it a different direction, it is on me to respect that.
Others don’t seem to notice but implicitly switch to an emotional frame. And that’s really lovely: I’ve learned a lot about colleagues that I wouldn’t normally have known. In particular, I hear more about racism, sexism, and classism in the workplace this way – opening the door to emotion also opens the door to events that are likely to have emotional consequences.
And then some people notice, pause, and then switch to an emotional framing. This is honestly my favorite, because I imagine that it is now slightly more likely that they’ll do this for someone else, making the whole workplace a bit more humane.
The other thing I’m trying (and largely failing, because my superpower is having a two hour long conversation entirely about you) is to use more emotion words myself. Now I could argue that rather like a noticing/not emotional person, I should have the right to keep my emotions private. And I do.
But creating change often isn’t about exerting our right to comfort; it is about being personally uncomfortable in order to make other people more comfortable. Think of it like standing up to give someone your seat on a bus, except you’re standing out emotionally so others have the chance to do so as well.
Maybe this isn’t the hill you want to struggle up; there are plenty of other ways to make more inclusive workplaces that don’t evolve emotionality. But consider trying it as an experiment: at least for today, use and solicit more emotional wording and then reflect on how it changed your interactions. Do you feel…happier?
https://mattwallaert.com/wp-content/uploads/2018/12/directing-traffic.jpg1200900matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2024-12-12 15:17:202024-12-12 15:17:20Using more emotions words in business creates an inclusive environment
TLDR: Almost all economic increases come from change and so profit maximalism demands a world where everything changes, all of the time. But change has a cost for individuals and communities; be cautious, in yourself and your designs.
Recently, I was introduced to the Benedictine ‘Vow of Stability’: a promise that, upon entering a monastic community, you will remain a part of it for life. Philosophically, it is rooted in a belief that instead of seeking out an environment of perfection, you have an ethical responsibility to improve where you are. And practically, it was historically good to have monks that weren’t monastery hopping, trying to find greener grass.
My introduction to the VOS was secular; it came from someone who agreed early on in their marriage to pursue stability as a family wherever they could. They recognized that voluntary changes often created additional stress and even as changes could create a step up in circumstance, there was a hidden cost of turbulence.
As they told me about the choices they made, it struck me just how much of modern capitalism requires paying a change fee in order to advance.
The easiest example is their home, which has quadrupled in value over the 20 years they’ve been in it. They have often contemplated selling in order to harvest that value but that would require destabilizing their family. So instead they bought an empty lot nearby, with the goal of eventually building a home they can comfortably retire in while remaining part of the community.
Slightly more nuanced is education. Their school district has consistently been in the bottom 25% of their already less-than-stellar state. But rather than move, they have sought out enrichment activities for their kids and volunteered to make things better.
I’m not suggesting that their VOS-inspired choices are right. I’ve moved for job opportunities, to give my son an education that fits him, and to be in places that support my individual and family happiness. And I’m content with the choices I’ve made.
But I was in NYC last week and it was just so easy to get business done, because my network there is expansive. When we moved to San Diego, I essentially hit the reset button, both personally and professionally, as did the rest of the family. As individuals, it is worth considering whether all change is worth it.
Perhaps more importantly, when we design interventions, we must be on guard against demanding too many change fees from those we intend to benefit. Turbulence upsets not just the individual balance but also the group; when my son changes schools, it is not only he who loses friends – each of his friends also loses him. What can feel small rapidly becomes larger.
VOS behaviors are all around. When a stock that pays a dividend rather than trying to constantly escalate in price. When a factory stays rather than offshoring. Many changes are choices and we could all stand to choose a little more carefully.
https://mattwallaert.com/wp-content/uploads/2018/12/two-5.jpg9001200matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2024-11-22 02:02:582024-11-22 02:02:59Change has a cost for individuals and communities
TLDR: Learning is not a smooth curve; we frequently grow in spurts and jumps. But those rarely align with external validation like graduation or licensing. Credentialism isn’t just inequitable, it is a business and cultural liability. And there is a market opportunity in refusing to accept the bias.
Recently, there has been a trend in mental health to switch to associates: psychologists who have not yet been fully licensed and cannot practice on their own, without supervision. This is driven largely by cost, as associates are 20-40% cheaper than their fully licensed peers, largely because they cannot create their own practices and must sacrifice some money to a mediating entity.
The requirements for an associate to become fully licensed vary by state but generally include an exam, some number of supervised hours practiced across various populations, and an annoying amount of paperwork that gets returned to you if you misspell something. Bureaucracy gonna bureauc.
Obviously, experience matters. In study after study, we can prove that people do actually get better at things over time; some version of “practice makes perfect” arose in pretty much every language for a reason.
But that improvement is lumpy. When we look at learning, people tend to proceed in epochs, with long periods of level performance between jumps in ability. And those jumps are hard to pin down; if you do a cohort analysis, when people make a leap up is largely unpredictable. People who are learning and working together don’t necessarily hit milestones at the same time.
And so it isn’t necessarily true that you’re a better therapist today than you were yesterday and it certainly isn’t true that you’re a better therapist the day after the state declares you fully licensed than you were the day before. No one magically produces better work simply because they walked across a stage and got handed a degree.
And yet the day after graduation, your chances of getting a job go up dramatically. Credentialism, or the tendency to rely on formal qualifications over demonstrated ability, is rampant in hiring simply because across a large number of applicants, it is very difficult to find a better proxy.
The problem, of course, is that not everyone has equal access to credentials. Licensing fees can run into the thousands of dollars in some professions, let alone the time investment of a bewildering number of forms. Credentialism tends to amplify sexism, racism, and classism, because the systems that bestow credentials are themselves sexist, racist, and classist.
It is easy to look at associates as inferior and lambast the companies that are increasingly relying on them. Because hiring isn’t the only place credentialism occurs; consumers can just as easily interpret a credential as a valid signal of quality, without actually determining whether there is a real difference.
But ultimately, that drives up market pricing. Consumers who put false faith in credentials end up spending more, without any additional return. And because consumers are buying based on the credential, people are forced to get credentials to increase their wages; the tail is wagging the dog.
As employers, we have structural tools at our disposal: temp-to-hire as a method of determining actual ability, removing degree requirements, etc. And as consumers, we can use reviews as an alternative to credentials. But all of these are choices: ultimately, it is on us to give people the chance to demonstrate their ability to perform beyond their credentials.
And there is plenty to be gained. Besides combating inequity, in a market economy that overvalues credentials, there is a price opportunity in not making the same mistake: you can get more and better, for cheaper, if you’re willing to fight the bias that others are falling victim to. Fighting credentialism is a moral imperative but if you need an economic justification, it is certainly there.
https://mattwallaert.com/wp-content/uploads/2018/12/beLIEve.jpg8991200matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2024-11-22 02:01:442024-11-22 02:01:44Credentialism isn’t just inequitable, it is an opportunity
TLDR: Even if our processes buffer us from academic misconduct, we still need to be conscious of both the practices and people that we platform. Above all else, we must be applied, behavioral, and scientific.
Yesterday on BlueSky, Neil Lewis Jr. pointed out the latest Atlantic article by Daniel Engber on academic misconduct in behavioral science, and one of the themes was compensation and the outsized benefits that come with novel findings: tenure, grants, social status, etc.
My first reaction to the article was dismissal: in my version of applied behavioral science (SIDE), where every intervention gets validated by a pilot and published studies are used only as generative prompts during the Design phase, academic misconduct doesn’t have the same scale of negative impact. If someone made something up, the pilot will show that it doesn’t work and as long as we don’t also falsify the pilot results, all it did was waste time.
But then I started thinking about our role in the attention economy. Most of us still read and debate journal articles. I talk to clients about work that brings academia closer to application, without actually being in those labs and watching that data collection. We’re not just consumers; we also use our expertise to direct attention.
A few months ago, I blocked someone on LinkedIn. We first interacted back when I published the SIDE model; they insisted that Evaluation was unnecessary if an intervention was soundly based in theory. I objected and said the whole point of science was being willing to collect evidence that might disprove a generalized theory.
We sparred a few more times, most recently when they insisted that one cognitive model was more “scientific” than another simply because it was published in an academic journal. After a few rounds of comments, I blocked them and moved on.
But sometimes they pop in my feed because we’re both included on a “who to read in applied behavioral science” list. And I have a visceral reaction every time it happens, because I don’t want what I do to ever be associated with their approach.
Applied behavioral scientists can’t just opt-out of the discourse on academic misconduct, even if our methods shelter us from its ill effects. Because as experts in our field, we still have a role in who we platform. When we say “you should follow X” or “read Y”, we’re socially endorsing it.
So what to do, in a world where you can’t validate everything? For one, use papers and case studies as examples, not rules. And be sure you’re communicating to others that what matters about those examples isn’t the phenomenon but the methodology. Our job as applied behavioral scientists is to do and teach a scientific approach to changing behavior, not to create generalizable descriptions of the world.
Be careful who and what you platform. And if someone says in public that pilots are unnecessary and whatever methodology gets published in a journal is the thing we should do…don’t put them on the same list with me.
https://mattwallaert.com/wp-content/uploads/2018/12/thoughtful.jpg9001200matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2024-11-22 02:00:332024-11-22 02:00:33We need to be conscious of both the practices and people that we platform
TLDR: Narrow misses feel worse than wide ones because it is easier to imagine all the things that might have gone differently. But regret is a red herring; a near miss is an almost win and needs an “ante vitam” meeting.
_
Imagine that you’re late for your train in two parallel universes. In one universe, you arrive on the platform just as the train is pulling away. In the other, you missed it by 30 minutes. Which universe would you rather be in?
Most people intuitively understand that missing it by a few seconds will feel worse and they’re right: we feel more regret about narrow misses than wider ones because of counterfactual thinking. It is easy to imagine a dozen “if only” ways you could have saved a few seconds: got out of the car quicker, chose your shirt faster, not had to look for your keys. It is harder to imagine saving 30 minutes.
Anticipating that regret, most of us would say we’d rather have missed it by a mile.
But regret is a red herring; it tends to make us focus on our past instead of our future. And that change in framing makes all the difference. Because in reality, missing something by a few seconds means that next time, you’re very likely to make it. If you are going to be taking this train every day, it is far better to have missed it by an inch than a mile.
Back to the train platforms. Now you’re not alone: you’re traveling with your best friend.
When you miss by inches, the fingerpointing begins. Because when we feel regret, we often externalize it and begin the blame game; it feels better to grumble about how slow your best friend is than confront the reality that it might have been your fault.
This happens all the time in workplaces: near misses devolve into analysis paralysis as Product, Design, Marketing, and Tech focus on who to castigate. Blame is easy because any number of decisions by any of the departments would have resulted in a win.
Think about the 2024 presidential election. As Democrats argue about “the reason” for losing, the reality is that with a popular vote margin of only about 1%, most of the cited reasons are valid and a change in any of them could have swung the balance. Racism, sexism, communication, the lack of a primary, whatever…they’re all on the table in the way they wouldn’t be with a 30% deficit.
It is important to teach your team to identify the size of a gap and to change your strategy accordingly. With wide misses, you need a few heavily resourced interventions capable of closing a large gap. But with almost wins, it is more important to spread your resources across a number of smaller bets, since any of them is enough to tip the balance. Making this explicit can help cross-functional teams quickly move away from blame and toward solutions.
The frame change can be as simple as a name change. A post mortem is “after death” and makes sense for wide misses. But for an almost win, consider an ante vitam meeting, because an intervention that almost worked is just “before life”.
https://mattwallaert.com/wp-content/uploads/2018/12/strike-out.jpg9001200matthttps://mattwallaert.com/wp-content/uploads/2018/12/smalllogo.pngmatt2024-11-22 01:59:312024-11-22 01:59:31A near miss is an almost win and needs an “ante vitam” meeting
For the last several years, I have been making myself available for free, first-come-first-served meetings in the style of academic office hours. They’re 30 minutes, 1:1, virtual, and guided by the participant on topics ranging from career advice to applied behavioral science. And they’re specifically designed to address the inequities inherent in gatekeeping culture.
I’m a big believer in the power of framing to shift how we think about our behaviors. When we talk about volunteering our time to help out people via warm intros, it sounds positive. And it is; we could be spending that time on ourselves. But that same time, there is another frame: that meeting via warm intro is a form of active discrimination against those who don’t have the same social access. Yes, we’re helping people get ahead, but we’re also often helping the people who are already ahead to get even further ahead. Warm intro meetings are more likely to be white, male, educated, etc. because those are the people already most likely to have access to the social elite and so rather than addressing inequity, we’re magnifying it.
To measure our ability to create equitable access through open office hours, in 2021 we released our first Diversity Report, a concentrated effort to make sure that this system is in fact serving a broad range of underrepresented people. In our 2022 edition, we started doing trend analysis, which we’ve continued this year.
I use the plural repeatedly throughout this report. That’s because making office hours happen is a team effort; even if I’m the one actually showing up, there is a tremendous amount of work from a number of people to make sure that we follow up on action items, share our learnings, and prepare this report every year. In particular, Melanie Perera and Alaanah Sallay from Oceans and Lorraine Minister are instrumental in connecting job seekers with opportunities, sending out materials, and preparing the Mentor Minutes for social media. I am grateful for all they do and hope you take a moment to celebrate them.
Before looking at the results, a few quick notes on methodology. To gather the data, we set up a Google Forms survey and then used Zapier to automatically email participants with a link after each meeting, along with an end-of-year followup reminding them of the survey. In addition to asking for qualitative feedback to help us improve, we asked basic demographic questions about age, gender identity, sexual orientation, ethnicity, etc. No questions were required, all were multiple choice with options presented in random order, with “Other” and “Prefer not to say” options included.
For 2023, we went from ~550 meetings last year to more like ~750 meetings this year. We received 140 survey responses, giving us a response rate of ~19%.
That is lower than last year, so we’ve made some changes for 2024 to try to make sure we’re getting more accurate data. For example, we’re now using Zoom’s automatic followup feature to give people the survey immediately after the meeting, which should also improve data accuracy. Even with a lower response rate, however, we still have a significant sample, so we can make some reasonable assumptions that this data is representative of the larger population of participants. You could always make an argument that some segments are more likely to respond; caveat emptor.
I’m changing the format somewhat this year, as most data has remained relatively flat from last year (I’ve included the change from the 2022 numbers in parentheses), and so it is more appropriate to save the commentary for the end. As always, I’m open to questions and feedback on the analysis, as well as suggestions on what commitments you’d like me to make; just shoot me an email.
Age
The mean age in respondents was 36 (+2) and the median was 35 (+2). The best comparison is probably the median age of the US working population, which is 42, so overall we’re skewed a little younger. However, the standard deviation was around 9 (+0), with participants ranging from 20 to 60, so there was a good bit of variability.
Gender
Among respondents, 57% (+1) identified as women, 37% (-1) identified as men, and 6% (+1) identified as non-binary/genderqueer. This is a fairly large overrepresentation of women and potentially a large overrepresentation of non-binary/genderqueer people, although that number is harder to evaluate because of the correlation with age.
Sexual Orientation
78% (+0) of respondents identified as heterosexual, with 22% (+0) identifying as some form of LGBQ. This is significantly different than the base rate of 93% and 7%, respectively.
Race and Ethnicity
53% (+0) of respondents identified as White (base rate 77%), 9% (+2) as Black or African American (base rate 13%), 23% (+4) as Asian (base rate 6%), 1% (+0) Native American (base rate 2%), and 14% (-6) as More Than One Ethnicity (base rate 2%). In addition, 17% (+5) identified as Hispanic or Latino/a/x (base rate 18%), with Mexican, Mexican American, or Chicano/a/x as the largest group.
Other
27% (+7) of respondents are first-generation Americans (base rate 14%), while 27% (+4) are first-generation college graduates (base rate 35%). 37% (+10) view themselves as underrepresented in their field, while 10% (+1) are living in poverty and 9% (+4) identify as disabled. 27% (-14) did not add any additional tagging. 73% (-1) are currently living in the United States.
Commentary and Commitments
Demographically, there were surprisingly few changes this year to individual categories; most were within the margin of error for our sample size. The largest change actually came in the number of folks who identified with no underrepresented categories of any kind, 9.3% (-3.3). Neither gender identity, ethnicity, or sexual orientation meaningfully changed, which suggests that this year, more Straight White Males saw themselves as reflecting other underrepresented identities.
On the one hand, that could indeed be progress: we could be reaching a different audience than in previous years. Or perhaps Straight White Men are simply coming to recognize a broader array of potential ways in which someone can face challenges. The cynical view, however, is potentially quite worrisome: that the language of representation is being co-opted by those who struggling to find any possible way in which they can distance themselves from the trappings of privilege.
Regardless of how you choose to interpret it, it does present a compelling case for why intersectionality needs to become the default way of looking at representation. Saying that 90.7% of office hours were devoted to underrepresented folks obscures the fact the very real difference between facing one demographic underrepresentation and several.
We made two important commitments in our 2022 report, both of which we were able to honor. The first was to repurpose more of the content generated in office hours for a wider audience. It isn’t particularly efficient for me to say the same thing over and over again, when we could be using office hours for custom questions and content. So we did more editing this year to distribute clips of the advice I repeated most often.
We also promised to increase our available tooling. This year, for example, we refined our job tracking spreadsheets so we can more easily refer people for open positions and released self-paced courses on applied behavioral science, thanks in large part to Lorraine Minister’s efforts as our Head of Education.
For 2024, we’re going to continue to focus on scalability:
AI-identified content. Using new tooling from a stealth partner, we’re now able to automatically identify the phrases and examples I use most often and clip them for sharing. The shift from manual to automatic identification should significantly increase our ability to release in a timely fashion.
Structured education. Our approach to office hours was born out of my experiences in academia and I believe that 1:1 conversations benefit most when they’re an augment to structured learning content. So we’ll be introducing new guides and classes this year to help cover some of the basics and make our 1:1 time more efficient.
Side Note: It is eerie how many of these percentages were the same as last year, despite having a different sample size. I went back and checked the data repeatedly, just because it felt so unusual that, for example, the percentage of White participants remained exactly identical. It is a good reminder that things often don’t change as much as we think they do; even if they feel different day-to-day, the prevailing pressures that created the circumstance remain the same and so repeats are likely. It also reminds me that I need to do more to put my finger on the scale to make sure that next year, things do change.
Also, CapGemini still hasn’t released diversity numbers. So that makes it two years in a row that I’ve done what they are unwilling to.