I believe that behavioral science, correctly applied, can change the world. But, as with any emerging discipline, there is a period of self-definition in which people fight (with varying amounts of actual animosity) about who can claim what title and where the borders of the field are. 

Personally, I’ve largely been uninterested in the debate about who can and cannot call themselves a behavioral scientist (though to be clear, as a non-PhD, it benefits me not to start drawing lines). But that’s different from what it actually means to be doing behavioral science; as the name of the field suggests, it is the behaviors that should concern us. So I have become increasingly interested in how we might break down the various components of behavioral science into smaller units of work that could be credibly offered independently, while firmly maintaining the integrity of the behavioral science process as a whole.

To begin, let’s be clear that I am actually talking about applied behavioral science, which is explicitly concerned with changing behavior. This is distinct from academic behavioral sciences (like social psychology, behavioral economics, etc.), which further our understanding of the basic principles that underlie human behavior. That doesn’t mean academic folks don’t care about change or that applied folks don’t care about knowledge, just that each prioritizes one over the other. In my case, as an applied behavioral scientist, that means that while I still sometimes publish peer-reviewed papers, my primary work is changing the behaviors of populations.

My simple definition of applied behavioral science has always been “behavior as an outcome, science as a process,” which has the benefit of being easy to explain to people without exposure to the discipline and sounding pithy when you say it in a presentation. But if you’re trying to buy behavioral science services, or understand how you might begin to build them internally, that definition isn’t terribly useful. 

To make it more practical, I propose a four-stage model below that balances an understanding that each part is essential with the need to break it down into units of work that can be spread across internal teams and external vendors when necessary. But be warned: each handoff increases the potential for loss, particularly when there is an incomplete understanding of the adjoining stages. A tightly integrated process managed by people who understand the end-to-end process will always have the greatest likelihood of creating meaningful behavior change; that we can name the parts should not detract from the need for a whole.

  • Behavioral Strategy: the defining of a desired behavioral outcome, with population, motivation, limitations, behavior, and measurement all clearly demarcated. Plain version: figuring out what “works” and “worth doing” mean in behavioral terms by collaborating with stakeholders.
  • Behavioral Insights: the discovery of observations about the pressures that create current behaviors, both quantitative and qualitative. Plain version: figure out why people would want to do the behavior and why they aren’t already by talking to them individually and observing their behavior at scale.
  • Behavioral Design: the design of proposed interventions, based on behavioral insights, that may create the pre-defined behavioral outcome. Plain version: design products, processes, etc. to make the behavior more likely.
  • Behavioral Impact Evaluation: the piloting (often but not always using randomized controlled trials) of behavioral interventions to evaluate to what extent they modify the existing rates of the pre-defined behavioral outcomes. Plain version: figure out whether the products, processes, etc. actually make the behavior more likely.
  • Behavioral Science: combining all four of those processes. Plain version: behavior as an outcome, science as a process.

Step 1: Behavioral Strategy

Because the process is linear and each step requires that the previous step was done (although not necessarily by the same person), we need to start by defining the behavioral outcome we want to achieve. In the latest version of the Intervention Design Process (or IDP; the applied system I propose in my book), we do that using a behavioral statement: When [population] wants to [motivation], and they [limitations], they will [behavior] (as measured by [data]). Arriving at that statement is deceptively hard work and requires running a disciplined process with stakeholders to define each of those variables. But done correctly, it paints a picture of the world we want to create when our interventions are working. 

Given that the process prioritizes what we want the result to be rather than the interventions that actually create the result, my proposed term is behavioral strategy. While it doesn’t have to include a cost/benefit ratio that defines how much an intervention can cost relative to the impact that it has, certainly knowing this can shape the rest of the process and allows stakeholders to more clearly understand the actual stakes. 

Inside a company, both Strategy and Product teams try to answer this question regularly, although they often express it in imprecise, non-behavioral terms that create misalignment later. Externally, a strategy firm like McKinsey could likely spin up a unit that did this work in a reasonable way but like internal teams, they currently tend not to focus specifically on behaviors and don’t offer this as a service today.

Step 2: Behavioral Insights

The next step in the IDP is understanding the distance from the world we want by understanding the pressures that create the world of behavior we have today. Insights can be both quantitative and qualitative, so collectively I propose behavioral insights and then splitting as needed: qualitative behavioral insights and quantitative behavioral insights, since there are specialists that concentrate on one approach or the other. 

Existing user researchers and data scientists frequently do this work (Spotify has Quantitative User Research, for example), and as long as they’re doing the work with an explicit emphasis on generating insights to change behavior, these teams could slot in here. If you wanted to buy it as a service, IPSOS’ behavioral science team seems to do behavioral insights as a specialized form of market research that focuses on behavior and other agencies may be able to provide insights if specifically pointed toward behavioral outcomes.

Step 3: Behavioral Design

Having mapped the behavior we want and understanding why it doesn’t yet occur, in the IDP we next get into pressure mapping and intervention design. There are lots of ways to create behavioral interventions that don’t use pressure mapping, like design thinking, but ultimately we are always trying to generate proposed interventions that may change behavior. I say “proposed” and “may” because while we have supporting evidence (because the design process is based on the behavioral insights we defined above), we haven’t actually tested whether the interventions create the behavior. 

Design and Product departments do this within companies today, although often lacking the behavioral focus, so it seems appropriate to call this behavioral design. And an agency like Fjord could potentially do this externally, so long as they are given an articulated behavior outcome and the relevant behavioral insights (neither of which they are likely to create themselves).

Step 4: Behavioral Impact Evaluation

Finally, we have the evaluation of the proposed interventions, to see to what degree they actually create the outcome articulated in the behavioral strategy. Since this is already called impact evaluation in the non-profit world, behavioral impact evaluation feels like a fitting update to include the behavioral focus. The theoretical gold standard is a randomized controlled trial, in which participants exposed to the intervention are compared against a control group, but that may not always be feasible; remember, in applied behavioral science, we only need to be as right as the cost/benefit ratio dictates. In a perfect world, doing this process also results in the observation of additional behavioral insights (because trying to change a system often reveals underlying truths about it) but I don’t think we should try to make this a specific requirement of this process. 

Very few companies actually run rigorous pilots today, although it does happen in some Product and Data Science organizations (and Marketing loves non-theory-driven RCTs in the form of A/B tests), so this is probably the largest potential growth area for behavioral science as a whole. In the non-profit sector (where impact evaluation is sometimes built into grants), an agency like Social Impact will do an RCT on your interventions for you, if you’re careful to make sure they translate “impact” in behavioral terms.

Combined: Behavioral Science

To me, behavioral science requires the combination of all of the above. If you can’t define a behavioral outcome (AKA don’t do behavioral strategy), then you miss out on the whole point of “behavioral” in this discussion; you can’t run a scientific process if you can’t measure what works, and you don’t know what “works” means if you don’t define it. 

Similarly, you could run pilots and measure behavioral outcomes but without behavioral insights, that’s not science: your interventions aren’t necessarily designed based on replicable understandings (my favorite example of this is Marissa Mayer testing 41 shades of blue at Google; because there was no theory behind the iterations, you could only know what worked in that limited moment but not why) and so if they don’t work, you’re not actually any closer to something that does. It is only when all four processes come together that you truly get to both of the words in the term behavioral science…and neatly arrive back at “behavior as an outcome, science as a process.”

Some people who currently offer behavioral science services are going to hate this taxonomy, because it threatens their identity, both personally and professionally. And I understand that feeling: removing ambiguity can feel like a loss, when clarity reveals you’re only covering some of the territory. And not offering some services isn’t always by choice; for example, I’ve often heard consultants complain that they can’t sell behavioral impact evaluation to clients because they already “know” it will work after the behavioral design phase.

But the purpose of this guide is arriving at a shared understanding of applied behavioral science and its components, and part of that is recognizing that no one piece of the field is better than any other. There is no shame in only doing part of it, as long as we clearly explain the other parts and push the importance of doing the full process. By creating areas of intersection and smooth handoffs, we can better allow for specialization and move the world incrementally closer to behavior as an outcome, science as a process. And that’s work worth doing, in any form.

Side Note: My belief in this model is why I’ve decided to join frog as the Executive Director of Behavioral Science. My role is two-fold: help my fellow frogs apply behavioral strategy, insights, design, and impact evaluation in their projects and help our clients build their own applied behavioral science capabilities. While I’ve worked hard to evangelize the field broadly in my previous roles (including writing Start At The End, which was as close to a handbook as I can get, and doing 30+ talks a year), ultimately my career to date has been about creating a long series of behavioral interventions that accomplished internal business goals. In contrast, at frog I’ll be focusing specifically on behavioral science as a process, both internally and externally. As we see more senior behavioral scientists within large companies, we have the opportunity to leverage existing cross-disciplinary expertise to further support that work. And frog, particularly as part of CapGemini Invent, is the right place to do that. The agencies I mention in the examples can all learn to do parts of the behavioral science process. But because they typically do only their siloed step, they think of their stage’s deliverable in isolation. At frog, because we can and have done the full cycle, we know each step is just a milestone, so we can take a more holistic view and plan our work to naturally connect to the next necessary step. And through our Org Activation practice, we can teach organizations alongside projects to help them grow their own capabilities. Behavioral science doesn’t belong to frog – it belongs to everyone. And it is with that belief firmly in mind that we look forward to growing this discipline together.

Jason Fried, the CEO of Basecamp, has been making some changes at the org and decided that they “deserve an announcement”. While worth reading in their entirety, the changes are geared around taking the challenges of leading a company and addressing them by promoting monoculture (in the veil of individualism; Jason would call it “being responsible for [only] ourselves”) as a solution. He quotes Aldous Huxley in his introduction: “We live together, we act on, and react to, one another; but always and in all circumstances we are by ourselves.”

The changes are a stark departure from his previously expressed views on diversity (I say “expressed” because social signaling around diversity is different from the articulation of specific work policies) and are antithetical to most existing science on behavior change toward positive outcomes within an org. And so, as with an earlier post responding to points made by Domm Holland about how to grow a team, I drafted this post to offer a counterpoint to some of Fried’s changes (I won’t call them recommendations, since in his individualistic frame they are made for Basecamp only, although then why bother publishing them and emphasizing how much you “give back to the community” by speaking and publishing on management topics?) by surfacing potential alternatives.

Before breaking down the changes point-by-point, I have to apologize. Because how we create should be evidence-based, I normally compare the outcomes associated with the author’s recommendations to a benchmark (for example, looking at Fast’s diversity compared to Google). In the case of Basecamp, because they have so few employees and most don’t identify themselves with a picture on either LinkedIn or Basecamp’s website, it is impossible for me to currently tell you much about the monoculture that is the result of Fried’s policies. Should diversity data become later available, I will update this post with it alongside a relevant benchmark.

It is also worth noting that diversity data may not even be an appropriate benchmark for these recommendations; Fried’s measure for the changes seems to actually be profitability (despite the Basecamp Jobs page, which explicitly states “diversity has deeper value beyond monetary”), although the relationship between diversity and profitability is well-established. Certainly Basecamp’s product has a diverse subscriber base, so in making these policies public, change could be measured not through attrition of employees but of users; if Basecamp became unprofitable because of Fried’s policies because everyone unsubscribes, presumably he would see that as a failure.

Fried Point #1: No more societal and political discussions at Basecamp. Today’s social and political waters are especially choppy. Sensitivities are at 11, and every discussion remotely related to politics, advocacy, or society at large quickly spins away from pleasant. You shouldn’t have to wonder if staying out of it means you’re complicit, or wading into it means you’re a target. These are difficult enough waters to navigate in life, but significantly more so at work. It’s become too much. It’s a major distraction. It saps our energy, and redirects our dialog towards dark places. It’s not healthy, it hasn’t served us well. And we’re done with it at Basecamp.
Commentary: Certainly word choice matters: describing social justice conversations as “a major distraction” and the result of “sensitivity” while emphasizing the need for the workplace to be “pleasant” is a direct appeal to the desire for a monoculture. But it is difficult to interpret the policy itself. What does it mean to say that there will be no more societal or political discussions in a workplace? Does wearing a #BLM t-shirt on a Zoom call mean you’re fired? Even with nebulous consequences, the policy seems certain to reduce psychological safety, which Google previously found to be the greatest predictor of team success. It is difficult to imagine that scale items like “Members of this team are able to bring up problems and tough issues.” and “People on this team accept others who are different.” would be positively impacted by an explicit command not to talk about differences, especially when they’re potentially unpleasant to the rich white male in charge of the company.
Alternative Tip #1: Embrace differences that contribute to the psychological safety of the group by creating spaces and systems that allow for the discussion of all areas of impact, regardless of their relationship to the the social power structure, while mindfully balancing short-term velocity with long-term value. Encourage and guide respectful, validating discussion.

Fried Point #2: No more paternalistic benefits. For years we’ve offered a fitness benefit, a wellness allowance, a farmer’s market share, and continuing education allowances. They felt good at the time, but we’ve had a change of heart. It’s none of our business what you do outside of work, and it’s not Basecamp’s place to encourage certain behaviors — regardless of good intention. By providing funds for certain things, we’re getting too deep into nudging people’s personal, individual choices. So we’ve ended these benefits, and, as compensation, paid every employee the full cash value of the benefits for this year. In addition, we recently introduced a 10% profit sharing plan to provide direct compensation that people can spend on whatever they’d like, privately, without company involvement or judgement.
Commentary: When social psychologist Daniel Kahneman won the Nobel Prize in Economics in 2002, it was for his work challenging a notion that many previous economic models relied on: homo economicus – the infinitely rational person. At this point, decades of research has shown that money is objectively not fungible, despite Fried’s assertion that it should be, and non-monetary incentives at work are a key component of both job satisfaction and performance. Some categories of benefits like continuing education have different tax treatments when not lumped into pay and the non-monetary benefits negotiated by workplaces typically have outsized impact on workers at the lower end of the pay spectrum, making pure cash payments specifically inequitable. Finally, even if Fried were correct in saying that Basecamp should not care what you do outside of work, it presupposes an artificial barrier that is empirically untrue: what you do outside of work directly affects what you do at work and vice versa.
Alternative Tip #2: Invest in greater quality-of-life benefits that have recognized impact across diverse populations. Be clear that behavior is behavior, in and out of the workplace, and that where there is a demonstrated relationship between the two, they will be considered together.

Fried Point #3: No more committees. For nearly all of our 21 year existence, we were proudly committee-free. No big working groups making big decisions, or putting forward formalized, groupthink recommendations. No bureaucracy. But recently, a few sprung up. No longer. We’re turning things back over to the person (or people) who were distinctly hired to make those decisions. The responsibility for DEI work returns to Andrea, our head of People Ops. The responsibility for negotiating use restrictions and moral quandaries returns to me and David. A long-standing group of managers called “Small Council” will disband — when we need advice or counsel we’ll ask individuals with direct relevant experience rather than a pre-defined group at large. Back to basics, back to individual responsibility, back to work.
Commentary: The pairing of accountability and autonomy at work is a necessary precursor to equity, both in celebrating success and managing through failure. But committees are not the antithesis of individual accountability/autonomy. And a mandate that counsel comes only when sought by the accountable person from the sources of their choosing presupposes that that person knows who actually can provide value or is willing to hear them (this is particularly dangerous when coupled with Fried Point #1, since nobody should be talking about anything outside their hypothetical swim lane in the first place) and that there is neither serendipitous nor contrarian value. This is simply not true. As repeatedly proven both in the academic literature and by applied studies done by folks like Cloverpop, large groups of diverse people typically act as strong advisors to individual decision makers. As a small side note, it is also highly incongruent to talk about responsibility for DEI work (process) and then advocate for individual responsibility around outcomes.
Alternative Tip #3: Maintain high individual accountability for clearly expressed outcomes, coupled with high autonomy on the process to reach them. Allow for a diversity of voices to inform (not dictate) that individual accountability and ensure appropriate forums for those diverse voices to be heard by the accountable individual.

Fried Point #4: No more lingering or dwelling on past decisions. We’ve become a bit too precious with decision making over the last few years. Either by wallowing in indecisiveness, worrying ourselves into overthinking things, taking on a defensive posture and assuming the worst outcome is the likely outcome, putting too much energy into something that only needed a quick fix, inadvertently derailing projects when casual suggestions are taken as essential imperatives, or rehashing decisions in different forums or mediums. It’s time to get back to making calls, explaining why once, and moving on.
Commentary: As with Fried Point #1, I don’t actually know what this means as a policy and yet it seems meant to fix a litany of decision making issues (many of which are the product of poor decision structures and a lack of the autonomy/accountability pairing suggested in Alternative Tip #3). While making quick decisions with short explanations certainly increases velocity, an important characteristic of progress, it actively degrades overall progress by reducing the probability that you’re heading in the right direction, especially as confirmation bias makes it increasingly difficult to see the error of a decision direction as more decisions are made in that direction.
Alternative Tip #4: Have clear decision making paradigms (including criteria like reversibility), with established review points (both during and after a decision) to balance velocity against accuracy.

Fried Point #5: No more 360 reviews. Employee performance reviews used to be straightforward. A meeting with your manager or team lead, direct feedback, and recommendations for improvement. Then a few years ago we made it hard. Worse, really. We introduced 360s, which required peers to provide feedback on peers. The problem is, peer feedback is often positive and reassuring, which is fun to read but not very useful. Assigning peer surveys started to feel like assigning busy work. Manager/employee feedback should be flowing pretty freely back and forth throughout the year. No need to add performative paperwork on top of that natural interaction. So we’re done with 360s, too.
Commentary: As with many of the Fried Points, he equates the result of a badly implemented system with the system itself. While there are legitimate, empirically-backed issues with some review processes that Fried identifies (including the tendency to be periodic rather than as-it-happens, with an emphasis on summary rather than specific feedback), that is not an inherent flaw of specifically peer review, with Fried singles out to target. Manager-only reviews have a long history of centralizing bias, especially since cis white men continue to disproportionately be the ones doing the reviewing. As with the Points 3 and 4, getting diverse feedback from a larger group works not only enlarges the actual performance space considered (literally the 360) but mitigates the inherent bias that comes with a single dyad.
Alternative Tip #5: Ensure that feedback is timely and specific, while ensuring that it also comes from a diversity of sources, not just in level but in working relationship and context.

Fried Point #6: No forgetting what we do here. We make project management, team communication, and email software. We are not a social impact company. Our impact is contained to what we do and how we do it. We write business books, blog a ton, speak regularly, we open source software, we give back an inordinate amount to our industry given our size. And we’re damn proud of it. Our work, plus that kind of giving, should occupy our full attention. We don’t have to solve deep social problems, chime in publicly whenever the world requests our opinion on the major issues of the day, or get behind one movement or another with time or treasure. These are all important topics, but they’re not our topics at work — they’re not what we collectively do here. Employees are free to take up whatever cause they want, support whatever movements they’d like, and speak out on whatever horrible injustices are being perpetrated on this group or that (and, unfortunately, there are far too many to choose from). But that’s their business, not ours. We’re in the business of making software, and a few tangential things that touch that edge. We’re responsible for ourselves. That’s more than enough for us.
Commentary: Every company is, inherently, a social impact company: the products and services we create continually change behavior in the world around us. We might not all be double-bottom-line, or certified B-Corps, but what and how we create matters. Fried knows this; he notes that Basecamp “gives back” (although without acknowledging that many of these activities also contribute to the Basecamp bottom line) presumably because he believes that those activities create change. The false distinction that he is drawing is that there are somehow borders to that change, artificial lines we draw on a map. That simply isn’t true. The edges of our impact are defined by the impact itself; the decisions we make ripple not only where we want them to but far, far beyond. I can understand the desire to operate in a convenient world where Fried gets to decide what he doesn’t care about; rich white men have been doing that for ages. But the practical reality is that our choices have consequences and we must seek to confront them head on.
Alternative Tip #6: Understand that the impact of your company is defined by what it demonstrably impacts: not its aspiration to create impact, nor its desire to avoid it. Create processes to understand that zone of impact, to measure it, and to make conscious choices about how you change it.

Side Note: In talking about this with Tim Morey, he reflected that a lot of these edicts sound like a return to shareholder value added, the antiquated notion that companies should be judged on profit alone, usually in the quarter-over-quarter sense. And I’m struck on how much of this technocratic libertarianism, like SVA, is just an excuse for short-term profiteering that enrich the richest among us. I’m reminded of the Warren Buffet quote about investment: “Our favorite holding period is forever.” If you were trying to create permanent value, to build toward a utilitarian ideal that maximized outcomes across the consideration set, why would you seriously consider any of the policies above? And why would you support those who do?

Recently, I accepted a new job offer.  And I was excited, as so many folks are when they find meaningful work.  It felt like such a great gig: my entire reporting line would be women!  It would be global in scope!  It would be spreading behavioral science among Fortune 500 CEOs!  I turned down three competing offers, told my family, and started working on plans for the first few months.

Then the employment agreement arrived.

It had the standard bevy of non-competes and non-solicits, which are problematic in their own right but still a standard that hasn’t been broadly challenged (although the FTC is working on it).  And yes, they wanted to own my IP, which has its own thorny definitional challenges when your job is uncovering and appplying underlying truths about human behavior (“Who owns science?” can turn into a very long conversation).  But those issues were navigable and in twenty years of working, despite all my apprehensions, I’d never actually had any problems arise.

Yet the packet also referenced agreeing to policies, like a code of conduct, that weren’t attached.  So I asked the executive recruiter if they could send them over, because one of the things drilled into me by my father was to never sign a contract I didn’t understand (FBFP, or “fucked by fine print”, was a common condition in rural Oregon).

“I’m sorry, our policy is not to share our policies outside of the company and since you don’t work here yet, we can’t send those to you.”

Ruh roh.

That stance isn’t actually all that uncommon, as the standard of many enterprise companies is to be closed by default, because they believe it minimizes risk.  In this case, risk that they’ll get bad press or lawsuits around something in the policies.  Or risk that candidates will read the policies and refuse to join, rather than reading them once they’re already dependent on the paycheck and cognitive dissonance changes their mind (literally).

And I get it, I really do.  Even though I believe employment policies should be open by default because public scrutiny is actually the best protection against catastrophic risk, I recognize it is something upon which reasonable people could disagree.

But closed by default isn’t just about legal risk; it is also about creating inclusive workplaces.  In the case of my offer, it turns out that joining would require me to stop tweeting (except to retweet officially sanctioned company propaganda…I mean, uh, very valuable thought leadership pieces) and blogging (because I might leak the secret recipe to behavioral science on an unapproved channel; its “behavior as an outcome, science as a process”, just FYI).

That was enough for me to turn down the offer, because I’m privileged enough to have other options and money to fall back on if I didn’t.  My new employer also asked me to agree to their code of conduct as part of my employment but when I asked to read it, they simply sent it over.  And just like my previous boss, Vivek Garipalli at Clover Health (“I don’t care if other companies literally follow you around the office every day; they can’t execute like we can.”), my new boss believes that public science is a good thing: he likened it to football, where playing in public helps the sport evolve.

But not everyone gets a happy ending.  Sometimes, “my poverty, but not my will, consents.”  And as a white guy, it is a) likely that the code of conduct I never got to read is built around my cultural norms and b) I probably wouldn’t even get in that much trouble for violating it, just because of how accountability works in corporate America.  But that is dramatically different for many, many people.  Often those with the fewest options are also those the environment is least inclusive of. 

And yes, it is possible that, tweeting and blogging aside, the policies I never ready were very sensible.  But closed by default means we can’t know and in the absence of evidence of the contrary, we should assume the norm.  And unfortunately, as evidenced by the preponderance of white males at the top of the social pyramid, racism and sexism and other forms of bias are the norm.

If you think that a bad code of conduct sounds far fetched, remember that Google is just now acknowledging that people have the legal right to talk about their salaries.  And we had to pass laws so that black people could wear their hair naturally at work (and they aren’t even universally adopted yet) because 80% of black women felt pressured to change their hair style in order to fit in.  Does the code of conduct mention salary data or hair?  No idea, because you can’t read the policy until after you join and I didn’t.  But again, until we specifically know that something is an exception, we should assume the norm.

In the end, closed by default policies minimize some risks but create others.  You lose talent that won’t sign in the blind or refuses to work in an environment that doesn’t value their ability to have a public opinion.  You decrease diversity by preventing the formation of an open, inclusive environment.  And given the preponderance of evidence that open environments are associated with profitability (Satya Nadella’s tenure at Microsoft is an excellent case study), closed by default creates very real profit risk.

We need to be default open.  To share our code of conduct, when people ask.  To publish research and processes, because execution is really the only moat.  And if the sharing sparks controversy or others iterate on it, we can use that feedback to build something better.  Because we can’t have it both ways, preaching innovation through failure but hiding out of fear of our own failures.  Truth will out; out yourself.

Side Note:  There are a host of industries that profit on ambiguity by trading on information asymmetry, either directly or indirectly.  But because of the endowment effect, asymmetry is hard to perpetuate because once we have access to information, it instantly becomes more valuable because we psychologically “own” it. That is why workplaces will always drift toward more liberal policies; once you’ve had access to more relaxed rules around social media, spending, dress code, etc, the price of giving that up will simply get higher and higher and higher. Thus shifting to a liberal workplace early is actually a competitive advantage, even if Jamie Diamond doesn’t see it yet.

Over the weekend, Fast’s CEO Domm Holland posted a short Twitter thread about growing from 2 to 120+ people in 18 months while maintaining “an exceptionally high talent bar” and offered some tips based on Fast’s hiring process.  But as I was reading the thread, I was struck by how many of the practices seemed likely to perpetuate a monoculture.  So I drafted this post to offer a counterpoint to some of his recommendations by surfacing potential alternatives.

Note that I am not responding to all of Holland’s tips: “treat your people well” is grounded in strong evidence for supporting a diverse and inclusive workplace that leads to high performance.  I’m also not responding specifically to Fast’s culture, although the thread responses do point out potential issues across a variety of domains, from rescinding offers to using Nigerian devs at very low wages to build V1 of the product and then terminating them without cause.

Before going through the tips, it is important to remember that hiring practices should be evidence-based and so I needed to look at Fast’s hiring data before responding to Holland’s thread; maybe their practices are a secret recipe for diversity and I’m simply wrong about their monocultural nature.  Since Fast doesn’t have a public diversity report that I could find, I gathered 110 people on LinkedIn who identify themselves as working at Fast.  I then coded for perceived ethnicity/gender (using names and Twitter/other photos if they didn’t have one on LinkedIn), had another blind rater do the same, and compared; there was only one person we coded differently, so I removed them from the sample, leaving 109.  

Obviously, this methodology leaves out other important forms of diversity and coding gender/ethnicity using LinkedIn photos and names is flawed; ideally, all companies would release their self-reported diversity data to avoid these limitations.  

To understand Fast’s diversity data, I compared it to Google’s hiring data from their 2020 diversity report.  Since Holland specifically calls out growth in the last 18 months and focusing on “the top 1% of major tech companies”, Google feels like an appropriate benchmark.

FastGoogle
Male68.8%67.5%
Female31.2%32.5%
White62.4%43.1%
Asian28.4%48.5%
Latinx4.6%6.6%
Black4.6%5.5%

Given that Fast achieved essentially identical gender diversity and significantly less ethnic diversity than Google, which has itself recently paid fines for documented hiring biases, a reply to Holland’s thread seems justified by the data.

Holland Tip #1: “We strictly do all recruiting internally” and “recruiting internally keeps tighter quality control”
Commentary: It isn’t entirely clear what Holland means here, since recruiting can include any number of responsibilities, from sourcing and screening to interviewing and negotiation.  There are entirely valid reasons to keep any and all of those processes internal but unless specifically designed to increase diversity, internal processes tend to favor the status quo.  And since the status quo in tech is overwhelmingly white and male and privileged, that means perpetuating those characteristics.
Alternative Tip #1: Because diversity begets diversity, use outside resources (including diverse external sourcing that you pay for) to challenge the status quo when needed.

Holland Tip #2: “we receive A LOT of inbound” and “yet still put 90% of effort into outbound sourcing to find the exact background & skill set we are looking for”
Commentary: A wide funnel of inbound is a great sign of traction but not always useful for increasing diversity, since those likely to apply are those likely to know, and those likely to know are those likely to already be in your network.  So outbound is an absolutely critical part of increasing diversity.  The red flag here is how that outbound is happening: not to broaden the inbound pipeline but to narrow it.  “Exact background & skill set” are often code words for biased filters like specific universities or companies that already have issues with bias; just as predators aggregate pollutants from the animals they eat, relying on biased sources means you aggregate those biases.
Alternative Tip #2:  Keep a wide funnel in your inbound by using clear job descriptions with bias reducing features and use outbound in a targeted way to widen, not narrow, wherever you have measured diversity issues.

Holland Tip #3: “we almost exclusively hired experienced people”
Commentary: “Experienced” is an interesting euphemism but Holland fortunately defines his usage as “hiring people who are currently thriving in a place that would be our next level up.”  This seems at odds with his first tip, since hiring specifically from competitors is very much like using outside recruiters (you are relying on the filtering of others) but as with Tip 2, the net effect is that you aggregate their biases.
Alternative Tip #3:  Monitor source diversity to avoid overindexing on single sources, be they specific schools, companies, or industries, and target increasing source diversity independently to achieving candidate diversity.

Holland Tip #4: “a-players attract a-players” and “the more we have focused on the best people, the higher the quality of applicant we get”
Commentary: The tendency of like to attract like (often called homophile) is well documented and when used purposefully, it can actually be a diversity superpower: diverse companies tend to get more diverse because of it.  But when type matching (like “a-players”), homophile generally reduces diversity by creating monolithic patterns for what is considered “the best”.
Alternative Tip #4:  Recognize and plan for broad representations of talent by having clear plans for recruiting unique skills and talents, then valuing and utilizing them.  Think of Venn diagrams that touch but don’t overlap more than 50%.

Holland Tip #5: “our team are the hiring panel, most of our team interview, screen and make hiring decisions”
Commentary: Unless a hiring manager is specifically attuned to increasing the diversity of their team, using hiring panels (rather than single hiring managers) increases diversity because it increases the number of potential advocates for underrepresented talent.  But this only occurs if the panel itself is diverse (again, homophile applies), trained and attuned to increasing diversity, and when unanimous decisions are not required.
Alternative Tip #5: Use diverse panels with specific training and allow for non-unanimous decisions; augment with external panel members when there is not enough internal diversity or availability.

Holland Tip #6: “internal referrals are exceptionally value [sic], they know the best people they have worked with”
Commentary: Internal referrals are specifically homophilic; as Holland says, we know who we have previously worked with.  This leads to the compounding of problems/virtues: diverse teams get more diverse, monocultures get more monocultural.
Alternative Tip #6:  Be clear about the value of diverse referrals by clearly publishing diversity data and encouraging diverse referrals both individually and systematically (either through rewards or refusal to consider overrepresented referrals).

Hiring high performing teams is hard and so for founders that believe in doing hard work, Holland’s thread is a seductive opus to that effort.  But as a white male founder, it is easy to think that easy things are hard simply because we believe they should be, even when the easiest thing to do is hire other white males: they are probably who you know, who you’ve worked with, and who are easiest to attract.  

So when we talk about doing the work, we need to recognize what the real work is.  Literal decades of research at this point tell us how important diversity and inclusion are to high performing teams; a failure to explicitly address those challenges in a thread about hiring practices is to acknowledge that you either don’t understand the current state of diversity in the workplace or aren’t actually committed to team as a strength.  Either way, we can and should do better; as Holland says at the end of his thread, “people deserve it”.

Side Note: I recently read some of my older blog posts and was aghast as the privilege encoded in them (along with the really terrible advice resulted).  While Holland’s thread created a cacophony of justifiable anger directed at both him and Fast, I wonder if that facilitates individual change or mires it in defensiveness; could I look back at my entries and acknowledge their flaws had I been attacked for them at the time?  Also, I wonder at the shelf life of even the alternative tips I list here; hopefully new science proves some of these suboptimal and we get ever better at creating the more equitable world we want.

The tech and VC worlds, like the world at large, are facing a reckoning over their history and practice of exclusionary bias. And while bias is incredibly complex and we have a long way to go, there are plenty of systemic changes that aren’t complex at all, like changing who and how we talk with people for the first time.  The insistence on warm and double opt-in intros actively reduces the diversity of conversations people have and while the privileged have the right to continue those practices, it is empirically true that they ultimately result in systemic inequity.

Before we talk about why, and what to do about it, it is important to first define terms.  A warm intro (versus cold outreach, when you simply approach someone you do not know without preamble) is one in which a third party introduces you to the person you want to talk to.  So if Daniel wants to talk to Matt, he gets Sam (who knows both Daniel and Matt) to make an introduction that includes both of them.  A double opt-in intro is one step further: before Sam introduces Daniel to Matt, Matt requires that Sam ask him if he wants the intro, enabling him to privately say no rather than having to bear the social burden of saying no to Daniel directly (the double comes from both Daniel and Matt opt-ing in, since in theory Sam could intro with neither of their permission, although this rarely happens in practice).

The science is very simple.  At every additional decision point, the opportunity for bias increases.  If anyone can email Matt freely, the only bias that exists is his willingness to write back; one opportunity for bias.  If Matt only accepts warm intros, Daniel must have enough social capital to get access to one of Matt’s gatekeepers and then Matt has to be willing to write back; two opportunities for bias.  If Matt only accepts double opt-in intros, Daniel must have enough social capital to get access to one of Matt’s gatekeepers, the gatekeeper must have enough social capital to get Matt to accept the intro, then Matt has to be willing to write back; three opportunities for bias.

This has pernicious secondary effects as well.  For example, if Sam knows that Matt is unlikely to talk to Daniel for reasons of bias, he is less likely to offer the intro in the first place in order to preserve his social credibility with Matt, thus creating a world that reinforces Matt’s biases; the net effect is that the only people who get introed to Matt are, for example, white males, reinforcing Matt’s believe that the overwhelming majority of people worth talking to are white males.  It is the very definition of a self-fulfilling prophecy.

To argue that these systems do not create bias is to argue one of two things: either that that bias does not exist in the first place or that one of those steps is free from bias.  It is certain that bias does in fact exist: in VC, female-only founding teams got less than 3% of total funding in 2020. Ditto for Black and Latinx founding teams.  The only alternative explanations to bias for those results is that either female/Black/Latinx people are not creating startups and trying for funding (demonstrably false) or that they are less fundable (which, unless you’re arguing that entrepreneurship is genetic, has to be a result of systemic bias).

It is also certain that gatekeeping systems create systemic bias.  Clues as simple as a name or email domain can be enough to change opportunity outcomes; I chose Matt, Sam, and Daniel deliberately as stereotypically white male but what if the intro request had come from Jazmin Jones?  And while an individual could argue that their intro system exists as an outlier to the general trend, unless they are specifically tracking diversity metrics that demonstrate that to be true, the assumption should be that the general trend holds.  Because in the face of absolutely overwhelming evidence of systemic bias, arguments that hurdles don’t create bias are specious at best and actively sexist/racist/classist at worst, unless specifically disproven with equally overwhelming data on the exception.

So what do we do about it?  In a perfect world, the powerful would open the floodgates and accept all comers, but that is both unlikely to happen and impractical.  But because bias is supported by systems, so can anti-bias action.  Combatting bias doesn’t have to be an all-or-nothing proposition; even if we can’t eliminate it, we can reduce it using the same sorts of systems that support it.

Intros are an easy place to start.  We don’t have to stop accepting warm intros, but we can stop relying on them exclusively.  Eliminating the need for opt-in makes the intros themselves incrementally less bias, with one fewer opportunity.  And creating systems that circumvent intros, like publishing an email address publicly and committing time to reading through it in the order received or creating an opportunity form (along with transparency around process and a willingness to iterate) through Google Forms or Typeform are also key.

Even better, devote time to an open calendaring system.  Personally, I use recurring Google Calendar appointment slots and then publicly distributing a bit.ly link (bit.ly/MattWallaertMeet) that allows people to claim time for themselves, which over five thousand people have done.  I can vary the length of the slots, hours I devote per week, and timing of those hours on the backend with minimal disruption in order to control the amount of time I’m spending; if people are forced to book too far out, I can also change the visibility of the bit.ly link (by, for example, including or not including it in my Twitter bio) until I can clear the backlog. I even have a stock sentence (“bit.ly/MattWallaertMeet will let you schedule a call.”) setup as a text replacement for a shortcode on my phone and computer to make it easy to distribute.

The slots themselves lay out rules for engagement, like sending preread in advance and not attempting a LinkedIn connection without speaking first.  A Google Meet is automatically created, although they also have the option to call me directly if they prefer.  The calendar invite also includes a link to a Google Form to leave anonymous feedback, in case someone has a negative experience.

Open calendaring doesn’t mean I am available to people 24/7 but rather allows me to spend a specific quantity of time on activities that directly reduce bias.  One of the critiques that people like Fred Wilson have leveled against open systems is that the demand is simply too great for the supply, but that’s not actually a critique: bias is about how we allocate supply, not about the overall demand.  People far more important than me are devoting time to grooming their personal networks; I’m simply advocating they groom differently.

Open systems are still imperfect.  Even knowing about a published email or calendar link is a form of bias, in the same way knowing what equity is and any other kind of knowledge.  But they are orders of magnitude better than a double opt-in system, most specifically because they combat the most important bias: our own.

Side Note: The entire paragraph on how bias works can be applied to employment phone screens as well.  To be clear, I’m not talking informationals, in which a recruiter or hiring manager explains the job in greater detail or clarifies details like salary or the need to relocate; I’m talking screens, which necessarily “catch” some people in them when recruiters are allowed to not pass them along based on the call.  Every additional round of vetting introduces increased bias, which is best combatted through panel hiring and other known anti-bias systems.  Individual recruiters can address their own biases and do great, anti-racist and anti-sexist work. But as with the statistics on VC funding, if relying on individuals to address their own biases worked, we wouldn’t have the employment demographics we do now.  Systemic answers consistently win and we should not let clinging to our own power (like the feeling of being able to decline a candidate) hold us back from making key changes.

Almost seven years ago, I gave a TEDx talk that would prove to be prophetic.  The focus was on trying to resolve two seemingly irreconcilable facts: that many new college graduates were unemployed and yet there were abundant job openings for college graduates with no experience.  My explanation was one of expectations: that because college students were recruited into college with promises of high future salaries, taking anything less than a high salary would mean realizing a loss.  As long as they stayed unemployed, they could continue to dream big.  The antidote, I suggested, was the centering of meaning rather than money as the reason for both work and education.  If we shifted focus to “work worth doing”, as Teddy Roosvelt put it, we could smooth the transition into the workplace.

Fast forward to now and while it is a trait that is alternatingly praised and ridiculed, Millennials and Gen Z are noted for their borderline obsession on making a meaningful impact through their work.  But while we made exactly the transition I proposed, in a rather spectacular fashion, the underlying problem remains: the current COVID-fueled unemployment crisis aside, there is a massive disjoint between the work available and the workers who could do it.  Because while I was right about centering meaning, I was also right about realizing losses.

In the original talk, I discussed how taking a job for $40K when you expected to be making $50K feels like losing $10K, even though in a rational economic sense $40K is a gain over $0.  This irrational behavior is one of the many insights of behavioral economics and explains why, for example, someone will ride a stock all the way to zero because “it might go back up” rather than taking a sensible loss and moving on; until they sell, the loss isn’t “real”.

What I missed was that the same argument can be made for meaning.  As we centered impact in the narrative of meaningful work, we created an identical problem to the unrealized salary loss in two ways.  

The first is the unrealized loss of role impact: new grads envision themselves as stepping into jobs in which they will be empowered to have immediate influence.  This has been heightened by an emphasis on entrepreneurship as a quick route to being the boss; I once asked a class of MBAs how many expected their next job to be CEO of a startup and almost all of them raised their hands (with several c-suite startup jobs under my belt, I have never once been CEO).

But of course, stepping into immediate impact is rare.  While companies are doing a better job of adopting organizational structures that create autonomy and accountability, even in the best of circumstances it takes time to develop both the skills and processes necessary to make meaningful changes.  If new grads are unprepared for that reality, they may view roles as too junior or meaningless and choose to remain unemployed even though, just as a $40K salary is better than $0K, having some impact in a workplace is better than none and the sooner you start working, the sooner you are able to increase your impact.

To some extent, however, the unrealized loss problem of role impact has always been present: it is in the nature of all of us, and especially the young, that we feel we ought to be allowed to create greater change.  It is the second shortfall, organizational impact, that has become more acute in the time since the original talk.

When I quoted “work worth doing”, I meant not only the work we do inside of companies but the meaningful impact the companies themselves have on the outside world.  But as the gap between the rich and poor in this country continues to widen, the perception of companies has become increasingly problematic.  Whether because of systemic (the implications of gig work and the utter inability of many companies to hire and retain black people at every level), executive (#MeToo and leadership abuses like those at Away), or social justice (funding hate groups, having business dealings with Trump) failings, there is almost always a reason to say that a company is less than ideal; brands are “cancelled” on a daily basis.  Being unemployed means retaining the ability to claim the moral high ground; at least you aren’t part of a system of oppression.

Entrepreneurship factors in here as well.  Many entrepreneurs think of their businesses as improvements on existing models and it is seductive to believe that we can replicate the success of others without also replicating their shortcomings (or introducing new ones of our own).  Until we express those shortcomings ourselves, they remain tantalizingly unrealized.  So rather than take a job at an existing company with flaws, we can start something flawless, even if its practical impact is near zero; 62% of people think starting your own business is a good career move, but less than a third of business started make it to the 10-year mark.

So what do we do about it, if the problem isn’t just meaning over money, but motivating people to engage in a system where participation feels like realizing a loss?  Fortunately, loss aversion is a well studied phenomenon and existing behavioral interventions hold potential to be applied here.

For example, a number of successful experiments have combated the tendency of investors to hold on to losing stocks (called the disposition effect) to avoid realizing losses.  One standout, suggested by my friend Dan Egan of Betterment, is reframing.  In stocks, we do this by noting that selling at a loss can be seen as a gain in taxes saved.  In employment, reframing salary (“Here is what you can do with $40K.”), role (“Here is what you’ll learn in this job.”), and organizational impact (“Here is who you could help at this company.”) as gains may shift the decisions you make.

Another possibility is timeboxing.  When investors are reluctant to let go of a losing stock, advisors sometimes encourage them to set a limit for how long they will hold it.  The same can be done with employment, like explicitly committing that if a $50K+ job has not materialized within three months, you’ll accept something lower.

And finally, there is the traditional investor strategy of hedging: explicitly pairing opposing strategies to protect against risk.  For every $50K job you apply for, apply for a $40K one; for each company with seemingly untainted morals, apply for one that is working to change itself.  Applying is a nice hedge because it is soft: you’re not agreeing to take the job, but rather opening up the possibility (and, once you have an offer, other psychological biases like the endowment effect will do the rest).

I still believe in work worth doing and I’m not suggesting there aren’t legitimate reasons not to apply for a particular job; I’m still angry at Toast for offering a payout to white guys while making women and POC work longer hours for less money and it is a great reason not to work for them.  But when the fear of realizing losses keeps people in a state of perpetual unemployment, unable to make the gains they so desperately want, we need more than just emphasizing the meaning of work.  Because work worth doing isn’t about working somewhere that is already perfect; it is about making things better through our work, iterating through engagement.  And engagement beats perfection every time.

Side Note: In the original talk, I ended on a Star Trek note, about how in a post-scarcity world, the human spirit still yearns toward meaning.  I think a lot about “boldly going where no one has gone before” and recently, I’m struck by how often that where is philosophical in nature.  Uncharted territory used to mean the places we literally didn’t have maps for but lately, it feels like the maps themselves are wrong because the territory is shifting beneath our feet.  We don’t yet know what an America that confronts its racist legacy looks like or what a more equitable, inclusive version of the workplace is.  But we don’t have to travel in space to find out: instead, we can travel in time, by planting our feet and swinging our fists and fighting for our future.

Teenage Mutant Ninja Turtles.  Spice Girls.  Cowboy Bebop.  Epic squads make for great storytelling.  But the reason they’re compelling is essentially the same: optimum distinctiveness.

When we think about identity, everyone is trying to find a balance between uniqueness and belonging.  Good squad-based narratives play on this by balancing a central uniting theme that differentiates them from the rest of the world (teenagers that are mutants and ninjas and turtles, defined against anyone that isn’t all four of those) with within-group archetypes that are clearly differentiated.  These archetypes are what psychologists call “optimally distinct,” like Venn diagrams that kiss but don’t actually overlap: Michelangelo the prankster, Donatello the egghead, Leonardo the leader, Raphael the rebel.

This optimal distinctiveness is a key part of why we tend to identify with only a single character.  There is always a healthy debate between fans about the best character (which is part of the fun) but not within any individual fan; internally, I’m not conflicted about which character I identify with because they are designed for me to have a favorite.’

Moreover, this elimination of duplication also makes the division of labor easier.  It is clear that in a situation involving science, you go to Donatello; there is no conflict about that as the right course of action.  If Shredder has invented some super shrink ray, Raphael is not your turtle and he’s not offended, because he knows that he has his own area of expertise.  You might like him better than Donatello, but it is still clear what his function is within the greater whole, because his optimal distinctiveness ensures it.

This is part of what makes the Turtles feel cohesive.  They are literally the perfect family, fighting only because their values are actually distinct.  It is like the ideal Socratic debate, with everyone arguing from a clearly differentiated vantage point.

It also makes them prepared for a wider range of situations.  If you think of all the Turtles as being relatively equally skilled, optimal distinction means that those skills will be distributed over the largest possible surface area, making it maximally likely that a problem will fall within the covered domain; if you draw the four kissing Venn diagrams, you can immediately see how overlap would lower the total area.  The Turtles can deal with both an alien brain in a robotic body and a mutated four-armed fly because they’ve got very different angles to work on those problems from.

Unfortunately, in reality, teams that form organically rarely assort into optimally distinct configurations; that it happens in TMNT is fiction right up there with the existence of radioactive reptiles.  We’ve all experienced this problem for ourselves in the complicated negotiations of childhood imaginative play: you get to be Leonardo on Monday, he gets to be Leonardo on Tuesday, she gets to be Leonardo on Wednesday, all because being the leader is the popular archetype.

But in companies, unlike childhood friend groups, we have the relative luxury of being able to design our organizations.  We don’t have to rely on organic creation and can instead focus on creating optimal distinctiveness to take advantage of the benefits it provides.

The easy version of this is role.  On my behavioral science teams, I always have trios of Project Manager, Quantitative Researcher, Qualitative Researcher, with a flexible shared pool of Interventionists, to ensure that there is significant difference in terms of both responsibility and viewpoint.  Because roles are hopefully well-defined, they make for handy shortcuts to the kind of dispersion we want.

But role is only one way of looking at optimal distinctiveness.  Each of the Turtles has a signature weapon, a different fighting style, even a different headband color.  It isn’t just role, it is also skills and personality and culture and all the many things that make up a person.  And that diversity is key, both for resilience across a variety of situations and for increasing the chances that team members simultaneously feel both special and unique.

People often underestimate how important that feeling of uniqueness actually is.  Google is famous for their research showing psychological safety as the best predictor of high performing teams and managers usually take this to mean an absence of negative consequences for risk taking.  But if you look at the actual survey measure of psychological safety Google uses, you find this gem:

Working with members of this team, my unique skills and talents are valued and utilized.

If that doesn’t describe the Turtles, I don’t know what does.

There is probably a need for an entire followup article on how you actually go create an optimally distinct team and maybe I’ll eventually get around to writing it, but for the moment, I’ll say that I think it starts with the job description creation and hiring process.  We already talked about how well-defined roles create optimal distinction, so to the extent that the JD is truly the definition of a role, hiring managers should be putting in far more effort than they generally do.

But I also think we underutilize the interview process as a point of creating differentiation.  You aren’t just interviewing a set of skills that match a role; you’re interviewing an entire person, with all the associated complexities.  It makes sense not to simply replicate what you already have, since recruiting your fourth Michelangelo means missing out on your first Raphael.

Side Note: I was a Donatello kid.  Which is not shocking, given my tech/science geek bent, but in many ways it was actually a rejection of the others.  I’m not fun (just ask my friends) so Michelangelo was out.  I’m not a leader, so goodbye to Leonardo, plus he spent entirely too much time having some sort of internal conflict that I still don’t really understand.  And I certainly didn’t want to think of myself as the hothead rebel, although I may have turned out more like Raphael than I intended.  But when I see Donatello now, the truth is he was sort of a flat character.  No internal conflict, no genuine emotion, just a shallow fixation with problem solving; he’s the Phillips head screwdriver of the series.  Maybe that still is me, mostly fixated on the solutions, but without any great emotional depth.  Time will tell; maybe I’m just Casey Jones, waiting to swoop in with a hockey stick.

(I’m vocal in my support of the work First Round Capital has done to help the entire entrepreneurial ecosystem and the First Round Review is a big part of that. When they asked me for thoughts about what candidates should be asking in their interviews, I sent the below; the finished article, with tips from Aubrey Blanche, Adam Grant, and others, is a good read with over 40 questions.)

Here’s the thing about candidate questions: I always try to ask myself “Would I change my decision about working here based on the answer?”  If you wouldn’t, then there is no point in asking the question.  So focus on dealbreakers: what would make you walk away?

One way to do that?  Look at what has made others walk away; Glassdoor is great for this.  One of the truths of any situation is that absent different inputs, you can expect the same outputs.  So talking about why others left can help you understand the inputs and outputs involved in what might make you leave.

  • If you’re interviewing at Coinbase, the question might be “Are you willing to say the words “Black Lives Matter”?  Is your boss?”
  • If you’re interviewing at Toast, it might be “Can you comment on your labor practice violations?”
  • If you’re interviewing at Away, it might be “Why did your VP of People and Culture resign as soon as your CEO returned?”

You may not choose to walk away from those companies as others did.  But if you can understand why they did, you can make a more informed decision.

You could do the same thing based on issues you feel strongly about.  I tend to ask about how they approach compensation and particularly compensation fairness, since it is a dealbreaker for me and I didn’t ask that often enough early in my career.  I also focus on factual questions that don’t have to do with how persuasive they are: what is the title of the most senior underrepresented person at the company?

Side Note: Many of the answers in the article are rooted in privilege but here’s the reality: most job seekers don’t feel like they’re in a position to say no to an offer, especially in this climate.  I am often reminded of Shakespear’s line from Romeo and Juliet: “My poverty, not my will, consents.”  But even if you’re making a decision based on your poverty, it is important to understand the will, because there will come a time when your poverty is not so strong and you want to remember your options.

(I was talking with a mentee about her sister’s admissions essay for college and she asked if I would share mine. And so I’m doing that publicly, because the world is better with transparency; as a first-gen student, I didn’t have access to examples and might have done better if I did. For context, this was written for Swarthmore College, which I ultimately attended as class of ’05.)

I do not know why I find this significant but it is on my mind as I read the essay guidelines for Swarthmore.  And so I write…

I hate time zones.  Through the miracle of a sun that determines how we set our watches, it is 6:37 in the evening here in Hong Kong and 2:38 in the morning at my home in Oregon.  As I sit at my computer, struggling with a decision that is tearing my apart, my parents are asleep, unaware that I so desperately need their guidance.  As much as I need the air that whispers past my lips, in-out, in-out, I need to here my father’s voice reminding me that only I can choose.

Suddenly, forcefully, I am struck with memories of home.  I cannot imagine the day when my parents pass from this world; I’ve been living essentially on my own for the last two years of my life, making decisions from day to day in a country that is not yet mine to call home, yet still I turn to them in my times of deepest need.  My friends insist that I am the only person they have ever met that actually likes his parents.

I am a rare thing here at my school in Hong Kong as well.  Though I spent my life taking it completely for granted, I have discovered a special distinction that I never know was my own; I am of a rare species, an American-accented “native speaker.”  Besides getting me an illegal job as an English tutor (I rationalize the fact that I do not have a worker’s visa with the comfort that my job is that of a public service), it has also qualified me to be petitioned by literally dozens of classmates, wanting the stamp of approval from not only a native speaker, but an American, as if being from the US gives me the special power to discern exactly what colleges want in a prospective student.

Though I have never admitted it, the responsibility frightens me.  The essays I read are a tangible part of people’s futures, a bit of their hopes and dreams put down on paper.  I feel a bit guilty at my doubts about college as the next step in my life every time a foreign student tells me that they are basing their entire future on getting into a prestigious university.  Without the big name attached to their diploma, they will describe themselves as failures and worse, so will their family and friends.  A friend from Pakistan is not allowed to apply to any school that is not first tier.  “My father refuses to allow me to go anywhere else,” he said.  “Its either Ivy League or nothing.”

I cannot understand that kind of pressure.  When I told my parents recently that I was thinking of deferring college for a year or possibly not going at all, my father just laughed over the long-distance line.  “We trust you to do what you think is best,” said my mother.  “Just keep your options open.”  Everyone else I had shared my plans with looked at me like I had the plague, and started pointing out why I couldn’t get a 1590 on my SAT’s and not go to college; I could have kissed that brown plastic telephone as yet again, my parents reaffirmed my faith in humanity. 

But now, the brown plastic telephone sleeps in its cradle and so do my parents, so many miles away.  Now, when I so desperately need them to reassure me that I will get through this.  If only it had come a few days from now, when my parents will visit Asia for the first time in their lives and hold me once again.  With my brother in Africa and me in Asia, this was the first Christmas my family was not all together.  I tried not to notice.

Today, I notice.

Even with my own college applications gathering virtual in my computer, I spend much of my day wading through the essays e-mailed to me by my fellow students.  It sort of strange cosmic pattern, all the e-mail I get seems to begin with the words “Dear Matt- How was your Christmas?  I’m very sorry to give you more work but…” and have two or three essays attached for proofing.  All, that is, except for one girl who managed to send me six essays and a peer review, and another who just wanted to wish me a Merry Christmas.  She isn’t applying to college until next year.

One mainland PRC student was particularly polite and so I started on her essays almost as soon as I received them (if you don’t at least try to be courteous, your essay goes to the bottom of the stack; hey, I’m human.)  Her first piece told of her dedication to honesty above personal gain.  She told the eloquent story of how, in a local scholastic competition, she tied with another student for the highest score.  In order to break the tie, each student was offered their choice of a 10, 20, or 30-point question.  If answered correctly, the points would be added to their score; if incorrectly, the points would be subtracted.

Her opponent was forced to pick first and, after choosing a 20-point question, he answered incorrectly and was docked the points.  All this PRC girl had to do was choose a 10 point question and she would be the winner by default, a course of action that seemed so logical that the match judges were about to call the winner as she sat there, until she suddenly exclaimed she wanted a 30 point question.

“At that moment, I was wandering between honor and honesty. I was confused and unable to abandon either of them. Both seemed important to me but I could not own them at the same time.”

In true Hollywood style, she missed the question and first place, all because of her honesty and integrity.  I was touched, even to the point that I stopped reading and went into the next room to share her morality with a friend.  “This is why I like reading these essays,” I told him.  “Every time, I find out how truly beautiful people can be.”

And then I read her second essay.  Equally touching, it was the story of how she had struggled upwards from poverty, winning a scholarship to a prestigious school full of wealthy children who she was desperate to impress.  She was ashamed of her job selling newspapers on the street and was terrified that someone would discover her secret, until finally, a teacher learned of her struggle and took her for a life lesson at a luxurious hotel.

She said, “look at the receptionists in front of the gate. Every day, their job is to open the door for millionaires, show the way to them and carry the luggage for them. Their manners are neither overbearing nor servile, but polite, confident and appropriate. They may not necessarily admire those millionaires. They have their own satisfactory choice. That’s enough. Money is not everything. As soon as you know your choice is fit for you and accept everything in your life bravely, you will acquire self-esteem and confidence.”

I almost cried as my stomach constricted in my stomach and destroyed my former warmth.  It was another truly touching story.  Unfortunately, I’ve heard it before.  A year ago, another PRC student asked me to read this exact same application essay.  Not similar, not nearly alike, but identical.  So much for honesty and integrity and the hard road walked.  In order to compete for a single spot, a first prize, this girl is taking every 10-point question she can.  And now I’m involved.

I want to hate her for making me have to make this decision.  There is no moral guidebook for this, and I am completely lost.  You wanted to know about something that has changed me; here is my inner earthquake as it happens.  One essay and so many small wounds.  Can anything be trusted enough to inspire?  In the end, after all this, how can I ever convince you that this essay itself is truth, something to be honored and treated with importance?  Beyond that, I am left with a decision to make.  To do nothing is to condone a direct lie, to be a silent partner as she signs the line that says this essay is her original work.  Or I can tell her that I know and risk a confrontation that may do more harm than good.  Or I notify the university itself and deprive her of the opportunity that means so much to both her and her family.  Can I justify that, stealing away someone’s future because they deliberately lied?

More than anything, I am hurt.  Her lie is a betrayal and however irrational, I am angry with her for making me question every essay I read from now on.  There are some questions we shouldn’t have to ask.

9:04 pm.  Another hour and the sun will be up in Oregon, its 6am rays streaming in my parent’s window and lighting up their world.  Another three hours and it will be safe to wake up the brown plastic phone and seek reassurance in the humanity of family.  My father will advise me on a course of action, or more likely, remind me to trust myself.  Someday, I’ll figure out what that means.

(This is a post primarily about the results of an experiment, but I did build a tool as part of it that you can use to gather behavioral feedback from former coworkers; you can go directly to WorkWithMeAgain.com to use it for free without reading about the journey or my data.)

It started, as so many of my projects do, with a conversation on Twitter. I proposed a public Glassdoor-like system with a single rating: would you be willing to work with this person again? Slice that by demographics and we might have a tool worth building to increase inclusion in the workplace.

After much healthy debate about the risks (it could be used as a harassment tool, for example), much of the value seemed to stem from the collection of feedback itself. Shifting away from the goal of a public system for discovering bad actors, it is worth asking: does the average person even know whether former coworkers would want to work with them again?  

I certainly don’t and especially not in any systematic way; despite being much-touted by thought leaders in the management space, organized feedback collection is relatively rare.  For years, I’ve had a link at the bottom of my email that allowed people to leave me anonymous feedback via Google Form and I’ve received plenty of responses, some helpful and some less so (apparently, someone doesn’t like my cowboy boots).  But I never collected demographic data or any concrete behavioral metrics, like willingness to work together again.

So I started building.  I iterated through several versions of both the survey questions and the analysis before arriving at the results I’ll talk about today.  The dominant behavioral measure is a simple 1-5 scale: “Given the opportunity, would you actively avoid or seek out working with me again?” With that, I asked gender and ethnic identity, age when we worked together, reporting relationship, and recency of the relationship.  

To avoid as much bias as possible, the final survey was sent out indiscriminately to ~800 people I’ve worked with over my ~15 year career.  The sampling wasn’t perfect, because people have moved between companies, I’m not connected to everyone I ever worked with, and many people who have worked with me don’t have LinkedIn profiles (due to socioeconomic skews of who is on LinkedIn).  But this whole project is a lesson in not letting the perfect be the enemy of the good; it is easy to find reasons that feedback will be incomplete, but that isn’t a reason to not get feedback at all.

In all, I received ~70 completed surveys, which seems extraordinarily high given what we know about average survey response rates.  Since I phrased the survey request as an experiment, the results of which I was committed to publishing, there may have been some increased promoting pressure but I’d like to believe that we might actually be on to something here; people want to give feedback, if we choose to hear it.

Before we get into the result that drove the title of this post, let’s start with resolving one important concern that came up in the Twitter discussion: that many people would choose not to add demographics for fear of being identified.  This didn’t appear to be the case, as only ~10% chose not to volunteer full demographics.  Assuming that response rates were high enough, it is thus likely that meaningful demographic filters can be applied.

Now the results at a high level, with the boring stuff first.  If you want the nitty gritty, I’ve linked to a copy of the report so you can see what you’ll actually get if you do this yourself using WorkWithMeAgain.com, although I have redacted the free entry portion to preserve the anonymity of respondents.  I also hadn’t yet added the recency question (which was the great suggestion of Leigh Honeywell) in the version of the survey I sent out, so there is no data for that section.

The basic finding is that statistically speaking, there were no significant differences in the averages within demographic groups.  Men/women/non-binary people and white/non-white people were equally likely to want to work with me.  And the averages were all above three on the scale, meaning generally all of those groups would seek out working with me again.

Does that mean I’m free of bias?  Of course not; as a white male, I absolutely have racist and sexist associations that express themselves as behaviors, despite my intentions.  What it does mean is that at least in this sample, these biases were not large enough to meaningfully affect any particular demographic subsegment, which is a good thing.

There are some interesting correlations that are neither good nor bad.  For example, the older someone was when they worked with me, the more likely they are to want to work with me again, which means I may need to change how I relate to younger people in the workplace.  I can come across as condescending, so this is a known problem that I actively work on, and that data backs it up.

Where the data starts to get interesting is when we look at standard deviations.  So rather than how much would people seek me out generally, how much disagreement was there within the group about that behavior.

I am, unsurprisingly, a fairly polarizing coworker.  I’m outspoken on social issues, have a distinctive personality and work style, and frequently clash with others who I perceive as acting against the group interest.  I’m not unaware of this, although I do vary at times in how much I see it as a strength, weakness, or simple fact.

This polarization, however, is not evenly distributed.  Even though men and women/non-binary people don’t differ in their average desire to work together again, they do differ in their polarization: women have much stronger feelings about avoiding or seeking me out, as do people I managed and people who worked with me across teams.  In my case, because I have enough data to cross gender and ethnicity, the finding is really around white women: men and non-white women consistently would seek out working with me, while white women are basically either 1s or 5s.

This is in someways rather puzzling. For example, I once lead a customer service team of ~100 people who were almost all non-white women and subsequently outsourced the function, resulting in a significant layoff. While I remain convinced that it was the right decision that was made in the best interest of our customers, all of those people would be entirely justified in avoiding working with me again. So I would have expected a potential main effect of ethnic identity or at least an interaction with gender.

I’ll confess, I don’t know what the varying feedback from white women means yet.  And that’s a fine thing: survey data like this rarely gives us a conclusive answer.  Instead, good quantitative analysis clues us in to where to do qualitative analysis; data tells us what, not why.  For me, the survey is a jumping off point to conversation that I can use to improve and I intend to seek out further conversation, particularly with white women who I’ve worked with before.

Feedback isn’t a silver bullet for changing our behavior.  But it can be a start.  Personally, I’ve committed to publishing this data and to following up with the 800 people in the initial sample, as well as periodically adding new co-workers with each new job.  As part of that, I’ll also resurface the link to the anonymous feedback form I already have setup; for any of the 10 people who would prefer not to work with me again, if you are willing, I’d deeply appreciate you using that to give me feedback on what I could have done differently to change your experience with me.  Finally, readers of the writeup can use the same form to give suggestions on actions I should take or email me at matt@mattwallaert.com if you don’t need anonymity.  I want to do better, I’m committed to doing better, and I hope when I one day rerun this experiment, the results will be different.

If you’re committed to doing the same kind of work, the tools I used are now publicly available at WorkWithMeAgain.com.  It will allow you to copy the Google Sheet that contains all the calculations, as well as instructions on how to launch the survey to your former coworkers.  I am absolutely convinced that putting in the effort to gather meaningful behavioral feedback can be a key component in how we change our individual behaviors, as the collective culture is simply made up of those individual behaviors; if enough of us do this, we can live in a substantially better world.

Side Note: As usual, this project required collaborators: frontend coding from Grayson Null and design by Patricio Hunterkhozner. Plus feedback from a WhatsApp group full of folks working toward inclusion and 70 people who were willing to share their thoughts about working with me again. It has been a good year for WorkWorthDoing projects (Mediocre White Men and Project 4255, with another on the way soon) but they don’t happen without a bunch of people willing to work at reduced rates and in suboptimal circumstances.

Posting my survey results, online and unfiltered, is making me nervous. And that’s a novel feeling, since I usually don’t really feel nervous in circumstances that don’t involve heights (I really don’t like heights). That makes this a rare brave moment, because if I don’t normally feel the feelings, I’m not actually being brave: it is easy to do something with low inhibiting pressure. There is no concrete outcome I am afraid of…but I’m certainly afraid.