Have you ever sat in a meeting where a project was described as a success, yet all the details of failure that led to that success were left out?
Alternatively, have you ever watched while data was cherry picked to make things seem rosier than they actually are?
These are common embodiments of a very common, yet little known, phenomenon called "Success Theater".
Success theater is, at its core, an informal operating system that says to employees: "you're expected to win, and you should only discuss wins. Failures need not be exemplified."
More concretely, success theater describes "the efforts that we make to make things look good, even if actual performance isn't good or getting better."
Ultimately, it is an avoidance of data that conflicts with your opinions. It's a fear of confronting failure or uncomfortable conversations and reflection.
I like this definition (and the whole article) from John Cutler:
"Success Theater is celebrating hitting the quarterly goal without acknowledging the corners you had to cut and the people who will have to clean up the mess. It's listening to someone rattle off vanity metrics. And it's being told you're not a team player for having some doubts about a recent 'win.' ... And it's rampant in our internal meetings, blog posts, and presentations."
Success theater is … exactly what it sounds like it would be.
Of course, in the real world, failure is going to happen. In fact, it needs to happen in order to succeed.
Tons of value can come from documenting and sharing lessons from failed campaigns. Survivorship bias can warp the expectations of both leaders and new team members, and digging up your 'graveyard of knowledge' can help elicit new ideas and insights that could contribute to major breakthroughs in marketing and optimization.
In general, it's best we confront failure honestly, instead of hiding from it.
The Ruinous Path of Success Theater: GE's Horror Story
One of the most popular stories about success theater is a Wall Street Journal piece on General Electric.
According to the story, former CEO Jeffrey Immelt was constantly optimistic in the face of future projections. In addition to his optimism and "can-do attitude," he and his fellow top deputies also shielded themselves from any bad news or data that contradicted that narrative.
Unfortunately, the results were catastrophic:
By 2018, the stock price had dropped by roughly 44%. In the same year, they announced they would cut their annual dividend for only the second time in their 125-year history.
They also announced they were taking a $6.2 billion charge in their fourth quarter related to their insurance operations and needed to set aside $15 billion over seven years to bolster insurance reserves at the GE Capital unit.
Finally, they had to restate their earnings for 2017 and 2016, and the Securities and Exchange Commission (SEC) began investigating them for these accounting issues.
Identifying and Diagnosing Success Theater
GE is an extreme example of Success Theater starting at the top and trickling down, eventually taking over the company. But smaller and subtler versions of it could happen at your company.
Liane Davey outlined some signs to look out for:
- Everything is an opportunity (If your organization finds creative ways to describe issues to obscure the real issues, worry)
- Every plan is a hockey stick (If your company is always predicting that next year will be the year it all comes together, you've got a problem)
- You hit the number at any cost (You know you've got a problem when you start taking short-term gains that create long-term pain)
- You shoot the messenger (If the harbingers of trouble can no longer be heard, you have a problem)
Additionally, I've found a simpler sign — if you notice data cherry-picking and storytelling frequently (and on purpose), that's a strong symptom of underlying success theater.
Success theater can also show up in our industry-wide narratives. You can see it in the constant flurry of 315% conversion uplift case studies shared on Twitter, and at the top of GrowthHackers (and also, of course, in the glorious rise and tremendous fall of WeWork).
How to Combat Success Theater
There's no silver-bullet solution to avoid success theater, and it will probably always exist to some degree. We just want to curb it to the point where it doesn't poison productivity.
I talked to several growth leaders and also wrote down my own favorite ways of combating success theater, which I'll share now.
1. Write About and Share Failed Experiments
Our team at HubSpot places a huge value on learning, and not just on reporting our wins week-over-week.
We keep a weekly calendar spot to write down our learnings for the week. We talk about those insights during Monday meetings. We're all encouraged to write internal posts about failed campaigns — or even mediocre campaigns. We don't want to build a graveyard of knowledge where we're only writing about the big winners and seeking applause.
We also have a company-wide Failure Forum where we share our failures and celebrate the risks we take on the path to innovation.
When Joanna Lord was at Porch, she talked about a ritual they developed at the company involving a pink fuzzy animal named Mr. Sparkles. Whoever failed the biggest each week would get Mr. Sparkles. As she put it:
"You put him on your desk and it's like this badge of honor that you like did something so bold that you literally messed up the site badly.
And you know what I love? You see my CEO walk around the room and he's high-fiving the Mr. Sparkles owner. And people are like, 'What did you do? What did you do to get Mr. Sparkles?'
But the reality is we've made it a positive thing. We've made it a badge of honor. You are living out the Porch-y way in being bold. What can you do in your culture to make it fun and acceptable? And almost, you know, become famous for it."
In a broader industry-wide context, blogger and marketing consultant, Ryan Robinson, has long been an advocate of transparently sharing the behind-the-scenes stories of his own business failures with the readers of his blog:
"I've launched several businesses over the past decade, and most of them have landed somewhere between mixed results and utter failure. I go out of my way to highlight those experiences in long-form stories to the readers on my blog, because it's important to illustrate that the path to achieving meaningful results will be filled with missteps and lessons to be learned along the way. My failure-related articles are consistently the most popular with my readers."
2. Be Careful With Case Studies
I've ranted about CRO case studies a ton by this point, so it's suffice to say I don't trust most of them.
Even if the data is accurate and you're not reading about a blatant false positive on a sample size of 14, you're likely looking at a PR piece that is almost certainly subject to Survivorship Bias.
Very few companies and thought leaders are incentivized to write about their failures and inconclusive experiments, so you're only seeing the tip of the iceberg.
Sure, you can absolutely get some inspiration from case studies. But don't look around in exasperation and think that everyone's winning except you. It's all an Instagram-esque illusion.
3. Embrace Transparency
Shannon Callarman, a Content Marketing Specialist at ShipBob, told me about a cool ritual they've developed at her company to share candid feedback and ask probing questions:
"Every month, ShipBob's leadership team leads a forum called 'Ask Me Anything' that allows employees to ask candid questions about the growth of the business and internal operations, and they'll get an honest answer."
When leaders embrace uncomfortable conversations and open themselves to feedback and questions, it shows the rest of the organization that it's alright to do the same.
Along the same lines, I find it worthwhile to explicitly outline and publicly post your team and company values. This helps create both an explicit and implicit operating system where your employees and leaders are encouraged to embrace your values (in this case, honesty and transparency). A great example of this is this post from BounceX's CEO, Ryan Urban.
4. Embrace Being 'Wrong'
In Ronny Kovahi's talk at CXL Live a few years ago, he brought up a great point on A/B test results: the best case scenario is when you test something you thought was 'meh', and it wins.
If you thought it was going to win, however, and it wins, you haven't learned much. Value in experiments comes when the "absolute value of delta between expected outcome and actual outcome is large."
Yes, being wrong is the best possible thing that can happen to you when you run A/B tests!
This is also the strategic basis for Andrew Anderson's Discipline Based Testing Methodology — test a wide variety of options, some of which might be totally radical, and let the winner surprise you (you'd never expect Comic Sans to win a font test, right?).
Celebrate being wrong and learning new things.
5. Hug the Messenger
When leaders expect only good news, those who bring up problems or constructive feedback are looked upon unfavorably.
In companies like this, the messenger is ritualistically shot.
A better way to act is to embrace those who are brave enough to ask interesting questions and point out possible flaws in the data.
Nigel Stevens, founder of Organic Growth Marketing, encourages an embrace of failure internally and externally:
"Whenever I only hear of 'wins' for a while, I start to get nervous. Because that tells me we're not proactively sharing — and learning from — the things that don't work."
Stevens adds, "Fortunately, we've developed a very healthy culture of saying 'hey, this completely flopped, and now I'm sharing it with others so you all know.'"
6. Lower the Cost of Failure (and Experimentation)
One of the most impactful things you can do within an organization, especially if you're working on growth or conversion optimization, is to lower the cost of failure.
A/B testing does this by nature — you can only lose as much as your losing variant lost during the course of the test.
However, you can further decrease the cost of failure by making it easier and cheaper for everyone to set up and run trustworthy experiments.
After enough optimization, the big and easy wins stop coming so frequently, so growth looks a lot like this:
This is, of course, a riff on Nassim Taleb's ideas on optionality — he uses a chart similar to the above to describe trial-and-error tinkering and how it leads to stark innovation.
Simply put, it's impossible to achieve any degree of outstanding innovation without a tremendous amount of tinkering. The more at-bats you allow, the more optionality/luck/upside you can generally collect.
Content marketing blogger and expert Levi Olmstead mentioned this attitude (fail fast) being key in previous startups he worked at:
"A core value we often repeated was 'fail quickly and continuously, iterate quickly and continuously.' Without failing, you can never learn from past mistakes. In my experience, many ideas have strong pillars but they're not full-fledged ideas. To create a strong, sustainable strategy, you need to learn how to fail and how to turn those lessons into future successes."
Making experiments cheaper and easier to run isn't an easy feat, but luckily there's a lot of material out there on the subject (I'd start with this paper from the booking.com team).
7. Invest in Trustworthy Data
Data attribution modeling is still a large digital marketing challenge today. With different data models like first-touch attribution, last-touch, multi-touch, and others, teams need to choose one and stick with it.
Adam Enfroy, who runs the popular marketing blog AdamEnfroy.com, witnessed this data challenge firsthand while managing digital teams for different SaaS companies, including BigCommerce:
"Success theater often runs rampant in companies when teams aren't aligned on which data platform is the final source of truth. This leads to different teams reporting on conflicting data sources that drive the (often misleading) narrative they want to tell."
Let's take a weekly business review meeting as an example.
Consider this — the digital marketing team pulls from Analytics Platform A because the ROI looks favorable with last-click attribution. At the same time, the business intelligence team pulls from Analytics Platform B, which doesn't look as good but is more aligned with finance. Then the partner team pulls from a different data source entirely so they can take credit for more revenue.
In the end, this leaves confused executives looking at three different data sources — and listening to three different stories — while no actionable insights can be gleaned.
To avoid success theater, invest in trustworthy data and align teams on which platform is the ultimate source of truth. This involves strong alignment between BI, marketing, sales, and partner teams.
Additionally, seek to stress test and constantly monitor the veracity of your data. As my friend Chris Mercer always stresses, "trust, but verify."
8. Invest in Education
Old joke …
CFO asks CEO: "What happens if we invest in developing our people and they leave us?"
CEO: "What happens if we don't, and they stay?"
This is true of any business function, but I find it particularly true in more technical fields like growth, conversion optimization, and SEO.
There many ways to do this — invest in a library for your office, start a company book club, send your team to conferences, pay for training programs like CXL Institute or Reforge.
HubSpot invests a ton in education and I always feel supported in learning new things (currently taking a course on Python and machine learning from Udacity).
Small companies can do this, too, though (in fact, they need to). Ben Johnson, Head of Content at Proof, has seen the impact of this mentality at his SaaS company:
"We're a small team of 15, so a lot of time, you're not going to be able to learn how to do something from someone in-house. For that reason, we're always reading, meeting with more experienced individuals, and using our naivety as a strategic advantage."
Johnson adds, "Our leadership is great about encouraging this growth mentality across the organization — making introductions, providing funding for marketing conferences and programs like Reforge, and paying for a book per month. Overall, I think creating a culture of curiosity and helping your employees get the resources they need is a key part of building a growth culture."
9. Diagnose Narrative Fallacy
Storytelling is natural to humans, but it can also dilute efficiency and decision making. The Narrative Fallacy, popularized by Nassim Taleb, describes our tendency to ascribe a clean causal "why" to something that happened in order to simplify our understanding of the world.
For example, if you A/B test two headlines against each other, you may determine that version B won "because it invoked social proof", while someone else may attribute the win to "the clarity of new message".
A good way to transform your culture is to try to curb storytelling where you can, because while the narrative fallacy is limiting when ascribed to wins, it's detrimental when people try to explain away suboptimal campaigns.
Mark Lindquist, marketing strategist at Mailshake, mentioned one version of this is when people move the goalpost of what they define as "success" after the campaign is run (technically, this is known as the Texas Sharpshooter Fallacy):
"Consider an SEO who is brought into an organization to grow top-of-the-funnel leads. 18 months in, they've 50x'd organic traffic, but leads are only up 15%."
Lindquist continues: "A story you could tell yourself would be 'well, this is great for our brand. We're getting our name out there and we're on our customer's radars when they are ready to buy.' That may be true, but it probably isn't, and it certainly isn't based on any data. Start your marketing campaigns with clear goals, and if you don't reach those goals, don't pat yourself on the back for accomplishing something you never set out to do in the first place. Be radically transparent with yourself and your team."
Ultimately, the purpose of experimentation is to encourage innovation and to mitigate risk, and in the process, most of your ideas are likely to fall flat of what they were intended to do. The best next step is to dust off and iterate and keep learning and trying.
Success theater may feel good temporarily, but risks limiting the scope of the program. You should celebrate wins, but feel comfortable sharing failures, too.
No comments:
Post a Comment