How is optimization changing, and what should I do about it?

5 min readFeb 22, 2024

The world of optimization doesn’t stand still. What held true four years ago might not mean the same thing today.

Today, we’re celebrating a pretty big milestone at The Good.

We have just released the 100th episode of our podcast, Drive and Convert. It’s been an incredible four-year journey, packed with actionable insights, industry trends, and plenty of laughs along the way.

But here’s the thing: the world of optimization doesn’t stand still. What held true four years ago might not mean the same thing today. And that’s okay! At The Good, we embrace evolution. We’ve grown alongside the industry, constantly learning, refining our perspectives, and even admitting when we were, well, wrong.

So, for this edition of the Good Question, I wanted to do something special.

We’re taking a deep dive into past podcast episodes, revisiting old predictions, analyzing how our thinking has evolved, and sharing the latest, most up-to-date advice we’ve gleaned from experience. It’s a retrospective, a celebration, and a chance for you to unlock even more optimization wisdom.

All of these episodes are also available on our website or your favorite podcast streaming platform.

What I Said: Benchmarks are bullshit

What I Learned: Singular benchmarks are still bullshit…but sophisticated, multi-KPI benchmarking can provide great value

Okay, let’s talk about benchmarks. You’ve probably heard someone at The Good say something like benchmarks are bullshit.

And here’s the thing: I still stand by my core message–blindly chasing industry averages or competitor performance is a recipe for disaster, or worse, mistakes. The logic was sound because once you reached that benchmark, you wouldn’t stop optimizing, right? It just meant you had a new, higher bar to chase.

But here’s where I want to clarify some of the thinking. Benchmarks, despite their limitations, still offer valuable insights. They can expose industry trends, identify potential areas for improvement, and even serve as a starting point for setting your own goals. However, the key lies in using them holistically and strategically.

You can’t build a bridge to your goals with benchmarks alone. So, how do we build a robust, data-driven bridge to optimization success? By taking a holistic approach. Here’s the new perspective:

Benchmarks are guidelines, not goals. They offer a starting point, not a finish line. Use them to understand industry trends, but tailor your goals to your unique audience, business objectives, and conversion funnel.
Dive deeper than a single metric. Conversion rate is important, but it’s just one piece of the puzzle. Look at other metrics to paint a complete picture of your website’s performance.

What I Said: You should test everything.

What I Learned: You should validate your decisions.

We are guilty of preaching the “test everything” mantra. Over time, that has evolved into a more thoughtful and realistic approach to optimization.

I still believe in data-backed optimization, but “A/B testing everything” isn’t always the answer. It can be slow, expensive, and sometimes unnecessary. That’s why we’ve adopted more diverse strategies, including rapid validation.

What’s the difference? Instead of A/B testing every hypothesis, we use research and quick experiments to validate optimization opportunities. This allows us to:

Move faster: Save months-long A/B tests for big opportunities and use quicker tests to get results when we need to make fast, user-tested decisions.
Minimize risk: Validate ideas before investing in full-scale changes, saving time and money.
Continue being data-driven: We still use data to guide our decisions, but we gather it through various methods, not just A/B testing.

But keep in mind that just because rapid validation is faster, it doesn’t mean that it’s cheaper. You still need to invest in research and expertise, but the return can be even higher.

So, what does this mean for you? Ditch the “A/B test everything” mentality and embrace a strategy that incorporates all of the kinds of research and testing available to you. It’s a more efficient and data-driven way to optimize your website and achieve your optimization goals.

What I Said: Amazon’s checkout process will shape how other brands design their checkout experience.

What I Learned: Amazon focused on two other initiatives.

Back in Episode 14 of Drive and Convert, I played a fortune-teller, predicting the future of CRO. But my crystal ball may have been dusty. One prediction I made was that Amazon’s checkout would revolutionize the way we buy online.

Amazon didn’t change the game; they changed their focus. Instead of checkout domination, they doubled down on “Buy with Prime” and “Amazon Pay.” This move crushed standardized third-party checkouts like Bolt and left PayPal scrambling.

But credit cards have demonstrated surprising staying power. While Apple Pay, Google Pay, and Amazon Pay offer slick one-click options, credit cards are continually used for their widespread acceptance and familiarity.

So, who won the checkout war? The consumers–with a plethora of secure, speedy payment options. While I was wrong about the Amazon takeover, the overall checkout experience has undoubtedly improved.

Predictions are tricky, especially in the ever-evolving world of optimization. But the silver lining is that Amazon’s focus has shifted towards better user experiences and customer empowerment, which is always a win.

Continuous Learning, Continuous Improvement

As you can see, ideas evolve and predictions can go awry. But the beauty lies in embracing this continuous learning journey.

The more ways we look at optimization, the smarter our solutions get. By acknowledging change and embracing new perspectives, we unlock better strategies for you.

Much like optimization, learning is also a journey, not a destination.

Dive into all 100 episodes of Drive and Convert on our website or your favorite podcast platform. Each episode is packed with actionable insights, industry trends, and maybe even a few more confessions along the way.

What have you changed your mind about or evolved your thinking on recently?

Originally published on thegood.com