Averages are a lazy AF way of measuring; enough with that, with Armando Biondi

Join us for a special keynote presentation from Breadcrumbs‘ CEO and Co-Founder, Armando Biondi.

[Transcript] Averages are a lazy AF way of measuring; enough with that!

Although transcriptions are generally very accurate, just a friendly reminder that they could sometimes be incomplete or contain errors due to unclear audio or transcription inaccuracies.

Armando Biondi

Hi, welcome everyone; it’s Wednesday the 28th. A pretty important day if you ask us. It’s a Hot Takes Live 3! 

Ready to rock and roll; we are going to give a few additional seconds more for people to start trickling in, and then we’re gonna kick it off.

In the meantime, we are very excited to have you here! Before getting started, here are a couple of housekeeping items. First and foremost, a little bit about what Breadcrumbs is. 

Breadcrumbs is Enterprise grade scoring for everyone. It’s designed for operators instead of data scientists, and it’s optimized for speed, observability, and flexibility. Last month, we released our biggest innovation yet; it’s called Breadcrumbs Copilot. You can use our AI-assisted tool to create your scoring model in seconds. 

Not days, not weeks, not months, seconds, and you stay in control by editing your scoring model at any time, plus you get rich analytics on each aspect of your scoring model performance. Even better, you can try it for free at the following link breadcrumbs.io/copilot.

If you’re curious and want to have a chat about it, just drop me a note at armando@breadcrumbs.io; more than happy to have a chat and show you around a little bit.

So what is Hot Takes Live? It’s the third edition, so everything comes in three. We have 30 incredible speakers for you today across three tracks, and it’s a very simple idea. 

It’s counter-intuitive insights, surprising data points, and thought-provoking points of view at the intersection of growth revenue and Ops. All in up to 10-minute sessions, not more than that! Plus, five minutes of Q&A. That’s the reason why it has been dubbed the TED for B2B Tech. 

So very excited to get started! We have really a full agenda for you today. We were mentioning three tracks stages: one is going to be the Marketing stage, one’s going to be the Sales and Product stage, and one is going to be the Ops stage. 

I’m going to stay on this slide for a couple of dozen seconds just to get you situated and orientated across what’s going on. It’s a lot, as you can see, so we are going to go through five sessions per track and then take a break around noon Eastern, just so that you can take a breath and us as well, and then we’ll resume the day. 

Also, you can notice that the sessions are staggered a little bit between each other, so there is always something going on. There is always something for you to attend to and listen to. If your session ends and you’re curious about jumping on the next one, and it might be a different track/stage, you can do that very easily. 

Also, this agenda is on the event homepage in case you don’t have it handy, but it’s going to be available for you throughout the day, so you can check it out over there at hottakeslive.com, or it’s going to be as a resource here.

So, switching gears to the afternoon, the three stages continue through a second series of sessions. Another thing to call out while you skim through the titles and decide what’s interesting and compelling to you, if you didn’t do that already, is the fact that you can see the speakers are really go-to-market senior leaders that are doing it right now. You can see senior-level speakers, VP-level speakers, as well as director and senior director-level speakers all in a go-to-market function in Marketing, Sales, Product, or Ops.

This is really a great opportunity for you to take a peek into their brain and listen in on the things that they care about right now, the things that they’re working on as we speak, and also have a little bit of a view on how some of the companies that you know and love (you can see very well known names over there as well as up and upcoming companies) are thinking about their go-to-market engines as we speak.

Then we’re going to wrap it up around 2.10 PM Easter, so we’re gonna try to be on time as much as possible, but you know, it’s a lot, so let’s see where we land.

All right, also before getting started, the last thing that I should mention is that, in case you didn’t see that already, we do have a free ticket to the HubSpot Inbound 2023 conference for you, so you will find the way to enter the sweepstakes in the chat, but FYI the winner will be announced on our social channels tomorrow June the 29th. 

All right, all good! Ready, Set, Go! Let’s get started with the first Hot Takes Live, which is my own. 

And here it is, “Averages are a lazy AF way of measuring; enough with that.” A little bit about me before jumping into the middle of my hot take. I’m the co-founder and CEO of Breadcrumbs; I’ve been a three-times entrepreneur at this point as well as a two-time senior leader. Plus, I’m an angel investor in about 300+ companies at this point; my LinkedIn profile is slightly out of date, so I should actually refresh that!

One of the things that I find thinking over and over again is the fact that most operators unconsciously default to averages when looking at metrics and KPIs. 

Case in point, have you ever heard or said the following, “Our trial to paid conversion is 2%, our average contract value is 4K, and our email open and reply rate is 41% and 3%, respectively.”?

Now, that’s an ok(ish) start, but be aware that you’re doing a disservice to yourself and your company by not going one level down. In fact, I want to show you two examples of what that means from a practical perspective to further contextualize and solidify this idea. 

Example number one, this is an analysis that Gary, my co-founder, did. We’re gonna have him over the Q&A to double-click on this further, but this is an analysis that was done for Hootsuite back in the day.

You can see that blue line at the bottom of the chart; that’s the average conversion rate of two percent that I was mentioning. You can also see that if you cohort different clusters based on their different behavior, a way more compelling and interesting narrative emerges; in fact, you can see the highest cohort that’s kind of the cream of the crop, three percent of the entire population, that’s performing incredibly well, 7.2X almost on the average.

And then there is a higher cohort which represents about 24% of the population, that’s still performing really well, +2.7X on the average. Then you have kind of a medium-ish and very, very low-performing cohort that represents, in the aggregate, 73% of the population and is underperforming by a little bit or a lot, 0.9X down to 0.1X on the average. 

Again one thing is the average. The other thing is what the cohorts are and what are the different behaviors/makeups that are contributing to that average point. 

So question for you: isn’t this a way more informative, educated way of having that conversation and a better way to focus on the things that move the needle? 

Second example; this is an analysis that they did for a portfolio company of mine. I was helping the CEO with the somehow newly launched commercial motion. We’re looking at about 300 commercial customers, and you can see that the ACV, the average annual contract value, for those was 4K. 

Now, for the ones of you that do know benchmarks with regards to ACV, 4K for commercial it’s barely okay, but not really. Ideally, you would want to see 5K as a floor for commercial, up to 15K, maybe 20K, maybe even 25K for your commercial customers. 

At first glance, not great, right? But then, if you double-click on different cohorts, you can see that the cohort under 5K represents about 78 percent of the population and has an ACV of 2.9K. 

Also, if you look at the 5K to 15K, a cohort that’s just under 20 of the population, interestingly enough, ACV goes up to 6.5K, so it’s now above that 5K threshold that we were mentioning.

Finally, there is also a 15K plus cohort that, again, the cream of the crop, just under two percent, has a dramatically higher ACV, up to 3X the previous cohort at 22.6K, and so really pushing through the high-end of that benchmark that I was mentioning. 

And so again question for you: isn’t this a better, more informative, as well as informed, way of having a conversation?

This leads me to my recommendation, or better, to my series of recommendations. 

Number one: always always always ask or get a cohorted breakdown of the key relevant data sets that you’re looking at. You don’t have to do this for every data point all the time, but when it matters, be aware that double-clicking on some of them actually helps quite a bit. 

Number two: expect to see a power-law distribution following the Pareto principle, which is the 80/20 rule. The 20% of the input will generate 80% of the output; there is always that relationship. Also, tradeoffs to parse through the data are going to be required, so expect to have to go through that exercise as a team.

Number three: data tells a story, but breakdowns tell a better story by putting into focus what really moves the needle. Way easier, way better to focus on the stuff that really matters and have decisions and strategies around that slice of your population that really move the needle and impact your revenue and growth objectives. 

Key takeaway. If you ask me, this is the one sound bite, this is the tweetable quote. Data-driven decisions: every one of us constantly talks about them; we are a data-driven company, I’m a data-driven operator, and we make data-driven decisions.

Yes, but data-driven decisions based on average data will only get you average results, and we can do better than that.

That’s it on my end; that’s my hot take for today to kick the day off! Now, Gary, if you don’t mind jumping in for Q&A so that we can unpack some of the things that we were touching on.

Gary Amaral

Happy to. Hi, Armando; great job! We have a few questions already in the chat, but I’ll start with one of my own, and then we’ll jump into some of those.

I guess for everybody listening, from a practical perspective, in terms of how it impacts your operations, how it impacts your revenue, why the cohorted view? Obviously, it tells you more granularly, but how can it impact your business?

Armando Biondi

The one thing they tend to think about is that as you get that deeper level of understanding of what’s contributing to the overarching number, you essentially are applying the decision framework. You say, “Hey, there is going to be a different makeup for each and every one of those cohorts,” and so tracking those independently but still in an aggregated way will give you a more informed point of view.

And so instead of saying, ”Hey, we need to improve our T2P conversion from two percent to three percent,” now let’s talk about what we are doing for that high-performing cohort and how does that high-performing cohort looks like.

Going back to the example of ACV, one thing is looking at ACV on average, and that would include an aggregate makeup of all those 300 commercial customers. But the reality is that some of them are not like the others, and focusing on having a teardown/breakdown of that will drive you to a deeper level of understanding and, as a consequence, different types of actions around it.

Gary Amaral

Yeah, I think you hit on one of the keywords that I think of: I think of focus, and I think of sacrifice. The average is a great bellwether, directionally telling you where things are going, but if you’re going to make operational decisions, i.e., I want to increase my ACV, well, perhaps the secret there is actually focusing on that segment that has the higher contract value and de-prioritizing the activities to that long tail of lower-paying customers.

If you’re thinking about increasing my open rate or my click-through rate on an email program, well, maybe it’s not about tweaking calls to action, or maybe it’s about wholesale, like eliminating an underperforming program as opposed to small marginal changes.

Armando Biondi

Yeah, it always comes down to trade-offs. When you talk about focus like they implied part of the conversation, which I think having breakdowns helps surface, is the fact that you know we are always constantly operating in a world of limited resources, limited time, limited headcount, and limited focus.

So you constantly need to make trade-offs, and so being able to have a better decision framework, to your point, focus on the stuff that matters versus de-prioritizing the stuff that doesn’t too much or doesn’t serve your overarching objective, it’s a better way of operating.

Gary Amaral

Let’s go to the audience; Vincenzo asks, “Can you further segment the ACV cohorts, i.e., SMB, mid-market, Enterprise?” And as an extension to that question, “Can you generalize the ROI for each segment of the cohort relative to investment in CAC?” 

Armando Biondi

Yes, absolutely, and you should, you absolutely should! One recommendation there, I usually go by the rule of three. Three or four cohorts are good enough. Some people and some companies might go overboard and start to be overly granular. 

I don’t necessarily think that being overly granular helps either because that increases the amount of work and time that you need to get that visibility and doesn’t necessarily improve too much the following conversation. Having five or ten or fifteen cohorts instead of three, sure. But how do you know we are helping the conversation? 

Gary Amaral

I think it’s totally kind of like, you develop your own framework for it. I’m a big fan of quintiles. Using quintiles gets you rapidly to the Pareto principle because it’s groups of 20%, so I think you can go overboard in the analysis. Still, as long as you think of a framework that works for you and you use that repeatedly, I think this becomes kind of second nature.

Okay, cool, so Christine asks, “Why not just take the median?” 

Armando Biondi

Ah, that’s a really good question, and hey, Christine! You guys will see her later on as a speaker. 

I think the median is, again, directionally useful. Still, you’re sacrificing the edges, and edges are, in some cases, relevant because they are not an aberration to your focus but point to what the future could look like. Having a way to represent edges in parallel to the median area, I think it is actually really relevant for a comprehensive conversation.

Gary Amaral

Yeah, so I think the median is great. It’s a great shortcut to buffer out the outliers, it’s a great way to get the noise of the extremes out of the conversation, and it’s actually a good way to set a target for your average, in my opinion.

But yeah, I think the granularity of these cohorted views are the ones that really drive to operational decisions, this idea of focus and sacrifice.

Right, awesome, Armando. We’re at time and so excited to get all of our speakers up and running.

To everyone watching, we will be kicking off in about a minute here on the main stage with Tim Kilroy, and all the other stages will be starting very shortly. Grab yourself a coffee, grab yourself water, and buckle in for a bunch of spicy hot takes. 

Armando Biondi

Let’s do it! Thank you, Gary.