Dashboards: The ad-hoc generating machine

Dashboards are the default mode of data communication in our industry. Our audience understands and expects them, and we’ve become quite skilled at building them. If you’re using the modern data stack, odds are you’ve built a ton of these. However dashboards are a really poor catch-all form of data presentation - they are complex to build, erect walls between the data and business team, and function as ad-hoc generating machines. In this blog post, I’ll explore when dashboards work, when they don’t, and how to make the call on what data presentation you need.

Dashboards are for operational oversight, not storytelling

Back in the 2000s, the gauge chart was all the rage in visualization - hard to believe, I know. Vendor after vendor would demo their product and recite some form of car metaphor when describing the purpose of a dashboard. This became cliche and eventually archaic, but looking back I think there was something to it.

A dashboard is best suited to managing a defined, repeatable process with agreed-upon KPIs, clear desired business outcomes, and understood and impactful real-world next steps.

When you have these elements and nothing else, you have a great dashboard. I call this data-presentation fit. You have data-presentation fit when the presentation, data, and use case work together to clarify understanding for your audience. We’ll explore this topic in future posts.

The car metaphor checks all the boxes for data-presentation fit; the auto company and driver agree precisely on how the car is to be driven and what actions result from the data in the dashboard. If it were otherwise, the dashboard would be a dangerous distraction from safely arriving at your destination. It’s the same with business dashboards, which are often confusing or raise more questions than they answer. This isn’t because the dashboard is poorly designed, but because a dashboard wasn’t the right choice of data presentation in the first place.

The ad-hoc generating machine

One sign that your dashboard has poor data-presentation fit is that it generates an unmanageable volume of low-value ad-hoc questions, as your audience gets suggestive information but isn’t clear about what the data means or what to do about it. This ambiguity puts huge pressure on your data team as they wrangle a mass of frustrated users asking non-strategic questions. It doesn’t add value and it’s frankly not fun.

I have seen many data teams respond by building secondary and tertiary dashboards - dashboards all the way down! At first, this feels very ‘agile’, but the end result is a massive, ungovernable collection of data assets full of cobwebs and tumbleweeds, where any given dashboard has only a few useful charts and nobody can really find what they are looking for. This is the point of maximum dissatisfaction for data analysts, who are working hard but not answering the most important questions.

High value vs low value ad-hoc

Here is the distinction between the type of ad-hoc you want to embrace and the type that results from a dashboard with a poor fit.

High-value, strategic ad-hoc answers novel, important questions just as the business needs it. These are the questions your data team needs to focus on, and the ones your analysts want to answer.

They often require nuance to explain. They cry out for a narrative focus to go beyond a collection of charts to tell the full story of the analysis, how it was done and why it matters. These are the ad-hoc requests that change the way you do business.

Dashboards with poor fit don’t often generate these kinds of questions. Instead, your data team and analysts are locked in a cycle of explaining where data comes from, how conclusions were reached, applying simple aggregation changes, or slightly tweaking existing dashboards. This is the kind of work that leads to analyst burnout.

In reality, you are going to have a mix of both types. Low-value ad-hoc isn’t unimportant, it’s just not impactful and becomes a problem when it gums up the machine and prevents you from doing high-value work. The goal is to minimize them by having the right presentation for the right context.

What about self-service dashboards?

What about a self-service dashboard, that can reduce or eliminate the burden of low-value ad-hoc right?

To a degree, yes. Putting filters, slicers and other interactive elements expands the explanatory power of a dashboard, but sacrifices usability, simplicity and fit for purpose. These dashboards inevitably become ‘export to excel’ data apps for power users - which isn’t necessarily bad, but is a poor substitute for query and data exploration tools.

Power users who push self-service dashboards to the breaking point uncover the most technically challenging low-value ad-hoc questions.

For these reasons Stephen Few, the grandfather of great dashboard design, argues persuasively against the concept of the self-service dashboard in his classic Information Dashboard Design. If you want a master class in data-presentation fit for dashboards, read it!

If not dashboards, then what?

At this point, you’re surely thinking, ‘Ryan really hates dashboards.’ Not at all! A dashboard with great data-presentation fit is perhaps the most powerful tool in the data team’s arsenal. It scales immensely and allows thousands of people to make consistently good decisions in a way nothing else can. It’s just not always the right fit.

Luckily we have options in 2022 that didn’t exist when I started in the mid-2000s which can step in when a dashboard lacks data-presentation fit. There’s a new focus on narrative presentation with the explosion of BI notebook tools, which helps with deep dive and data persuasion. Data-driven slide decks, infographics, and novel arrangements like the metric tree below are increasingly common and have powerful use cases as well. Simple ad-hoc requests require simple visual and text presentations. There are more options than ever for how to tell the most effective story using data and it's time we start using them.

Some tools can do some of these things - but we built Count as the one tool that can do them all. So no matter what data-presentation fit looks like for your audience, you are sure to find it.

A quick guide for finding data-presentation fit

This topic deserves a full exploration, but for now, I’ll leave you with this quick reference for which types of questions match best with a given presentation.