Data trends to watch in 2023

After a humbling and exciting 2022, I am back to make my best predictions about what we can expect in 2023. Here goes nothing!

1. Data teams get focused

Look, things are going to be undeniably different in 2023. If there's anything we've learned in the last few months it's that we in data are not insulated from larger economic forces.

At a practical level, many of us are heading into 2023 with reduced budgets, smaller teams, and even greater demand from our business partners. However, Joe Reis looked beyond this to examine an even greater threat to our data teams: how the perception and reputation of the data team itself might come under threat:

"The value created by data teams is under intense scrutiny. Delivering concrete results that drive the needle is mandatory. Data teams that can’t deliver will be dissolved." - Joe Reis, Money for Somethin'

And look, this is a grim prediction to start with if you were to stop reading here, but as Joe also wisely says, 'scarcity sharpens the mind' and this is what data teams will need to do in 2023.

With fewer tools and probably fewer people, we will have to redefine what we can do to add value, an exercise that when you get beyond the driving circumstances can be extremely powerful and liberating. Taking a focused look at the most effective ways we can help teams is only going to be a win/win.

2. A.I. finally finesses its way into the data stack

People have been predicting the rise of A.I. since Hal debuted its red eye on the big screen, yet outside of a few uses its role in our data stack is still non-existent. But something feels different now. With Chat GPT exploding onto the scene last month, this evolution of A.I. has declared itself distinctly different, and crucially more useful.

Exactly how A.I. finds its way into our data stack is less clear to me. The idea of NLP-enabled charting has been around for years without gaining much traction, but I suspect it will be something less obvious that opens the door to A.I. in 2023.

3. Documentation is the least sexy crisis of the year

Ah, the metric layer. Whether you're anxious to try out DBT's new semantic layer, a long-time user of Looker, or an old-school BI person who can't help but wonder how any of this is different from the data cubes of old, you can bet the topic of the metric layer is here to stay in 2023.

But to me, the biggest issue with metric layers, and enabling more self-service, is not about technology. When we speak to business users about accessing data, their biggest challenge is understanding the data model, and being confident about what they are actually querying - not writing the code itself.

Traditional documentation just isn't going to cut it. It's too hard to understand, and it's not rooted in a business user's understanding of the data. Rethinking documentation may not be the most exciting topic, but it is absolutely essential if we are to truly enable more self-service in 2023.

4. The 'Data is a product' counter-culture has its moment

The last few years have been dominated by the question: 'What can data learn from software development?' It has led to better version control, more robust pipelines, software that looks more and more VS Code, and sadly, at times, a more insular way of thinking.

The smallest demonstration of how this can go wrong was from one talk I attended at DBT's Coalesce. The speaker advised the attendees to make the data request process 'as difficult as possible' in order to discourage requests to the data team. And sure, that is out of context, but it's representative of a larger trend, one in which we are becoming less and less concerned with what happens to data when it leaves our pipelines, in favor of more and more upstream control.

The counterforce, and a much-needed balance to 'software engineering for data' can be found in the 'data is a product' movement. If the former has gotten us robust pipelines then the latter asks us to make that information actually useful. It's empathetic, and user-focused, and helps us navigate the many relationships required to make data useful.  

We will need these principles to guide us through a more critical business environment and to finally make our very robust and performant pipelines useful to our business partners.

5. Data quality lights up your Twitter feed

There are some interesting things going on in the data quality/observability space these days (including some things with A.I. 👀), and I anticipate this becoming a more popular topic in 2023.

To be clear, I expect this to be a big topic not necessarily because it is a big problem (which it may be to some), but for other reasons:

1. it is an obvious extension of the data engineering craze: you've got those pipelines, but tests are failing! How are we going to fix it?
2. the solutions will play into software engineering principles: let's see how other technical fields find and fix imperfections at scale
3. it bolts on nicely with the rest of the stack: all of these tools will integrate with DBT and/or your data warehouse, letting you add the next Lego piece to your ever-growing stack

Basically, I think for many it will be a solution space they would rather spend their time thinking about rather than more knotty problems like how to better demonstrate their team's value to the business.  

6. You get a newsletter! You get a newsletter! You get a newsletter!

One of my favorite trends from 2022 has been the diversity of voices we're hearing from. Just a few years ago, it felt like there were just a handful of blogs talking about data, and most information was still coming from SEO-optimized articles written by private companies.

Now, a tweet from anyone can spark a conversation, debates are happening in comment threads of LinkedIn posts, and more and more people are taking to the blogosphere to talk all about data.

I love it, and I hope this is only just the beginning. But how do you cut through the noise and find the newsletters and blogs for you?

Filtering and curating content has always been an issue when there are lots of creators. In 2023 I expect Twitter and LinkedIn will continue to serve as our data content curators (for better or worse). Long-form tweets and posts serve as short blurbs, summarizing content and testing interest before readers opt into the full story.  

In conclusion

In reality, no one can predict exactly what's in store for us in 2023. I am confident, however, that there are so many opportunities for each of us to make positive changes. Whether you want to get your voice out there with a new blog, or you're leading a data team that's facing a new, harsh reality, we are better poised than ever to be successful. It's just up to us to take it.