We recently put out a set of survey questions to our community to understand how organizations are tracking metrics today in the context of the modern data stack. We also wanted to understand some common obstacles that lead to data inconsistencies and secondary sources of truth.
This was our first time ‘surveying’ our community so we consider this more of an exploration than a formal research report, but we believe that these insights showcase some of the common struggles and trends we’re seeing in the data space.
The majority of respondents were data professionals—self-described data engineers, data scientists, data analysts, and analytics engineers. Respondents came from organizations of various sizes in a variety of industries.*
Here are some of our most interesting findings:
- Data teams see the need for a centralized metrics store, but they're running into roadblocks. While 87% of respondents saw the need for a centralized metrics store, only 38.5% currently had one in their organizations.
- The top problems that respondents encounter when reporting on metrics revolve around productivity. These include eliminating ad-hoc/repetitive tasks (59%), delivering faster time to insight (around 58%), and democratizing access to metrics (around 55%).
- Responsibility for metric ownership was distributed among data teams (around 73% say this is the responsibility of a centralized data team) and embedded analysts within business units (40% say that analysts live in respective business functions). 26% said that this responsibility also fell on business function leads.**
The metrics store is a vital piece of the modern data stack
Data teams manage many tools in their data stack. We were curious...which tools were most popular among our community? Most respondents marked Airflow (46%) for workflow management and Snowflake (37%) for data warehousing. Other solutions that came up were Google BigQuery, dbt, Looker, Amazon Redshift. Common business intelligence solutions were Looker, Mode, and Tableau (marked as freeform answers in the “other” category).
"These responses show that data teams are investing in the modern data stack. These are the same professionals that see the need for a centralized metrics store.” — Nick Handel, Co-founder and CEO, Transform
Popular metrics include user behavior, product usage, and revenue
We also wanted to understand the types of metrics that organizations are currently tracking. Some of the most common types of metrics were active/engaged users (78%), product usage (76%), and number of users (72%). You can see other popular metrics in the table below. Note that respondents could mark multiple metrics.
Data teams need centralized metrics to support formal and ad-hoc reporting
Data teams are typically the hub of information in an organization. This puts them at the center of strategic conversations, but it can also lead to an overwhelming amount of requests. The results confirmed this, with most respondents supporting multiple teams, including product, marketing, and engineering, in addition to other teams.
While serving multiple areas of the business, data teams are also expected to provide consistent reporting. When asked how often they report on common metrics, most respondents (over 65%) are reporting metrics on a daily basis or a monthly basis (52%).
Additionally, over 40% of respondents said that they are also doing ad-hoc reporting. 59% of respondents noted that one of the main problems they encounter when reporting on metrics is the need to eliminate ad-hoc and repetitive tasks so that they can bring more capacity to their day job.
“Most data teams are also doing a lot of ad-hoc analysis in addition to formal reporting. This makes up a lot of their workload and time.” — Nick Handel, Co-founder and CEO, Transform
This is where a metrics store can save a lot of analyst time, while ensuring consistency in metrics definitions across an organization. While 87% of respondents saw the need for a centralized metrics store, only 39% currently had one in their organizations.
“The fact that around 39% of respondents said they currently have a metrics store shows an emerging need for an upstream source of truth. And the fact that they’re not currently any generally available metrics stores indicates that people are building internal solutions. There has been a need for metrics stores for some time and yet the market is just catching up to what people have been seeing internally.” — Nick Handel, Co-founder and CEO, Transform
Current tooling isn’t fulfilling the need for accurate, consistent metrics
Data teams are now more aware of the concept of a metrics store—a centralized, versioned repository that sits on top of your data warehouse and allows data teams to maintain standardized definitions of business metrics.
When asked about how well their organization is aligned (ownership, definition, lineage, alignment, versioning, context, etc.) on a scale of one to five (one being not aligned and five being most aligned) around critical metrics, the majority of respondents (36%) selected a three out of five.
“This is pretty consistent with what we’re seeing in the data space. With your most critical metrics, you want to have the utmost confidence that they’re accurate and that everyone is defining them the same way.” — Nick Handel, Co-founder and CEO, Transform
This uncertainty may come down to the tools that organizations commonly use for maintaining and sharing metrics information. Respondents recorded that they are using a combination of BI tools (55%), internal wiki pages and Google Docs (48%), spreadsheets (40%), and some other combination of data catalog tools (33%).
“The fact that the chart above adds up to over 200% (people selected multiple tools) shows that there is no single source of truth right now. People are using a combination of tools that weren't designed with this specific intent of supporting good metrics governance. Given this, it’s not surprising that only 10% of people say they have the highest level of confidence in their metrics.” — Nick Handel, Co-founder and CEO, Transform
Our final thoughts
This survey gave us a chance to hear how people are thinking about metrics in organizations of all sizes. It also showed the need for a centralized metrics store as part of the modern data stack. We’re excited to continue learning from you.
Anything in this survey surprise you? Let us know what you think on Twitter or LinkedIn.
*Who took this survey?
Given the nature of this survey on our social channels, a large portion of respondents came from within the Transform social media network (Twitter and LinkedIn).
The majority of survey respondents were data professionals—Around 24% were data engineers, 22% were data scientists, 15% were data analysts, and 9% were analytics engineers. 17% identified as “other” and 14% identified as product managers.
Most respondents came from the technology industry, with retail and finance industries also in the top three. We also received answers from the Education, Manufacturing, Healthcare, Consulting, and other industries.
**People could select multiple answers so there are cases where organizations have all three scenarios.
