In a data catalog, you can discover, describe, and organize your company’s data sources to develop and manage an inventory of data.
A metrics catalog, on the other hand, is a curated collection of specific business metrics, which some organizations call key performance indicators (KPIs). Metrics that live in a metrics catalog have been defined in code and curated by a data team or data leader. Sometimes these metrics are created from tables in more than one data source.
The increasing number and complexity of data sources can create barriers to accessing information and analyzing existing data. Both data catalogs and metric catalogs can help bridge these barriers. Although they solve similar problems, they serve very different purposes and organizations should consider both tools in their data stack.
In this guide, you’ll learn how data catalogs and metrics catalogs are similar, how they differ, and why they work together to increase discoverability and clarity in an organization.
Data discovery and management
Data discovery is a key goal for both metrics catalogs and data catalogs.
Think of a data catalog as a library—it represents a collection of data asset information, helping organizations organize and maintain all of their data sources that stakeholders use to make informed business decisions. Data catalogs also aid in the collection, organization, access, and enrichment of metadata for data discovery and management. For example, a data catalog contains:
- Data access control: With data access control you can develop rules and implement access controls at scale with an automated data access control system that contains or can interact with a data catalog. This accelerates data access and analytics in a safe, compliant environment.
- Discovery: Self-service data catalogs make data more accessible by letting users search and discover it independently. Through the use of tags and filters, users can search for keywords and other criteria within the data.
- Metadata curation: Additional metadata, data categories, and other pertinent information can easily be added to a data catalog. You can keep track of things like reports, APIs, and servers in your landscape with metadata management.
A metrics catalog is a centralized location for people to explore and understand context around metrics. Instead of citing all of an organization’s raw data assets, it is a collection of metrics— quantitative indicators of a business’ performance. Examples include “north star” metrics like annual recurring revenue (ARR), monthly active users, or user retention percentage.
In a metrics catalog, both data teams and business users can see a variety of information about each metric, including a data visualization of values and a record of important context around each metric. In Transform, discovery features include:
- Metric collections: Collections allow you to group related or important metrics in one place for easier access and analysis. You can have a personal collection of metrics or group metrics by a specific initiative or time period.
- Team pages: Team pages allow you to organize all of the information that is more important for your team in one place.
- Searchability: Easily search by metric name or by tier of importance. For example, you may want to search for “Tier 1” metrics that have the greatest impact on the business (likely the metrics that report up to high levels of the organization).
Data lineage and metadata
Data catalogs aid in the collection, organization, access, and enrichment of metadata for data discovery and management. It identifies data owners, custodians, and subject matter experts, making it easier for users to collaborate across departments.
A data catalog can also offer business lineage, which helps users understand the movement of their data along the data supply chain. As a result, it helps identify essential business processes, provide data quality ratings, inform users of data access methods, and convey data use constraints.
A metrics catalog shows the lineage of each metric, including how a metric was defined in code and the data sources from which it was derived, including the source table or query. The lineage of a metric is particularly important because it’s common for organizations to report different numbers for the same metric. This is because people construct metrics using differing metric logic. When metric lineage is transparent, everyone understands why a metric is constructed, increasing trust.
Ownership and data governance
Data catalogs show what assets you have and where they’re located. Data engineers own the data assets and the metadata in the data catalog.
Data catalogs are a common tool for data governance. When it comes to data management, the data catalog shows what data assets a company has and where they are, while data governance specifies who owns, manages, and uses that data. You can use a data catalog to get your hands on data that’s in an easy-to-read format.
The structure provided by data governance facilitates the coordination of numerous technical and business aspects of an organization’s data assets across a wide range of data consumers in many departments. For tracking purposes, the data catalog should contain both technical and commercial ancestry information.
In a metric catalog like Transform, you can assign a metric owner to help with governance and metric management. Metric owners give data consumers (people who use the data for decisions) context into who they should approach with questions and who is the ultimate approver for a metric definition or any changes to a metric. Transform allows for either team or individual owners.
Who uses metrics catalogs and data catalogs?
Data scientists, engineers, and analysts often utilize data catalogs in their work. These experts sift through large amounts of data and are able to draw meaningful conclusions while ensuring that everything is added to the inventory and labeled correctly.
In day-to-day analysis and in more complex analysis, data analysts and data scientists rely on the important functionalities supplied by data catalogs. A lack of support for all data consumers in the company means that these solutions are insufficient for meeting all of the organization's governance requirements. Even though a cataloging tool may provide enormous information, a normal business user is unlikely to utilize it regularly. Consequently, many businesses find it difficult to justify the significant continuing expenditure necessary to keep governance data current in these toolsets.
Metrics catalogs are different from data catalogs in the sense that a metric catalog is made for both data teams and business teams. While data teams will be heavily involved in constructing and defining metrics, once they publish them to a metric catalog, business users can comment, ask questions, and track the performance of their metrics in one central place.
While some business users will choose to look at their metrics in business intelligence (BI) tools, there are times when stakeholders want to see a comprehensive overview of all of their metrics in one place. They can also choose to bring in metrics from a metrics catalog into their BI tool of choice for further analysis.
In this article, you learned what data and metrics catalogs are and how they can be used to inform business decisions.
Transform provides a metrics catalog that fully regulates business metrics, with clear ownership, approval cycle, annotations, and metrics.
This post is guest authored by Dairenkon Majime Rezende de Souza, data scientist, writer, and learning facilitator.