Data as a Product — The way of the mesh

What is data as a product?

Data engineers and data analysts are those who develop systems and pipelines for ingesting, managing, and converting unstructured data to usable information and insights. A data product manager is an individual who determines the customer’s need and the product’s or feature’s larger business objectives. A product manager defines success for a product and rallies a team to achieve it. So What does data as a product mean, and What does a data product manager do to be successful in delivery?
While there’s no clear definition of what a data product is. We can derive some insight by looking at some of the primary responsibilities of modern-day data teams within an organization:

• bring data from given source to destination for internal customers
• bring analytics and insights to the internal customers
• build self-serve data service to internal customers
• build self-serve machine learning and analytical services to internal customers
• deliver value to the business with all the above

With the current trend of data evolution in the industry, the data platform modernization transformation data mesh is now demanding more in data platforming and self-service capabilities. This fundamentally changes how companies use data and also the nature of data as a product.

Data as an asset

The following two activities are core functions of a data team that delivers data as an asset:

• bring data from given source to destination for internal customers
• bring analytics and insights to the internal customers

Data as an asset is the most fundamental working model for most companies, where data engineers and analysts work independently to support their part of the data system while belonging to separate teams. With roadmaps separated specific to each team’s business function: data ingestion, data modeling, or data science use cases. In this working model there is a high degree of separation between teams, often making communication and managing the roadmap stressful.

Data as a service

• build self-serve data service to internal customers
• build self-serve machine learning and analytical services to internal customers

In response to the drawbacks of data as an asset a new model has evolved, data as a service, which is taking advantage of modern data capabilities and designing data platforms in a scalable and reusable way. For example, the data platform team has built a service, enabling the capability for data analysts to ingest data with their data platform self-serve. In this case, data platform engineers have built common reusable capabilities for internal stakeholders to use time over time instead of having to go through tickets and roadmaps.

Another example would be an end-to-end AutoML time-series forecasting service provided by the data platform’s capability. In this case, data platform engineers and data scientists would work closely to build an ML capability that enables non-technical internal customers to use ML services and that deliver meaningful insights.

While the data as a service model greatly reduces cognitive load and improves time to market, it doesn’t always help a company to maximize the impact of data platform modernization. Data as a product addresses this problem.

Data as a product

• deliver value to the business with all the above

Data as a product is about bringing value to the business, on top of which, using modern-day self-serve capabilities with a focus to continuously deliver business value in the most scalable, cost-efficient way.

An example of the data as a product model: A data team is tasked with supporting a business initiative to provide demand forecasting in the global market. The product team identifies the business pillars and optimally plans the portions of DaaA and DaaS needed for each business case. The Data platform provides the following self-serve capabilities:

• ingestion
• transformation / warehousing
• train and serve machine learning models

The DaaA component of this use case would involve components that can not or do not make sense to be performed in a self-service capability. Pillar teams and analysts would then use this stage of data, continue to answer business questions, provide business insights, and build visualizations to support the business case.

With this model, a small group of cross-functional taskforces (data platform, engineering, business analysts), are able to provide quality analytics that delivers business value with much higher efficiency and a shorter time to market.
In summary, data as a product is about having a good understanding of business cases, technologies, and being able to deliver business values in the most timely and cost-efficient way possible.

Editor Credit: Teri Fowler