A must read by Zhamak Dehghani if you are a data driven company (or are on your way to be). I have read her book “Data Mesh” that goes beyond the article.
For me one of the (main) reasons why we struggle today is:
“When we zoom close enough to observe the life of the people who build and operate a data platform, what we find is a group of hyper-specialized data engineers siloed from the operational units of the organization…
…The data platform engineers are not only siloed organizationally but also separated and grouped into a team based on their technical expertise of big data tooling, often absent of business and domain knowledge.”
…
“For a distributed data platform to be successful, domain data teams must apply product thinking with similar rigor to the datasets that they provide; considering their data assets as their products and the rest of the organization’s data scientists, ML and data engineers as their customers.”
…
“The main shift is to treat domain data product as a first class concern, and data lake tooling and pipeline as a second class concern – an implementation detail. This inverts the current mental model from a centralized data lake to an ecosystem of data products that play nicely together, a data mesh.
The same principle applies to the data warehouse for business reporting and visualization. It’s simply a node on the mesh, and possibly on the consumer oriented edge of the mesh.”
Leave a comment