KM leverages data mesh
Knowledge management has often been stymied by siloed data and unsuccessful efforts to create comprehensive, centralized repositories. Data mesh is a method of managing enterprise information and processes that helps to overcome these limitations and facilitates the reuse of valuable enterprise information. It is characterized by four features:
♦ Data distributed into domains that reflect business units or teams rather than being centralized
♦ Data products created from datasets with the intent of making them easily accessible and consumable
♦ Federated computational governance in which governance is managed by the data owners and is tailored to each domain, but also includes standardization to ensure interoperability across domains, with automated execution of governance decisions
♦ Self-service both by developers and consumers of data products
Since data mesh is not a technology but an approach to managing information, it relies on a combination of enabling tools, including those that support integration, governance, security, and analytics. It is often implemented incrementally by domain and by feature. Most environments that use data mesh are hybrid environments, with some data centralized and some distributed.
Data products were introduced as a core component of data mesh architecture and represent a new paradigm in how data is served to the consumer. A data product is defined by a specified owner and is designed to enable consumers to easily find and securely use it. Data produces can represent integrated assets from a single or multiple data stores, and the actual design can vary to optimize the intended use case. “Consumers looking for data can quickly search a catalog to find and reuse a data product that meets their needs,” said Adrian Estala, VP, field CDO, at Starburst. “The required data assets have already been integrated and defined.”
Starburst is based on open source Trino (trino.io) and accelerates the development of data products by exploiting a lake-house architecture and federating across warehouses and other distributed data sources. “We accelerate the time to insight by promoting self-service capabilities for data products,” Estala added. Starburst offers a cloud-native, Galaxy solution and an Enterprise solution for on-premise architectures. “We provide an abstraction layer in between the diverse, complex data architecture and the BI and analytics tools that our customers use.”
Pharma is among the industries that first adopted data mesh architecture. “These organizations were already organized into clinical analytics teams that fit the domain team concepts that are described in a mesh approach,” Estala pointed out. “In a domain, the team has full familiarity with their data, the use cases, and the broader data risks.” Banks were another vertical that quickly picked up the data mesh ideas. “Every major bank we work with has incorporated some data mesh concepts into their broader data strategy,” he continued. “The focus on self-service, federated governance, and data product capabilities fits very well into their strategic analytics objectives.”
However, businesses across many verticals and of varying sizes can benefit.
Self-Service at Schneider Electric
Schneider Electric had its origins 180 years ago, during the first Industrial Revolution, and by the turn of the century, the company was entering the emerging electricity market. Now it produces a wide range of electrical products for residential, business, and industrial use. With an ongoing focus on innovation, Schneider Electric wanted to use new digital technologies to help make its numerous business units more agile and competitive.