As an emerging concept, data fabric is not yet part of the mainstream. “It is important to understand that data fabric is not a platform, tool, or technology,” said Ehtisham Zaidi, senior director analyst at Gartner. “It is a data design concept, which needs a collection of composable capabilities that is provided by several data management tools and best practices to deliver and combine data, often through automation techniques, via active metadata, AI, and machine learning.” When data sources are heterogeneous and inte gration is complex, a design pattern is needed that allows integration and delivery that is both automated and agile, which data fabric offers.
“The promise of data fabric is dependent on an organization’s ability to incorporate knowledge graphs into their data architecture,” he continued. “That in turn depends on ontologies that reflect a good understanding of an organization’s information, processes, and the connections among entities.” The first step is to collect and integrate as much metadata as possible, and it should not be restricted to technical metadata; it should include business and social metadata as well as performance metadata elements such as frequency of access.
A limiting factor in the growth of data fabric is the absence of metadata stan- dards, according to Zaidi. One of the few such standards is Egeria, an open meta-data exchange standard that provides linking and exchange of data among heterogeneous repositories and tools so that users of different tools can collaborate and share data. Another factor is expertise. “It is difficult to gather enough experienced data engineers with the right skills to translate requirements of uses case into architecture,” Zaidi added. However, with the ongoing increase in volume and diversity of data and only 2%–5% of the companies that could benefit from the data fabric having utilized it so far, solutions to these issues are likely to evolve.