Transforming Data Management: How Comcast Implements Data Mesh in Hybrid Data Lakes 🌐

At Comcast, we have long had a complex hybrid data lakes that consists of data lakes in on-prem and multiple cloud environments. Comcast uses Trino to bridge the data in these environments using an architecture we call Data Fabric. Data Fabric is an abstraction layer that uses an internally built connector that connects to multiple instances of Trino. This enables us to query across all of these environments from a single Trino instance.

In recent years, emerging architectures like Data Mesh have nicely complemented the goals we have been building to for years. While we have effectively implemented some aspects of a Data Mesh, there are still core tenants that cannot be addressed by Trino alone. This is the journey we are on at Comcast, and we like to share our experience so far, challenges we overcame, and the ones yet to be resolved. Data abstraction, availability, movement, and governance are the various topics we will touch upon in this session.

Sajumon Joseph, Sr Principal Architect

Pavan Madhineni, Sr. Manager; Product Development Engineering