Infrastructure Data Platforms made simple

As a Platform Engineer, DevOps, or SRE you may have heard the term Infrastructure Data Platform, but, as with most industry jargon, the name is a little opaque.

So, what is an Infrastructure Data Platform, and why should you care?

A data platform is an integrated solution that helps to centralize your data collection, storage, processing, analytics, governance, and management. Improving the access, usability, and security of the stored data. A Customer Data Platform, for example, is like a CRM on steroids, combining data from the CRM with things like web analytics, social media posts, anonymous data, and public data.

Because Data platforms grant more straightforward access to data, they can improve interdepartmental collaboration, understanding, and scalability.

An Infrastructure Data Platform, then, brings all your infrastructure data together in one place. Becoming the home of your CSPM (Cloud Security Posture Management), CEIM (Cloud Infrastructure Entitlement Management), ASM (Attack Surface Management), Cloud Asset Inventory, and much more.

This enables platform teams to build dashboards, analytics, and queries that reduce Time-to-Response and Time-to-Fix while also reducing the work needed to diagnose an issue or build an impact assessment.

An Infrastructure Data Platform is essentially an application of the modern data stack of Extract-Load-Transform to infrastructure data.

This isn’t actually that much of a new thing; vendors have been building solutions that fall under this category for a while, but with dashboards and UIs that limit you to the particular use case that they had in mind. Most CSPM platforms, for example, are just niche implementations of Infrastructure Data Platforms that focus on security-related issues.

While it’s not true that all such platforms fall into this category, as some are merely front-ends to the underlying platform APIs, the ones that can scale to meet enterprise requirements do.

By setting up an Infrastructure Data Platform, you’re unbundling these different vendor platforms, so instead of having a platform for CSPM, a platform for CIEM, a platform for ASM, etc. you have a singular platform where you have an ELT solution (like CloudQuery) that feeds a data storage solution (like Postgres), that can be accessed through a graphing and analytics platform (like Grafana).

Simply put, a single vendor cannot provide a universal solution that works for everyone in all cases.

It’s all about data access; by unbundling these niched platforms into an Infrastructure Data Platform, you can analyze the impact of an issue or planned work more quickly, mitigating and preventing problems before they arise.

Unbundling access to your data also enables you to conduct detailed post-mortems in a single platform, analyzing the chain of events back to the root cause.