• Which S3 buckets are publicly accessible right now?
• Which IAM roles haven't been used in 90 days?
• What changed in production yesterday between 2 p.m. and 3 p.m.?

Why Traditional CMDB Approaches Fail in Cloud Environments #

Best Practice 1: Automate Discovery from Cloud APIs #

Best Practice 2: Start with High-Value Resources, Expand Deliberately #

• API rate limits (AWS will politely throttle you, then less politely)
• Database bloat (10 million low-value records nobody queries)
• Query performance death spiral (finding signal in noise becomes impossible)
• Sync jobs taking hours instead of minutes

Best Practice 3: Use Tiered Sync Frequencies #

*/30 * * * * sync-tool sync critical-resources-config  # Every 30 min
0 * * * * sync-tool sync operational-resources-config  # Hourly
0 2 * * * sync-tool sync baseline-resources-config     # Daily at 2am

Choosing Your Cloud CMDB Approach #

Best Practice 4: Design for Multi-Account and Multi-Cloud from Day One #

• account_id (AWS account number, GCP project ID, Azure subscription ID)
• environment (prod, staging, dev, test)
• owner_team (for when you need to know who to blame—sorry, "collaborate with")
• cost_center (financial attribution)

-- Cloud CMDB query: find public S3 buckets across all AWS accounts
SELECT
  account_id,
  arn,
  region,
  tags->>'Owner' as owner,
  tags->>'Environment' as environment
FROM aws_s3_buckets
WHERE
  block_public_acls = false
  OR block_public_policy = false
ORDER BY account_id, region;

# AWS multi-account config
kind: source
spec:
  name: aws
  path: cloudquery/aws
  destinations: ['postgresql']
  spec:
    # Assume role in all organization accounts
    org:
      member_role_name: 'CloudQueryRole'

Best Practice 5: Integrate with Security, FinOps, and Incident Management Tools #

Security: Datadog, Elastic, Splunk #

• Public S3 bucket created (someone's getting creative with permissions)
• IAM policy grants *:* permissions (bold move)
• TLS certificate expiring in <30 days
• Security group allows 0.0.0.0/0 on port 22 (the "I'll fix this later" that never gets fixed)

FinOps: AWS Cost Explorer, GCP Billing, Azure Cost Management #

• Attribute costs to teams via resource tags
• Find idle RDS databases (zero connections for 7 days = expensive paperweight)
• Identify rightsizing opportunities (EC2 with <5% CPU = money pit)

-- FinOps: unused RDS instances (join cloud CMDB + CloudWatch metrics)
WITH connection_metrics AS (
  SELECT
    db_instance_identifier,
    MAX(database_connections) as max_connections
  FROM cloudwatch_rds_metrics
  WHERE timestamp > NOW() - INTERVAL '7 days'
  GROUP BY db_instance_identifier
)
SELECT
  r.account_id,
  r.arn,
  r.db_instance_class,
  r.estimated_monthly_cost,
  m.max_connections
FROM aws_rds_instances r
LEFT JOIN connection_metrics m
  ON r.db_instance_identifier = m.db_instance_identifier
WHERE m.max_connections = 0 OR m.max_connections IS NULL
ORDER BY r.estimated_monthly_cost DESC;

Developer Portals: Backstage, Port #

Incident Management: PagerDuty, Opsgenie #

BI & Dashboards: Grafana, Looker, Metabase #

Best Practice 6: Retain Historical Data for Compliance and Debugging #

• SOC 2: 1-2 years | ISO 27001: 2-3 years | HIPAA: 6 years | Financial services: 7 years

-- Cloud CMDB: S3 bucket configuration as of October 15, 2024 at 3pm
-- Get most recent sync before target time for each bucket
WITH ranked_syncs AS (
  SELECT
    account_id,
    arn,
    block_public_acls,
    encryption_rules,
    sync_time,
    ROW_NUMBER() OVER (
      PARTITION BY arn
      ORDER BY sync_time DESC
    ) as rn
  FROM aws_s3_buckets
  WHERE sync_time <= '2024-10-15 15:00:00'
)
SELECT
  account_id,
  arn,
  block_public_acls,
  encryption_rules
FROM ranked_syncs
WHERE rn = 1;

Implementation #

• PostgreSQL: Best for simplicity; run on AWS RDS, GCP CloudSQL, or self-hosted
• BigQuery: Best for massive scale (billions of records); Google Cloud native
• Snowflake: Best if already in your stack; excellent query performance

• AWS (via AWS Organizations role assumption for multi-account)
• GCP (via Resource Manager API for all projects)
• Azure (via Azure Resource Manager for all subscriptions)
• Kubernetes (via kubeconfig)

• Critical resources (IAM, security groups): every 30 min
• Operational resources (EC2, RDS, S3): hourly
• Baseline resources (dev environments): daily

-- Public S3 buckets across all accounts
SELECT account_id, arn FROM aws_s3_buckets
WHERE block_public_acls = false;

-- Untagged EC2 instances
SELECT instance_id, tags FROM aws_ec2_instances
WHERE tags->>'Owner' IS NULL;

Summary #

1. Automate discovery from cloud APIs (manual updates fail within weeks)
2. Define clear scope (start security/compute; expand deliberately)
3. Tiered sync strategies (critical 15-30 min, operational hourly, baseline daily)
4. Multi-account design from day one (centralized database; query across accounts)
5. Tool integration (SQL enables integration with security, FinOps, incident management)
6. Historical data retention (2-7 years for compliance and incident investigation)

Related Resources #

• What is a Cloud CMDB and Do You Need One? — Comprehensive introduction to cloud CMDBs, comparing traditional vs modern approaches
• The Real-Time Cloud CMDB: Why Ephemeral Infrastructure Killed Traditional Discovery — Technical analysis of sync frequencies and ephemeral resource tracking
• CMDB is Dead: Long Live the Infrastructure Lake — Architecture patterns for building modern cloud CMDBs with Infrastructure Lake approach

• CloudQuery Quickstart Guide — Get started syncing your first cloud resources in minutes
• AWS Multi-Account Setup — Configure AWS Organizations role assumption for multi-account discovery
• CloudQuery Hub — Browse all available cloud provider integrations and resource types

Frequently Asked Questions #

What is a cloud CMDB and why do platform engineers need it? #

How often should a cloud CMDB sync data? #

What's the difference between a cloud CMDB and a cloud asset inventory? #

Can CloudQuery be used as a cloud CMDB? #

How do I handle multi-account AWS environments in my cloud CMDB? #

What are the main cloud CMDB data quality challenges? #

How much does it cost to run a cloud CMDB for large enterprise environments? #