Understanding AWS Config Cost

AWS Config gives you a way to assess, evaluate and audit configuration resources with SQL-like interface. This is super useful, especially when the number of accounts is starting to grow and observability becomes key for governance, security, operations and cost. But as always, the devil is in the details. Or in this case, the cost of AWS Config service. In this blog we will go into how much it costs, how to reduce costs and what other 30x cheaper alternative you can explore (Ahem, CloudQuery).

You can think of AWS Config as a time-series database that ingests resources from AWS APIs (we are talking about configuration metadata), transforms them, and saves them into a table. In the AWS Config UI you can then query this table with SQL-like interface as AWS doesn’t expose the raw database.

AWS Config Pricing consists of multiple dimensions and depends on multiple configurations:

So let’s break it down and see how and why it can add up REALLY quickly.

Let’s say we are not using any rules, and we just use it strictly for observability, so we are only charged for items recorded. Now we have two options: periodic scans or continuous mode.

If we go with periodic, we are charged $0.012/resource (!) or $12 per 1,000 resources (!), and this is just for one scan. Let’s say you run it daily: you are charged $360/month/1000 resources. To put it in perspective, resources can be anything from EC2, S3 bucket to VPC, IP to configuration items. With one million resources (which is not far-fetched for any medium-size companies), you are looking at a $360,000 monthly bill, and this is just to see what resources you have. 💸

So you are saying, wait? You are using AWS Config all wrong - use the continuous mode, it is much cheaper, and it’s all real-time, right? Wrong :) Let’s dive in:

So let’s look at the previous example where we have one million resources. Before starting the continuous mode, AWS Config needs to record the initial snapshot. This will be an initial 0.003*1,000,000=$3,000. It sounds much better than the previous quote. But there is a catch! Now, ANY change done to a resource will cost us $0.003. Using the previous method, we were bound by the number of resources and number of times a month we run the job. Now, AWS Config is constantly listening to events, from anything like shutting down an EC2 machine to updating a configuration. This can quickly balloon to a million or more changes a day, so let’s do one million changes per day, and we will be looking at $3,000/day or $90,000 a month. Now I don’t want to think about a scenario where a developer had a bug in one of their terraform or python scripts that infinitely updates some random VPC settings in one of the accounts :) Or like in this case you have a service that is stuck in a restart loop.

Now that we racked up anything between $100,000 to $360,000 per month let’s continue pumping up the AWS Stock and buying bigger (or new) Yachts for shareholders!

To save some time calculating, this will largely be the same as config rules.

So how do we get visibility into what we have in all our AWS accounts in all regions, running SQL queries and not racking up $500k/month bills? There are different ways to exclude some resources or only monitor low count resources but overall, the stakes will be high as any mistake is costly. And there is really no good way of recording all the data without racking up that bill in AWS Config (at least at the time of writing). But there is another solution: taking a classic data ELT (Extract-Load-Transform) approach with CloudQuery.

With the CloudQuery AWS Plugin you can extract metadata and configuration from more than 1,000 AWS APIs to any supported destinations such as PostgreSQL, Snowflake, BigQuery, DuckDB, MotherDuck and others. Calling AWS Read APIs doesn’t incur additional cost (at least for now) and writing the database to a data-warehouse gives you full SQL control, ability to connect it to any BI system you have, and in general gives you full flexibility. CloudQuery is a pretty good tool and like anything good in life it doesn’t come for free. So let’s check what costs it will incur:

CloudQuery plugins can run locally on your own Kubernetes or EC2, so it keeps all your data inside your AWS account and doesn’t incur 3rd party cloud or network costs.

CQ plugins are priced by usage and cost $10/million rows. Going back to the previous example, let’s say we have a million resources and we want to sync them daily. This will result a bill of $10 * 30 = $300/month.

Now let’s add a strong EC2 machine of type c6g.4xlarge with 16 vCPU and 32GB RAM. That will cost approximately $400/month (and this assumes constant usage which is not needed as you can run it for an hour or so just for the time of the sync).

Now let’s Add a PostgreSQL RDS db.m4.xlarge with 4vCPU and 16GB RAM with Single AZ that will cost around $330/month. You can also go with something like BigQuery, Snowflake, Aurora, or Neon depending on your query workload and potentially get the price even lower, as you won’t need the compute up and running and coupled with storage. Now let’s add all that up, and we get $1300/month. This is a 30x price reduction, including all infrastructure costs, and you have big room to make the database or ingestion compute stronger if needed.

In this blog we showed how much AWS Config can cost and how to get around a 30x price reduction, for more flexible and performant capabilities than AWS Config. Depending on your spend and specific scenario, if similar to this case you will be able to save about $43k/month, or $516k/year. This will allow you to buy (or maybe just to rent) a new Yacht like in this picture:

Or maybe just reinvest it back in your business like in this picture:

Or anything else you think might be useful. Did you like the blog? Give CloudQuery a go here