Fixing AWS Config cost surprises

Enabling recording of all resources with AWS Config is one of the requirements for any organization that takes security seriously. AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. It provides comprehensive visibility into the configurations of your AWS resources, continuously recording and evaluating changes. It helps maintain compliance by defining rules and automatically remediating non-compliant resources, ensuring adherence to internal policies and industry standards. 

However, it can also be challenging from the cost perspective. If you ever had a Config recorder when your Lambda or a Fargate container was in a reboot loop, you know what I mean.

Our customer is using ephemeral servers for one of their workloads, recycling their EC2 nodes once the job is done. However, the situation came to a point where on the monthly statement the bill for Config would be more than the bill for the compute time usage in EC2. So we’ve set on a journey to optimize the Config bill but also stay compliant and keep all the services still working. First step is to look into what are the components / resources that comprise the bill. 

For many of you utilizing a more or less customized landing zone a mysterious resource called  AWS::Config::ResourceCompliance will occupy the first spot in the top10 list. What this resource does is that it collects compliance status of every single resource and its relationships. A new Configuration Item (CI) is recorded every time a new assessment is made which is either periodically or upon every change.

Many organisations during the deployment of their Landing Zone environments simply enable Config, set the recording and really forget about it. There are a few standard Config rules that are deployed by Config itself and a few more added if you use AWS Security Hub. These get all installed in  all the member accounts to keep watch and report to the central delegated account. Our customer recently introduced a different Cloud Security Posture Management tool (CSPM) and frankly wasn’t utilizing the capability provided by AWS.

Since last year it is finally possible to exclude single resources from AWS Config and that is what we went for in this case. In this case by disabling AWS::Config::ResourceCompliance the customer managed to save circa 15% of their AWS Config bill which in their case was quite a significant number.

If you dig into AWS Config a little bit you will also notice that each update to a resource triggers multiple updates to all resources that this resource is in a relationship with.  So for example updating a AWS::EC2::NetworkInterface will trigger 6 more resource updates which in turn will trigger their relationships as well. In fact if you are running a workload with high EC2 turnover rate (eg. for AWS Batch jobs) you might look into disabling this one as well if you feel you are not getting your money’s worth.

By excluding the resource types you're not interested in relationship tracking for, you can effectively cut a lot of spend on AWS Config.

Another thing you might wish to investigate is changing how often the tracking happens. By default, AWS tracks all the resource configuration changes in real time, when they happen. However, with the cost of it being as it is this might not be aligned with your organizations goals and how do you want to use Config. Not so long-ago AWS introduced a different way of scanning. It is now possible to record changes periodically and adjust the schedule for different resource types. By scanning only once a day it’s possible to avoid unnecessary cost if your situation allows for it. If you take the example above by reducing to 1 or 2 scans per day we can keep the NetworkInterface resource any only scan it once.

Few words of caution though. Some of the capabilities inside of AWS depend on AWS Config and its ConfigurationItems. Services like SecurityHub or TrustedAdvisor need the data for their checks.  AWS also said that some of its new AI capabilities will also use AWS Config to provide insights and answers to your queries.