Limiting CloudQuery access to AWS accounts

When using CloudQuery to sync data from AWS, a common practice is to use a read-only IAM role. This approach gives CloudQuery access to your resources without posing a security risk of unwanted writes. However, sometimes, you will want more granular control, limiting CloudQuery's access to only the specific resources defined in their sync configurations. Previously, obtaining the necessary permissions for this level of control was a manual, cumbersome process. That’s why we now provide an API endpoint that simplifies this process, allowing you to automate the collection of required permissions based on your chosen tables.

How to limit access to selected tables only

When the only tables you want to sync are from the EC2 and EKS services, you could use a role with permissions limited to those services, not broad read access across the entire AWS account. You can go through the integration documentation and see the permissions for each table, or you could use CloudQuery API to do that automatically.

The API endpoint that provides the required information is the GetPluginVersionTable. All you need to know is the plugin name, plugin version, and the table name. Here’s an example request to get permissions for the aws_ec2_instances table and extract the permissions using jq:

curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/v34.9.0/tables/aws_ec2_instances | jq -r '.permissions_needed.[]'

Output:

ec2:DescribeInstances

The only thing remaining is to list the tables from your AWS sync config,call the API endpoint for each table, and then extract the permissions_needed field from the JSON response.

Suppose you want to sync the following AWS tables using the CloudQuery AWS integration version v34.9.0:

• aws_ec2_instances
• aws_eks_clusters
• aws_ec2_hosts

Using the API endpoint mentioned above, you can retrieve the permissions for each table with an API call similar to the following (replace TABLE_NAME_GOES_HERE with the actual table name):

curl  -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/v34.9.0/tables/TABLE_NAME_GOES_HERE | jq -r '.permissions_needed.[]'

Calling the above API calls will return the following permissions needed for each table:

• ec2:DescribeInstances
• eks:DescribeCluster
• eks:ListClusters
• ec2:DescribeHosts

Based on the results, a minimal IAM policy can be created to grant access to the tables from the example above only with the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["ec2:DescribeInstances", "ec2:DescribeHosts"],
      "Effect": "Allow",
      "Resource": "*"
    },
    {
      "Action": ["eks:DescribeCluster", "eks:ListClusters"],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Putting it all together

Here's a full shell script that generates a JSON IAM policy based on the defined tables:

# Define the tables to check
tables="aws_ec2_instances aws_eks_clusters aws_ec2_hosts"

# Get the latest version of the aws plugin
latest_version=$(curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws | jq -r ".latest_version")


# Get the permissions needed for each table
permissions=$(echo $tables | xargs -n 1 -I {} curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/$latest_version/tables/{} | jq '.permissions_needed.[]' | sort -u)

# Print the sorted permissions as JSON IAM policy
echo "{
  \"Version\": \"2012-10-17\",
  \"Statement\": [
    {
      \"Action\": [$(echo $permissions | tr ' ' ',')],
      \"Effect\": \"Allow\",
      \"Resource\": \"*\"
    }
  ]
}"

The output of the above:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ec2:DescribeHosts",
        "ec2:DescribeInstances",
        "eks:DescribeCluster",
        "eks:ListClusters"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Wrap Up

By using CloudQuery’s new API endpoint, you can easily automate the creation of the least privileged IAM roles, eliminate manual errors in mapping tables to their required permissions, and streamline the setup of secure and granular CloudQuery syncs.

CloudQuery simplifies cloud asset discovery and governance, making it easy to audit, monitor, and manage cloud environments at scale. If you haven’t already, get started with CloudQuery today to enhance your cloud visibility.

Have questions or need help? Join the CloudQuery Developer Community to connect with other users, share insights, and get support. Let us know how your team is tackling cloud security by engaging with us on LinkedIn or X.