Set up process

• The AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN environment variables
• The credentials and config files in ~/.aws (the credentials file takes priority)
• You can also use aws sso to authenticate the plugin - you can read more about it here
• IAM roles for AWS compute resources (including EC2 instances, Fargate and ECS containers)

Environment Variables

export AWS_ACCESS_KEY_ID={Your AWS Access Key ID}
export AWS_SECRET_ACCESS_KEY={Your AWS secret access key}
export AWS_SESSION_TOKEN={Your AWS session token}

Shared Configuration files

[default]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY

[myprofile]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY

export AWS_PROFILE=myprofile

accounts:
  id: <account_alias>
  local_profile: myprofile

IAM Roles for AWS Compute Resources

User Credentials with MFA

aws sts get-session-token --serial-number <YOUR_MFA_SERIAL_NUMBER> --token-code <YOUR_MFA_TOKEN_CODE> --duration-seconds 3600

export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>
export AWS_SESSION_TOKEN=<YOUR_SESSION_TOKEN>

Examples

Basic example

kind: source
spec:
  # Source spec section
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v26.1.0"
  tables: ["aws_ec2_instances"]
  destinations: ["postgresql"]
  spec:
    # Optional parameters
    # regions: []
    # accounts: []
    # org: nil
    # concurrency: 50000
    # initialization_concurrency: 4
    # aws_debug: false
    # max_retries: 10
    # max_backoff: 30
    # custom_endpoint_url: ""
    # custom_endpoint_hostname_immutable: nil # required when custom_endpoint_url is set
    # custom_endpoint_partition_id: "" # required when custom_endpoint_url is set
    # custom_endpoint_signing_region: "" # required when custom_endpoint_url is set
    # use_paid_apis: false
    # table_options: nil
    # scheduler: shuffle # options are: dfs, round-robin or shuffle
    # use_nested_table_rate_limiting: false 
    # enable_api_level_tracing: false

AWS Organization Example

kind: source
spec:
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v26.1.0"
  tables: ['aws_s3_buckets']
  destinations: ["postgresql"]
  spec:
    aws_debug: false
    org:
      admin_account:
        local_profile: "<NAMED_PROFILE>"
      member_role_name: OrganizationAccountAccessRole
    regions:
      - '*'
    # Optional parameters
    # regions: []
    # accounts: []
    # org: nil
    # concurrency: 50000
    # initialization_concurrency: 4
    # aws_debug: false
    # max_retries: 10
    # max_backoff: 30
    # custom_endpoint_url: ""
    # custom_endpoint_hostname_immutable: nil # required when custom_endpoint_url is set
    # custom_endpoint_partition_id: "" # required when custom_endpoint_url is set
    # custom_endpoint_signing_region: "" # required when custom_endpoint_url is set
    # use_paid_apis: false
    # table_options: nil
    # scheduler: shuffle # options are: dfs, round-robin or shuffle

Configuration spec

• regions ([]string) (default: []. Will use all enabled regions)

  Regions to use.
• accounts ([]Account) (default: current account)

  List of all accounts to fetch information from.
• org (Org) (default: not used)

  In AWS organization mode, the plugin will source all accounts underneath automatically.
• concurrency (integer) (default: 50000)

  The best effort maximum number of Go routines to use. Lower this number to reduce memory usage.
• initialization_concurrency (integer) (default: 4)

  During initialization the AWS source plugin fetches information about each account and region. This setting controls how many accounts can be initialized concurrently.

  Only configurations with many accounts (either hardcoded or discovered via Organizations) should require modifying this setting, to either lower it to avoid rate limit errors, or to increase it to speed up the initialization process.
• scheduler (string) (default: shuffle):

  The scheduler to use when determining the priority of resources to sync. Currently, the only supported values are dfs (depth-first search), round-robin and shuffle.

  For more information about this, see performance tuning.
• aws_debug (boolean) (default: false)

  If true, will log AWS debug logs, including retries and other request/response metadata.
• max_retries (integer) (default: 10)

  Defines the maximum number of times an API request will be retried.
• max_backoff (integer in seconds) (default: 30 meaning 30s)

  Defines the duration (in seconds) between retry attempts.
• use_nested_table_rate_limiting (boolean) (default: false)

  If true, the plugin will limit the number of nested tables that are synced concurrently.
• enable_api_level_tracing (boolean) (default: false)

  If true, the plugin will extend table level traces to include API requests to AWS Services
• custom_endpoint_url (string) (default: not used)

  The base URL endpoint the SDK API clients will use to make API calls to. The SDK will suffix URI path and query elements to this endpoint
• custom_endpoint_hostname_immutable (boolean) (default: not used)

  Specifies if the endpoint's hostname can be modified by the SDK's API client. When using something like LocalStack make sure to set it equal to true.
• custom_endpoint_partition_id (string) (default: not used)

  The AWS partition the endpoint belongs to.
• custom_endpoint_signing_region (string) (default: not used)

  The region that should be used for signing the request to the endpoint.
• use_paid_apis (boolean) (default: false)

  When set to true plugin will sync data from APIs that incur a fee.

  Tables that require this setting to be set to true include (but not limited to):

  • aws_costexplorer*
  • aws_cloudwatch_metric*
• table_options (map) (default: not used)

  Table options is a premium feature. Even if some tables are free, syncing data for them (& their relations) using table options counts towards paid usage.

  This feature enables users to override the default options for specific tables. The root of the object takes a table name, and the next level takes an API method name. The final level is the actual input object as defined by the API.

  The format of the table_options object is as follows:

  table_options:
    <table_name>:
      <api_method_name>:
        - <input_object>

  A list of <input_object> objects should be provided. The plugin will iterate through these to make multiple API calls. This is useful for APIs like CloudTrail's LookupEvents that only supports a single event type per call. For example:

  table_options:
      aws_cloudtrail_events:
        lookup_events:
          - start_time: 2023-05-01T20:20:52Z
            end_time:   2023-05-03T20:20:52Z
            lookup_attributes:
              - attribute_key:   EventName
                attribute_value: RunInstances
          - start_time: 2023-05-01T20:20:52Z
            end_time:   2023-05-03T20:20:52Z
            lookup_attributes:
              - attribute_key:   EventName
                attribute_value: StartInstances
          - start_time: 2023-05-01T20:20:52Z
            end_time:   2023-05-03T20:20:52Z
            lookup_attributes:
              - attribute_key:   EventName
                attribute_value: StopInstances

  The naming for all the fields is the same as the AWS API but in snake case. For example EndTime is represented as end_time.

  The following tables and APIs are supported:

  table_options:
    aws_accessanalyzer_analyzer_findings:
      list_findings:
        - <AccessAnalyzer.ListFindings> # NextToken & AnalyzerArn are prohibited
  
    aws_accessanalyzer_analyzer_findings_v2:
      list_findings_v2:
        - <AccessAnalyzer.ListFindingsV2> # NextToken & AnalyzerArn are prohibited
  
    aws_cloudtrail_events:
      lookup_events:
        - <CloudTrail.LookupEvents> # NextToken is prohibited
  
    aws_cloudwatch_metrics:
      - list_metrics: <CloudWatch.ListMetrics> # NextToken is prohibited
        get_metric_statistics:
          - <CloudWatch.GetMetricStatistics>  # Namespace, MetricName and Dimensions are prohibited
  
    aws_costexplorer_cost_custom:
      get_cost_and_usage:
        - <CostExplorer.GetCostAndUsage> # NextPageToken is prohibited
  
    aws_ec2_images:
      describe_images:
        - <EC2.DescribeImages> # NextToken and ImageIds are prohibited. MaxResults should be in range [1-1000].
  
    aws_ec2_instances:
      describe_instances:
        - <EC2.DescribeInstances> # NextToken is prohibited. MaxResults should be in range [1-1000].
  
    aws_ec2_internet_gateways:
      describe_internet_gateways:
        - <EC2.DescribeInternetGateways> # NextToken is prohibited. MaxResults should be in range [5-1000].
  
    aws_ec2_network_interfaces:
      describe_network_interfaces:
        - <EC2.DescribeNetworkInterfaces> # NextToken is prohibited. MaxResults should be in range [5-1000].
  
    aws_ec2_route_tables:
      describe_route_tables:
        - <EC2.DescribeRouteTables> # NextToken is prohibited. MaxResults should be in range [5-100].
  
    aws_ec2_security_groups:
      describe_security_groups:
        - <EC2.DescribeSecurityGroups> # NextToken is prohibited. MaxResults should be in range [5-1000].
  
    aws_ec2_subnets:
      describe_subnets:
        - <EC2.DescribeSubnets> # NextToken is prohibited. MaxResults should be in range [5-1000].
  
    aws_ec2_vpcs:
      describe_vpcs:
        - <EC2.DescribeVpcs> # NextToken is prohibited. MaxResults should be in range [5-1000].
  
    aws_ecs_cluster_tasks:
      list_tasks:
        - <ECS.ListTasks> # Cluster and NextToken are prohibited. MaxResults should be in range [1-100].
  
    aws_guardduty_detectors:
      - list_detectors: <GuardDuty.ListDetectors> # NextToken is prohibited
        list_findings: <GuardDuty.ListFindings> # NextToken and DetectorID are prohibited
  
    aws_iam_groups:
      get_group:
        - <IAM.GetGroup> # Marker is prohibited. MaxItems should be in range [1-1000].
  
    aws_iam_roles:
      get_role:
        - <IAM.GetRole> # RoleName is required.
  
    aws_iam_users:
      get_user:
        - <IAM.GetUser> # UserName is required.
  
    aws_inspector2_covered_resources:
      list_coverage:
        - <InspectorV2.ListCoverage> # NextToken is prohibited. MaxResults should be in range [1-200].
  
    aws_inspector2_findings:
      list_findings:
        - <InspectorV2.ListFindings> # NextToken is prohibited.
  
    aws_rds_clusters:
      describe_db_clusters:
        - <RDS.DescribeDBClusters> # Marker is prohibited. MaxRecords should be in range [20-100].
  
    aws_rds_instances:
      describe_db_instances:
        - <RDS.DescribeDBInstances> # Marker is prohibited. MaxRecords should be in range [20-100].
  
    aws_securityhub_findings:
      get_findings:
        - <SecurityHub.GetFindings> # NextToken is prohibited. MaxResults should be in range [1-100].

  The corresponding requests mentioned above:

  • AccessAnalyzer.ListFindings
  • AccessAnalyzer.ListFindingsV2
  • CloudTrail.LookupEvents
  • CloudWatch.ListMetrics
  • CloudWatch.GetMetricStatistics
  • CostExplorer.GetCostAndUsage
  • EC2.DescribeImages
  • EC2.DescribeInstances
  • EC2.DescribeInternetGateways
  • EC2.DescribeNetworkInterfaces
  • EC2.DescribeRouteTables
  • EC2.DescribeSecurityGroups
  • EC2.DescribeSubnets
  • EC2.DescribeVpcs
  • ECS.ListTasks
  • GuardDuty.ListDetectors
  • GuardDuty.ListFindings
  • IAM.GetGroup
  • IAM.GetRole
  • IAM.GetUser
  • InspectorV2.ListCoverage
  • InspectorV2.ListFindings
  • RDS.DescribeDBClusters
  • RDS.DescribeDBInstances
  • SecurityHub.GetFindings
• event_based_sync ([]Event-based sync) (default: empty)

  Event-based sync is a premium feature. Even if some tables are free, syncing data for them (& their relations) using event-based sync counts towards paid usage.

Account

• id (string) (required)

  Will be used as an alias in the source plugin and in the logs.
• local_profile (string) (default: will use current credentials)

  Local profile to use to authenticate this account with. Please note this should be set to the name of the profile.

  For example, with the following credentials file:

  [default]
  aws_access_key_id=xxxx
  aws_secret_access_key=xxxx
  
  [user1]
  aws_access_key_id=xxxx
  aws_secret_access_key=xxxx

  local_profile should be set to either default or user1.
• role_arn (string)

  If specified will use this to assume role.
• role_session_name (string)

  If specified will use this session name when assume role to role_arn.
• external_id (string)

  If specified will use this when assuming role to role_arn.
• default_region (string) (default: us-east-1)

  If specified, this region will be used as the default region for the account.
• regions ([]string)

  Regions to use for this account. Defaults to global regions setting.

org

• admin_account (Account)

  Configuration for how to grab credentials from an Admin account.
• member_trusted_principal (Account)

  Configuration for how to specify the principle to use in order to assume a role in the member accounts.
• member_role_name (string) (required)

  Role name that the plugin should use to assume a role in the member account from the admin account.

  Note: This is not a full ARN, it is just the name.
• member_role_session_name (string)

  Overrides the default session name.
• member_external_id (string)

  Specify an external ID for use in the trust policy.
• member_regions ([]string)

  Limit fetching resources within this specific account to only these regions. This will override any regions specified in the provider block. You can specify all regions by using the * character as the only argument in the array.
• organization_units ([]string)

  List of Organizational Units that AWS plugin should use to source accounts from. If you specify an OU, the plugin will also traverse nested OUs.
• skip_organization_units ([]string)

  List of Organizational Units to skip. This is useful in conjunction with organization_units if there are child OUs that should be ignored.
• skip_member_accounts ([]string)

  List of OU member accounts to skip. This is useful if there are accounts under the selected OUs that should be ignored.

Event-based sync

• kinesis_stream_arn (string) (required if sqs_queue_url is not provided)

  ARN for the Kinesis stream that will hold all the CloudTrail records.
• sqs_queue_url (string) (required if kinesis_stream_arn is not provided)

  URL for the SQS queue that will hold the S3 Bucket Notifications.
• account (Account)

  Configuration for the credentials that will be used to grab records from the specified Kinesis Stream. If this is not specified the default credentials will be used.
• start_time (string for RFC 3339 timestamp) (default: the time at which the sync began)

  Defines the place in the stream where record processing should begin. The value should follow the RFC 3339 format, for example: 2023-09-04T19:24:14Z.
• full_sync (boolean) (default: true)

  By default, AWS plugin will do a full sync on the specified tables before starting to consume the events in the stream. This parameter enables users to skip the full pull based sync and go straight to the event based sync.

Skip Tables

kind: source
spec:
  # Source spec section
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v26.1.0"
  tables: ["*"]
  skip_tables:
    - aws_cloudtrail_events
    - aws_docdb_cluster_parameter_groups
    - aws_docdb_engine_versions
    - aws_ec2_instance_types
    - aws_ec2_vpc_endpoint_services
    - aws_elasticache_engine_versions
    - aws_elasticache_parameter_groups
    - aws_elasticache_reserved_cache_nodes_offerings
    - aws_elasticache_service_updates
    - aws_iam_group_last_accessed_details
    - aws_iam_policy_last_accessed_details
    - aws_iam_role_last_accessed_details
    - aws_iam_user_last_accessed_details
    - aws_neptune_cluster_parameter_groups
    - aws_neptune_db_parameter_groups
    - aws_rds_cluster_parameter_groups
    - aws_rds_db_parameter_groups
    - aws_rds_engine_versions
    - aws_servicequotas_services
    - aws_stepfunctions_map_run_executions
    - aws_stepfunctions_map_runs
  destinations: ["postgresql"]
  spec:
    # AWS Spec section described below

Supported Services and Events

Configuration Using Kinesis Data Stream

1. Configure an AWS CloudTrail Trail to send management events to a Kinesis Data Stream via CloudWatch Logs. The most straight-forward way to do this is to use the CloudFormation template provided by CloudQuery.

   The CloudFormation template will deploy the following architecture:

   

   The template contents can be found in CloudFormation Template contents section below.

   aws cloudformation deploy --template-file ./streaming-deployment.yml --stack-name <STACK-NAME> --capabilities CAPABILITY_IAM --disable-rollback --region <DESIRED-REGION>

2. Copy the ARN of the Kinesis stream. If you used the CloudFormation template you can run the following command:

   aws cloudformation describe-stacks --stack-name <STACK-NAME> --query "Stacks[].Outputs" --region <DESIRED-REGION>

3. Define a config.yml file like the one below

   kind: source
   spec:
     name: aws
     path: cloudquery/aws
     registry: cloudquery
     version: "v26.1.0"
     tables:
       - aws_ec2_instances
       - aws_ec2_internet_gateways
       - aws_ec2_security_groups
       - aws_ec2_subnets
       - aws_ec2_vpcs
       - aws_ecs_cluster_tasks
       - aws_iam_groups
       - aws_iam_roles
       - aws_iam_users
       - aws_rds_instances
     destinations: ["postgresql"]
     spec:
       event_based_sync:
         # account:
         #  local_profile: "<ROLE-NAME>"
         kinesis_stream_arn: <OUTPUT-FROM-CLOUDFORMATION-STACK>

4. Sync the data!

   cloudquery sync config.yml

Limitations

• Kinesis Stream can only have a single shard (this is a limitation that we expect to remove in the future)
• Stale records will only be deleted if the plugin stops consuming the Kinesis Stream, which only can occur if there is an error

Configuration Using S3 Bucket Notifications

1. Create anew SQS queue:

aws sqs create-queue --queue-name <REPLACE_WITH_QUEUE_NAME> bucket-notifications

2. Create a file defining the permissions for the SQS queue and save it as sqs-policy.json:

{
    "Version": "2012-10-17",
    "Statement": [{"Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": [
                "SQS:SendMessage"
            ],
            "Resource": "arn:aws:sqs:<REGION>:<ACCOUNT_ID>:<REPLACE_WITH_QUEUE_NAME>",
            "Condition": {
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:s3:*:*:<REPLACE_WITH_BUCKET_NAME>"
                },
                "StringEquals": {
                    "aws:SourceAccount": "<REPLACE_WITH_BUCKET_OWNER_ACCOUNT_ID>"
                }
            }
        }
    ]
}

aws sqs set-queue-attributes --queue-url <queue_url> --policy file://sqs-policy.json

4. Create a file defining the integration between the S3 bucket and the SQS queue and save it as s3-notification.json:

{
    "QueueConfigurations": [
        {
            "QueueArn": "arn:aws:sqs:<REGION>:<ACCOUNT_ID>:<REPLACE_WITH_QUEUE_NAME>",
            "Events": [
                "s3:ObjectCreated:*"
            ]
        }
    ]
}

aws s3api put-bucket-notification-configuration --bucket <REPLACE_WITH_BUCKET_NAME> --notification-configuration file://s3-notification.json

kind: source
spec:
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v26.1.0"
  tables:
    - aws_ec2_instances
    - aws_ec2_internet_gateways
    - aws_ec2_security_groups
    - aws_ec2_subnets
    - aws_ec2_vpcs
    - aws_ecs_cluster_tasks
    - aws_iam_groups
    - aws_iam_roles
    - aws_iam_users
    - aws_rds_instances
  destinations: ["postgresql"]
  spec:
    event_based_sync:
      # account:
      #  local_profile: "<ROLE-NAME>"
      sqs_queue_url: <OUTPUT-FROM-CREATE-QUEUE-COMMAND>

6. Sync the data!

   cloudquery sync config.yml

Limitations

• This method is not the fastest way to consume CloudTrail events as the data gets buffered before being sent to S3 and then bucket notifications can also have a delay
• Stale records will only be deleted if the plugin stops consuming the Kinesis Stream, which only can occur if there is an error

Template contents:

AWSTemplateFormatVersion: 2010-09-09
Description: Configures Cloudtrail Events to be piped to a Kinesis Data stream via CloudWatch Logs.

Parameters:

  KinesisMessageDuration:
    Type: Number
    Description: Number of hours Kinesis will persist a record before it is purged.
    Default: 24
  ExistingS3BucketName:
    Type: String
    Description: Name of the S3 Bucket that CloudTrail will use to store logs.
    Default: ""

Conditions:
  CreateS3Bucket: !Equals [!Ref ExistingS3BucketName, ""]

Resources:
  # Stream that CQ will poll for changes
  CQSyncingKinesisStream:
    Type: AWS::Kinesis::Stream
    Properties:
      ShardCount: 1
      RetentionPeriodHours: !Ref KinesisMessageDuration

  # IAM Role for allowing CloudWatch Log to write to Kinesis Stream
  CloudWatchLogsToKinesisRole:
    Type: AWS::IAM::Role
    Properties:
      Policies:
      - PolicyName: CloudWatchLogsToKinesisPolicy
        PolicyDocument: 
          Version: 2012-10-17
          Statement:
            - Effect: Allow
              Action:
                - kinesis:PutRecord
              Resource: !GetAtt CQSyncingKinesisStream.Arn
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: logs.amazonaws.com
            Action:
              - sts:AssumeRole

  
  CloudTrailS3Bucket:
    Type: AWS::S3::Bucket
    Condition: CreateS3Bucket
    Properties:
      LifecycleConfiguration:
        Rules:
          - ExpirationInDays: 30
            Status: Enabled

  CloudTrailS3BucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !If [CreateS3Bucket,!Ref CloudTrailS3Bucket, !Ref ExistingS3BucketName]
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: AWSCloudTrailAclCheck
            Effect: Allow
            Principal:
              Service: cloudtrail.amazonaws.com
            Action: s3:GetBucketAcl
            Resource: !Sub
                - arn:${AWS::Partition}:s3:::${Bucket}
                - { Bucket: !If [CreateS3Bucket,!Ref CloudTrailS3Bucket, !Ref ExistingS3BucketName] }
            Condition:
              StringEquals:
                'aws:SourceAccount': !Sub ${AWS::AccountId}
          - Sid: AWSCloudTrailWrite
            Effect: Allow
            Principal:
              Service: cloudtrail.amazonaws.com
            Action: s3:PutObject
            Resource: !Sub
                - arn:${AWS::Partition}:s3:::${Bucket}/*
                - { Bucket: !If [CreateS3Bucket,!Ref CloudTrailS3Bucket, !Ref ExistingS3BucketName] }
            Condition:
              StringEquals:
                's3:x-amz-acl': bucket-owner-full-control
                'aws:SourceAccount': !Sub ${AWS::AccountId}

  CloudWatchLogsGroup:
    Type: AWS::Logs::LogGroup
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Properties:
      LogGroupName: "CloudTrailLogGroup"
      RetentionInDays: 1

  # Role for allowing CLoudTrail to write to CloudWatch Logs
  CloudWatchRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Sid: AssumeRole
          Effect: Allow
          Principal:
            Service: 'cloudtrail.amazonaws.com'
          Action: 'sts:AssumeRole'
      Policies:
      - PolicyName: 'cloudtrail-policy'
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action: 'logs:CreateLogStream'
            Resource: !GetAtt CloudWatchLogsGroup.Arn
          - Effect: Allow
            Action: 'logs:PutLogEvents'
            Resource: !GetAtt CloudWatchLogsGroup.Arn
  CloudTrailTrail:
    Type: AWS::CloudTrail::Trail
    DependsOn:
         - CloudTrailS3BucketPolicy
    Properties:
      CloudWatchLogsLogGroupArn: !GetAtt CloudWatchLogsGroup.Arn
      CloudWatchLogsRoleArn: !GetAtt CloudWatchRole.Arn
      EventSelectors:
        - IncludeManagementEvents: True
          ReadWriteType: WriteOnly
      IncludeGlobalServiceEvents: True
      IsLogging: True
      IsMultiRegionTrail: True
      S3BucketName: !If [CreateS3Bucket,!Ref CloudTrailS3Bucket, !Ref ExistingS3BucketName]

  SubscriptionFilter:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      LogGroupName: !Ref CloudWatchLogsGroup
      DestinationArn: !GetAtt CQSyncingKinesisStream.Arn
      RoleArn: !GetAtt CloudWatchLogsToKinesisRole.Arn
      FilterPattern: ""

  

Outputs:
  KinesisStreamArn:
    Description: The ARN of the Kinesis Data Stream that CloudQuery will use to listen for changes.
    Value: !GetAtt CQSyncingKinesisStream.Arn

AWS Organizations

kind: source
spec:
  name: aws
  path: cloudquery/aws
  registry: cloudquery
  version: "v26.1.0"
  tables: ['aws_s3_buckets']
  destinations: ["postgresql"]
  spec:
    aws_debug: false
    org:
      admin_account:
        local_profile: "<NAMED_PROFILE>"
      member_role_name: cloudquery-ro
    regions:
      - '*'

1. Have a role (or user) in an Admin account with the following access:

   • organizations:ListAccounts
   • organizations:ListAccountsForParent
   • organizations:ListChildren
2. Have a role in each member account that has a trust policy with a single principal.

   The default profile name is OrganizationAccountAccessRole. The OrganizationAccountAccessRole is created by default in AWS Accounts created as part of an AWS Organization. We do not recommend using the OrganizationAccountAccessRole due to the level of permissions typically granted to the role, but instead recommend for AWS plugin users to create their own IAM roles in each member account with the appropriate read-only permissions. We also recommend ensuring that the IAM roles and policies used for AWS plugin adhere to company security standards.

   Reference IAM assets and the CloudFormation templates for deployment in an AWS Organization for CloudQuery can be found here.

Configuring AWS Organization:

1. It is always necessary to specify a member role name:

   org:
         member_role_name: cloudquery-ro

2. Sourcing credentials that have the necessary organizations permissions can be done in any of the following ways:

   1. Source credentials from the default credential tool chain:

      org:
            member_role_name: cloudquery-ro

   2. Source credentials from a named profile in the shared configuration or credentials file

      org:
            member_role_name: cloudquery-ro
            admin_account:
              local_profile: <Named-Profile>

   3. Assume a role in admin account using credentials in the shared configuration or credentials file:

      org:
            member_role_name: cloudquery-ro
            admin_account:
              local_profile: <Named-Profile>
              role_arn: arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>
      
              # Optional. Specify the name of the session
              # role_session_name: ""
      
              # Optional. Specify the ExternalID if required for trust policy
              # external_id: ""

3. Optional. If the trust policy configured for the member accounts requires different credentials than you configured in the previous step, then you can specify the credentials to use in the member_trusted_principal block:

   org:
         member_role_name: cloudquery-ro
         member_trusted_principal:
           local_profile: <Named-Profile-Member>

4. Optional. If you want to specify specific Organizational Units to fetch from you can add them to the organization_units list.

   org:
         member_role_name: cloudquery-ro
         organization_units:
           - ou-<ID-1>
           - ou-<ID-2>

   Child OUs will also be included. To skip a child OU or account, use the skip_organization_units or skip_member_accounts options respectively:

   org:
         member_role_name: cloudquery-ro
         organization_units:
           - ou-<ID-1>
           - ou-<ID-2>
         skip_organization_units:
           - ou-<ID-3>
         skip_member_accounts:
           - <ACCOUNT_ID>

Specific Accounts

accounts:
  - id: <AccountID_Alias_1>
    role_arn: <YOUR_ROLE_ARN_1>
    # Optional. Local Profile is the named profile in your shared configuration file (usually `~/.aws/config`) that you want to use for this specific account
    local_profile: <NAMED_PROFILE>
    # Optional. Specify the Role Session name
    role_session_name: ""
  - id: <AccountID_Alias_2>
    local_profile: provider
    # Optional. Role ARN we want to assume when accessing this account
    role_arn: <YOUR_ROLE_ARN_2>

• aws_accessanalyzer_analyzer_archive_rules
• aws_accessanalyzer_analyzer_findings
• aws_accessanalyzer_analyzer_findings_v2
• aws_accessanalyzer_analyzers
• aws_account_alternate_contacts
• aws_account_contacts
• aws_acm_certificates
• aws_acmpca_certificate_authorities
• aws_amp_rule_groups_namespaces
• aws_amp_workspaces
• aws_amplify_apps
• aws_apigateway_api_keys
• aws_apigateway_client_certificates
• aws_apigateway_domain_name_base_path_mappings
• aws_apigateway_domain_names
• aws_apigateway_rest_api_authorizers
• aws_apigateway_rest_api_deployments
• aws_apigateway_rest_api_documentation_parts
• aws_apigateway_rest_api_documentation_versions
• aws_apigateway_rest_api_gateway_responses
• aws_apigateway_rest_api_models
• aws_apigateway_rest_api_request_validators
• aws_apigateway_rest_api_resource_method_integrations
• aws_apigateway_rest_api_resource_methods
• aws_apigateway_rest_api_resources
• aws_apigateway_rest_api_stages
• aws_apigateway_rest_apis
• aws_apigateway_usage_plan_keys
• aws_apigateway_usage_plans
• aws_apigateway_vpc_links
• aws_apigatewayv2_api_authorizers
• aws_apigatewayv2_api_deployments
• aws_apigatewayv2_api_integration_responses
• aws_apigatewayv2_api_integrations
• aws_apigatewayv2_api_models
• aws_apigatewayv2_api_route_responses
• aws_apigatewayv2_api_routes
• aws_apigatewayv2_api_stages
• aws_apigatewayv2_apis
• aws_apigatewayv2_domain_name_rest_api_mappings
• aws_apigatewayv2_domain_names
• aws_apigatewayv2_vpc_links
• aws_appconfig_applications
• aws_appconfig_configuration_profiles
• aws_appconfig_deployment_strategies
• aws_appconfig_environments
• aws_appconfig_hosted_configuration_versions
• aws_appflow_flows
• aws_applicationautoscaling_policies
• aws_applicationautoscaling_scalable_targets
• aws_applicationautoscaling_scaling_activities
• aws_applicationautoscaling_scheduled_actions
• aws_appmesh_meshes
• aws_appmesh_virtual_gateways
• aws_appmesh_virtual_nodes
• aws_appmesh_virtual_routers
• aws_appmesh_virtual_services
• aws_apprunner_auto_scaling_configurations
• aws_apprunner_connections
• aws_apprunner_custom_domains
• aws_apprunner_observability_configurations
• aws_apprunner_operations
• aws_apprunner_services
• aws_apprunner_vpc_connectors
• aws_apprunner_vpc_ingress_connections
• aws_appstream_app_blocks
• aws_appstream_application_fleet_associations
• aws_appstream_applications
• aws_appstream_directory_configs
• aws_appstream_fleets
• aws_appstream_image_builders
• aws_appstream_stack_entitlements
• aws_appstream_stack_user_associations
• aws_appstream_stacks
• aws_appstream_usage_report_subscriptions
• aws_appstream_users
• aws_appsync_graphql_apis
• aws_athena_data_catalog_database_tables
• aws_athena_data_catalog_databases
• aws_athena_data_catalogs
• aws_athena_work_group_named_queries
• aws_athena_work_group_prepared_statements
• aws_athena_work_group_query_executions
• aws_athena_work_groups
• aws_auditmanager_assessments
• aws_autoscaling_group_lifecycle_hooks
• aws_autoscaling_group_scaling_policies
• aws_autoscaling_groups
• aws_autoscaling_launch_configurations
• aws_autoscaling_plan_resources
• aws_autoscaling_plans
• aws_autoscaling_scheduled_actions
• aws_autoscaling_warm_pools
• aws_backup_global_settings
• aws_backup_jobs
• aws_backup_plan_selections
• aws_backup_plans
• aws_backup_protected_resources
• aws_backup_region_settings
• aws_backup_report_plans
• aws_backup_vault_recovery_points
• aws_backup_vaults
• aws_backupgateway_gateways
• aws_batch_compute_environments
• aws_batch_job_definitions
• aws_batch_job_queues
• aws_batch_jobs
• aws_budgets_actions
• aws_budgets_budgets
• aws_cloudformation_stack_instance_resource_drifts
• aws_cloudformation_stack_instance_summaries
• aws_cloudformation_stack_resources
• aws_cloudformation_stack_set_operation_results
• aws_cloudformation_stack_set_operations
• aws_cloudformation_stack_sets
• aws_cloudformation_stack_templates
• aws_cloudformation_stacks
• aws_cloudformation_template_summaries
• aws_cloudhsmv2_backups
• aws_cloudhsmv2_clusters
• aws_cloudtrail_channels
• aws_cloudtrail_events
• aws_cloudtrail_imports
• aws_cloudtrail_trail_event_selectors
• aws_cloudtrail_trails
• aws_cloudwatch_alarms
• aws_cloudwatch_metric_statistics
• aws_cloudwatch_metric_streams
• aws_cloudwatch_metrics
• aws_cloudwatchlogs_log_group_data_protection_policies
• aws_cloudwatchlogs_log_group_subscription_filters
• aws_cloudwatchlogs_log_groups
• aws_cloudwatchlogs_metric_filters
• aws_cloudwatchlogs_resource_policies
• aws_codeartifact_domains
• aws_codeartifact_repositories
• aws_codebuild_builds
• aws_codebuild_projects
• aws_codebuild_source_credentials
• aws_codecommit_repositories
• aws_codedeploy_applications
• aws_codedeploy_deployment_configs
• aws_codedeploy_deployment_groups
• aws_codedeploy_deployments
• aws_codegurureviewer_repository_associations
• aws_codepipeline_pipelines
• aws_codepipeline_webhooks
• aws_cognito_identity_pools
• aws_cognito_user_pool_identity_providers
• aws_cognito_user_pools
• aws_computeoptimizer_autoscaling_group_recommendations
• aws_computeoptimizer_ebs_volume_recommendations
• aws_computeoptimizer_ec2_instance_recommendations
• aws_computeoptimizer_ecs_service_recommendations
• aws_computeoptimizer_enrollment_statuses
• aws_computeoptimizer_lambda_function_recommendations
• aws_config_config_rule_compliance_details
• aws_config_config_rule_compliances
• aws_config_config_rules
• aws_config_configuration_aggregators
• aws_config_configuration_recorders
• aws_config_conformance_pack_rule_compliances
• aws_config_conformance_packs
• aws_config_delivery_channel_statuses
• aws_config_delivery_channels
• aws_config_remediation_configurations
• aws_config_retention_configurations
• aws_costexplorer_cost_30d
• aws_costexplorer_cost_custom
• aws_costexplorer_cost_forecast_30d
• aws_datapipeline_pipelines
• aws_datasync_agents
• aws_datasync_azureblob_locations
• aws_datasync_efs_locations
• aws_datasync_fsxlustre_locations
• aws_datasync_fsxontap_locations
• aws_datasync_fsxopenzfs_locations
• aws_datasync_fsxwindows_locations
• aws_datasync_hdfs_locations
• aws_datasync_locations
• aws_datasync_nfs_locations
• aws_datasync_objectstorage_locations
• aws_datasync_s3_locations
• aws_datasync_smb_locations
• aws_dax_clusters
• aws_detective_graph_members
• aws_detective_graphs
• aws_directconnect_connections
• aws_directconnect_gateway_associations
• aws_directconnect_gateway_attachments
• aws_directconnect_gateways
• aws_directconnect_lags
• aws_directconnect_locations
• aws_directconnect_virtual_gateways
• aws_directconnect_virtual_interfaces
• aws_directoryservice_directories
• aws_dms_certificates
• aws_dms_event_subscriptions
• aws_dms_replication_instances
• aws_dms_replication_subnet_groups
• aws_dms_replication_tasks
• aws_docdb_certificates
• aws_docdb_cluster_parameter_groups
• aws_docdb_cluster_parameters
• aws_docdb_cluster_snapshots
• aws_docdb_clusters
• aws_docdb_engine_versions
• aws_docdb_event_categories
• aws_docdb_event_subscriptions
• aws_docdb_events
• aws_docdb_global_clusters
• aws_docdb_instances
• aws_docdb_orderable_db_instance_options
• aws_docdb_pending_maintenance_actions
• aws_docdb_subnet_groups
• aws_dynamodb_backups
• aws_dynamodb_exports
• aws_dynamodb_global_tables
• aws_dynamodb_table_continuous_backups
• aws_dynamodb_table_replica_auto_scalings
• aws_dynamodb_table_resource_policies
• aws_dynamodb_table_stream_resource_policies
• aws_dynamodb_tables
• aws_dynamodbstreams_streams
• aws_ebs_default_kms_key_ids
• aws_ebs_encryption_by_defaults
• aws_ec2_ebs_snapshot_attributes
• aws_ec2_ebs_snapshots
• aws_ec2_ebs_volume_statuses
• aws_ec2_ebs_volumes
• aws_ec2_image_block_public_access_states
• aws_ec2_instance_credit_specifications
• aws_ec2_ipam_address_history
• aws_ec2_ipam_byoasns
• aws_ec2_ipam_discovered_accounts
• aws_ec2_ipam_discovered_public_addresses
• aws_ec2_ipam_discovered_resource_cidrs
• aws_ec2_ipam_pool_allocations
• aws_ec2_ipam_pool_cidrs
• aws_ec2_ipam_pools
• aws_ec2_ipam_resource_cidrs
• aws_ec2_ipam_resource_discoveries
• aws_ec2_ipam_resource_discovery_associations
• aws_ec2_ipam_scopes
• aws_ec2_ipams
• aws_ec2_replace_root_volume_tasks
• aws_ec2_serial_console_access_statuses
• aws_ec2_snapshot_block_public_access_states
• aws_ec2_traffic_mirror_filters
• aws_ec2_traffic_mirror_sessions
• aws_ec2_traffic_mirror_targets
• aws_ecr_pull_through_cache_rules
• aws_ecr_registries
• aws_ecr_registry_policies
• aws_ecr_repositories
• aws_ecr_repository_image_scan_findings
• aws_ecr_repository_images
• aws_ecr_repository_lifecycle_policies
• aws_ecr_repository_policies
• aws_ecrpublic_repositories
• aws_ecrpublic_repository_images
• aws_ecs_cluster_container_instances
• aws_ecs_cluster_services
• aws_ecs_cluster_task_sets
• aws_ecs_cluster_tasks
• aws_ecs_clusters
• aws_ecs_task_definitions
• aws_efs_access_points
• aws_efs_filesystems
• aws_eks_cluster_addons
• aws_eks_cluster_node_groups
• aws_eks_cluster_oidc_identity_provider_configs
• aws_eks_clusters
• aws_eks_fargate_profiles
• aws_elasticache_clusters
• aws_elasticache_engine_versions
• aws_elasticache_events
• aws_elasticache_global_replication_groups
• aws_elasticache_parameter_groups
• aws_elasticache_replication_groups
• aws_elasticache_reserved_cache_nodes
• aws_elasticache_serverless_cache_snapshots
• aws_elasticache_serverless_caches
• aws_elasticache_snapshots
• aws_elasticache_subnet_groups
• aws_elasticache_update_actions
• aws_elasticache_user_groups
• aws_elasticache_users
• aws_elasticbeanstalk_application_versions
• aws_elasticbeanstalk_applications
• aws_elasticbeanstalk_configuration_options
• aws_elasticbeanstalk_configuration_settings
• aws_elasticbeanstalk_environments
• aws_elasticsearch_domains
• aws_elasticsearch_vpc_endpoints
• aws_elastictranscoder_pipeline_jobs
• aws_elastictranscoder_pipelines
• aws_elbv1_load_balancer_policies
• aws_elbv1_load_balancers
• aws_elbv2_listener_certificates
• aws_elbv2_listener_rules
• aws_elbv2_listeners
• aws_elbv2_load_balancer_attributes
• aws_elbv2_load_balancer_web_acls
• aws_elbv2_load_balancers
• aws_elbv2_target_group_target_health_descriptions
• aws_elbv2_target_groups
• aws_emr_block_public_access_configs
• aws_emr_cluster_instance_fleets
• aws_emr_cluster_instance_groups
• aws_emr_cluster_instances
• aws_emr_clusters
• aws_emr_notebook_executions
• aws_emr_security_configurations
• aws_emr_steps
• aws_emr_studio_session_mappings
• aws_emr_studios
• aws_eventbridge_api_destinations
• aws_eventbridge_archives
• aws_eventbridge_connections
• aws_eventbridge_endpoints
• aws_eventbridge_event_bus_rules
• aws_eventbridge_event_bus_targets
• aws_eventbridge_event_buses
• aws_eventbridge_event_sources
• aws_eventbridge_replays
• aws_firehose_delivery_streams
• aws_frauddetector_batch_imports
• aws_frauddetector_batch_predictions
• aws_frauddetector_detectors
• aws_frauddetector_entity_types
• aws_frauddetector_event_types
• aws_frauddetector_external_models
• aws_frauddetector_labels
• aws_frauddetector_model_versions
• aws_frauddetector_models
• aws_frauddetector_outcomes
• aws_frauddetector_rules
• aws_frauddetector_variables
• aws_fsx_backups
• aws_fsx_data_repository_associations
• aws_fsx_data_repository_tasks
• aws_fsx_file_caches
• aws_fsx_file_systems
• aws_fsx_snapshots
• aws_fsx_storage_virtual_machines
• aws_fsx_volumes
• aws_glacier_data_retrieval_policies
• aws_glacier_vault_access_policies
• aws_glacier_vault_lock_policies
• aws_glacier_vault_notifications
• aws_glacier_vaults
• aws_globalaccelerator_accelerators
• aws_globalaccelerator_custom_routing_accelerators
• aws_glue_classifiers
• aws_glue_connections
• aws_glue_crawlers
• aws_glue_database_table_indexes
• aws_glue_database_tables
• aws_glue_databases
• aws_glue_datacatalog_encryption_settings
• aws_glue_dev_endpoints
• aws_glue_job_runs
• aws_glue_jobs
• aws_glue_ml_transform_task_runs
• aws_glue_ml_transforms
• aws_glue_registries
• aws_glue_registry_schema_versions
• aws_glue_registry_schemas
• aws_glue_security_configurations
• aws_glue_triggers
• aws_glue_workflows
• aws_guardduty_detector_filters
• aws_guardduty_detector_findings
• aws_guardduty_detector_intel_sets
• aws_guardduty_detector_ip_sets
• aws_guardduty_detector_members
• aws_guardduty_detector_publishing_destinations
• aws_guardduty_detectors
• aws_healthlake_fhir_datastores
• aws_identitystore_group_memberships
• aws_identitystore_groups
• aws_identitystore_users
• aws_inspector2_covered_resources
• aws_inspector2_findings
• aws_inspector_findings
• aws_iot_billing_groups
• aws_iot_ca_certificates
• aws_iot_certificates
• aws_iot_jobs
• aws_iot_policies
• aws_iot_security_profiles
• aws_iot_streams
• aws_iot_thing_groups
• aws_iot_thing_types
• aws_iot_things
• aws_iot_topic_rules
• aws_kafka_cluster_operations
• aws_kafka_cluster_policies
• aws_kafka_clusters
• aws_kafka_configurations
• aws_kafka_nodes
• aws_keyspaces_keyspaces
• aws_keyspaces_tables
• aws_kinesis_streams
• aws_kms_aliases
• aws_kms_key_grants
• aws_kms_key_policies
• aws_kms_key_rotation_statuses
• aws_kms_key_rotations
• aws_kms_keys
• aws_lambda_function_aliases
• aws_lambda_function_concurrency_configs
• aws_lambda_function_event_invoke_configs
• aws_lambda_function_event_source_mappings
• aws_lambda_function_url_configs
• aws_lambda_function_versions
• aws_lambda_functions
• aws_lambda_layer_version_policies
• aws_lambda_layer_versions
• aws_lambda_layers
• aws_lambda_runtimes
• aws_lexv2_bot_aliases
• aws_lexv2_bots
• aws_lightsail_alarms
• aws_lightsail_bucket_access_keys
• aws_lightsail_buckets
• aws_lightsail_certificates
• aws_lightsail_container_service_deployments
• aws_lightsail_container_service_images
• aws_lightsail_container_services
• aws_lightsail_database_events
• aws_lightsail_database_log_events
• aws_lightsail_database_parameters
• aws_lightsail_database_snapshots
• aws_lightsail_databases
• aws_lightsail_disk_snapshots
• aws_lightsail_disks
• aws_lightsail_distributions
• aws_lightsail_instance_port_states
• aws_lightsail_instance_snapshots
• aws_lightsail_instances
• aws_lightsail_load_balancer_tls_certificates
• aws_lightsail_load_balancers
• aws_lightsail_static_ips
• aws_mq_broker_configuration_revisions
• aws_mq_broker_configurations
• aws_mq_broker_users
• aws_mq_brokers
• aws_mwaa_environments
• aws_neptune_cluster_parameter_group_parameters
• aws_neptune_cluster_parameter_groups
• aws_neptune_cluster_snapshots
• aws_neptune_clusters
• aws_neptune_db_parameter_group_db_parameters
• aws_neptune_db_parameter_groups
• aws_neptune_event_subscriptions
• aws_neptune_global_clusters
• aws_neptune_instances
• aws_neptune_subnet_groups
• aws_networkfirewall_firewall_policies
• aws_networkfirewall_firewalls
• aws_networkfirewall_rule_groups
• aws_networkfirewall_tls_inspection_configurations
• aws_networkmanager_global_networks
• aws_networkmanager_links
• aws_networkmanager_sites
• aws_networkmanager_transit_gateway_registrations
• aws_organization_resource_policies
• aws_organizations
• aws_organizations_account_parents
• aws_organizations_accounts
• aws_organizations_delegated_administrators
• aws_organizations_delegated_services
• aws_organizations_organizational_unit_parents
• aws_organizations_organizational_units
• aws_organizations_policies
• aws_organizations_roots
• aws_pinpoint_apps
• aws_qldb_ledger_journal_kinesis_streams
• aws_qldb_ledger_journal_s3_exports
• aws_qldb_ledgers
• aws_quicksight_analyses
• aws_quicksight_dashboards
• aws_quicksight_data_sets
• aws_quicksight_data_sources
• aws_quicksight_folders
• aws_quicksight_group_members
• aws_quicksight_groups
• aws_quicksight_ingestions
• aws_quicksight_templates
• aws_quicksight_users
• aws_ram_principals
• aws_ram_resource_share_associations
• aws_ram_resource_share_invitations
• aws_ram_resource_share_permissions
• aws_ram_resource_shares
• aws_ram_resources
• aws_redshift_cluster_parameter_groups
• aws_redshift_cluster_parameters
• aws_redshift_clusters
• aws_redshift_data_shares
• aws_redshift_endpoint_accesses
• aws_redshift_endpoint_authorizations
• aws_redshift_event_subscriptions
• aws_redshift_events
• aws_redshift_snapshots
• aws_redshift_subnet_groups
• aws_resiliencehub_alarm_recommendations
• aws_resiliencehub_app_assessments
• aws_resiliencehub_app_component_compliances
• aws_resiliencehub_app_version_resource_mappings
• aws_resiliencehub_app_version_resources
• aws_resiliencehub_app_versions
• aws_resiliencehub_apps
• aws_resiliencehub_component_recommendations
• aws_resiliencehub_recommendation_templates
• aws_resiliencehub_resiliency_policies
• aws_resiliencehub_sop_recommendations
• aws_resiliencehub_suggested_resiliency_policies
• aws_resiliencehub_test_recommendations
• aws_resourcegroups_resource_groups
• aws_route53_delegation_sets
• aws_route53_domains
• aws_route53_health_checks
• aws_route53_hosted_zone_query_logging_configs
• aws_route53_hosted_zone_resource_record_sets
• aws_route53_hosted_zone_traffic_policy_instances
• aws_route53_hosted_zones
• aws_route53_operations
• aws_route53_traffic_policies
• aws_route53_traffic_policy_versions
• aws_route53recoverycontrolconfig_clusters
• aws_route53recoverycontrolconfig_control_panels
• aws_route53recoverycontrolconfig_routing_controls
• aws_route53recoverycontrolconfig_safety_rules
• aws_route53recoveryreadiness_cells
• aws_route53recoveryreadiness_readiness_checks
• aws_route53recoveryreadiness_recovery_groups
• aws_route53recoveryreadiness_resource_sets
• aws_route53resolver_firewall_configs
• aws_route53resolver_firewall_domain_lists
• aws_route53resolver_firewall_rule_group_associations
• aws_route53resolver_firewall_rule_groups
• aws_route53resolver_resolver_endpoints
• aws_route53resolver_resolver_query_log_config_associations
• aws_route53resolver_resolver_query_log_configs
• aws_route53resolver_resolver_rule_associations
• aws_route53resolver_resolver_rules
• aws_s3_bucket_object_grants
• aws_s3_bucket_objects
• aws_sagemaker_apps
• aws_sagemaker_endpoint_configurations
• aws_sagemaker_endpoints
• aws_sagemaker_models
• aws_sagemaker_notebook_instances
• aws_sagemaker_training_jobs
• aws_savingsplans_plans
• aws_scheduler_schedule_groups
• aws_scheduler_schedules
• aws_secretsmanager_secret_versions
• aws_secretsmanager_secrets
• aws_securityhub_enabled_standards
• aws_securityhub_findings
• aws_securityhub_hubs
• aws_servicecatalog_launch_paths
• aws_servicecatalog_portfolios
• aws_servicecatalog_products
• aws_servicecatalog_provisioned_products
• aws_servicecatalog_provisioning_artifacts
• aws_servicecatalog_provisioning_parameters
• aws_servicediscovery_instances
• aws_servicediscovery_namespaces
• aws_servicediscovery_services
• aws_ses_active_receipt_rule_sets
• aws_ses_configuration_set_event_destinations
• aws_ses_configuration_sets
• aws_ses_contact_lists
• aws_ses_custom_verification_email_templates
• aws_ses_identities
• aws_ses_templates
• aws_shield_attacks
• aws_shield_protection_groups
• aws_shield_protections
• aws_shield_subscriptions
• aws_signer_signing_profiles
• aws_sns_subscriptions
• aws_sns_topics
• aws_sqs_queues
• aws_ssm_associations
• aws_ssm_compliance_summary_items
• aws_ssm_document_versions
• aws_ssm_documents
• aws_ssm_instance_compliance_items
• aws_ssm_instance_patches
• aws_ssm_instances
• aws_ssm_inventories
• aws_ssm_inventory_schemas
• aws_ssm_parameters
• aws_ssm_sessions
• aws_ssmincidents_incident_findings
• aws_ssmincidents_incident_related_items
• aws_ssmincidents_incident_timeline_events
• aws_ssmincidents_incidents
• aws_ssmincidents_response_plans
• aws_ssoadmin_instances
• aws_ssoadmin_permission_set_account_assignments
• aws_ssoadmin_permission_set_customer_managed_policies
• aws_ssoadmin_permission_set_inline_policies
• aws_ssoadmin_permission_set_managed_policies
• aws_ssoadmin_permission_set_permissions_boundaries
• aws_ssoadmin_permission_sets
• aws_ssoadmin_trusted_token_issuers
• aws_stepfunctions_activities
• aws_stepfunctions_executions
• aws_stepfunctions_map_run_executions
• aws_stepfunctions_map_runs
• aws_stepfunctions_state_machines
• aws_support_case_communications
• aws_support_cases
• aws_support_services
• aws_support_severity_levels
• aws_support_trusted_advisor_check_results
• aws_support_trusted_advisor_check_summaries
• aws_support_trusted_advisor_checks
• aws_timestream_databases
• aws_timestream_tables
• aws_transfer_agreements
• aws_transfer_certificates
• aws_transfer_connectors
• aws_transfer_profiles
• aws_transfer_servers
• aws_transfer_users
• aws_transfer_workflows
• aws_waf_rule_groups
• aws_waf_rules
• aws_waf_subscribed_rule_groups
• aws_waf_web_acls
• aws_wafregional_rate_based_rules
• aws_wafregional_rule_groups
• aws_wafregional_rules
• aws_wafregional_web_acls
• aws_wafv2_ipsets
• aws_wafv2_managed_rule_groups
• aws_wafv2_regex_pattern_sets
• aws_wafv2_rule_groups
• aws_wafv2_web_acls
• aws_wellarchitected_lens_review_improvements
• aws_wellarchitected_lens_reviews
• aws_wellarchitected_lenses
• aws_wellarchitected_share_invitations
• aws_wellarchitected_workload_milestones
• aws_wellarchitected_workload_shares
• aws_wellarchitected_workloads
• aws_workspaces_connection_alias_permissions
• aws_workspaces_connection_aliases
• aws_workspaces_directories
• aws_workspaces_workspaces
• aws_xray_encryption_configs
• aws_xray_groups
• aws_xray_resource_policies
• aws_xray_sampling_rules

Find all public-facing load balancers

SELECT * FROM aws_elbv2_load_balancers WHERE scheme = 'internet-facing';

Find all unencrypted RDS instances

SELECT * FROM aws_rds_clusters WHERE storage_encrypted IS FALSE;

Find all S3 buckets that are permitted to be public

SELECT arn, region
FROM aws_s3_buckets
WHERE block_public_acls IS NOT TRUE
   OR block_public_policy IS NOT TRUE
   OR ignore_public_acls IS NOT TRUE
   OR restrict_public_buckets IS NOT TRUE

Breaking changes

1. Changing a primary key for a table
2. Changing the name of a table
3. Changing the permissions required to sync a resource

Preview features

Current Preview features

• All tables that are prefixed with aws_alpha_
• table_options feature