Multi Account Configuration Tutorial
The plugin supports discovery of AWS Accounts via AWS Organizations. This means that as Accounts get added or removed from your organization, it will be able to handle new or removed accounts without any configuration changes.
kind: source spec: name: aws registry: github path: cloudquery/aws version: "v18.1.0" tables: ['aws_s3_buckets'] destinations: ["postgresql"] spec: aws_debug: false org: admin_account: local_profile: "<NAMED_PROFILE>" member_role_name: cloudquery-ro regions: - '*'
Prerequisites for using AWS Org functionality:
- Have a role (or user) in an Admin account with the following access:
- Have a role in each member account that has a trust policy with a single principal. The default profile name is
OrganizationAccountAccessRoleis created by default (opens in a new tab) in AWS Accounts created as part of an AWS Organization. We do not recommend using the
OrganizationAccountAccessRoledue to the level of permissions typically granted to the role, but instead recommend for CloudQuery users to create their own IAM roles in each member account with the appropriate read-only permissions. We also recommend ensuring that the IAM roles and policies used for CloudQuery adhere to company security standards.
- Reference IAM assets and the CloudFormation templates for deployment in an AWS Organization for CloudQuery can be found here (opens in a new tab).
Configuring AWS Organization:
It is always necessary to specify a member role name:
org: member_role_name: cloudquery-ro
Sourcing credentials that have the necessary
organizationspermissions can be done in any of the following ways:
Source credentials from the default credential tool chain:
org: member_role_name: cloudquery-ro
Source credentials from a named profile in the shared configuration or credentials file
org: member_role_name: cloudquery-ro admin_account: local_profile: <Named-Profile>
Assume a role in admin account using credentials in the shared configuration or credentials file:
org: member_role_name: cloudquery-ro admin_account: local_profile: <Named-Profile> role_arn: arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME> # Optional. Specify the name of the session # role_session_name: "" # Optional. Specify the ExternalID if required for trust policy # external_id: ""
Optional. If the trust policy configured for the member accounts requires different credentials than you configured in the previous step, then you can specify the credentials to use in the
org: member_role_name: cloudquery-ro member_trusted_principal: local_profile: <Named-Profile-Member>
Optional. If you want to specify specific Organizational Units to fetch from you can add them to the
org: member_role_name: cloudquery-ro organization_units: - ou-<ID-1> - ou-<ID-2>
Child OUs will also be included. To skip a child OU or account, use the
org: member_role_name: cloudquery-ro organization_units: - ou-<ID-1> - ou-<ID-2> skip_organization_units: - ou-<ID-3> skip_member_accounts: - <ACCOUNT_ID>
Note that in AWS plugin versions before v9.0.0, child OUs were not traversed when specifying an OU, and
skip_member_accounts were not supported. These options are only available in v9.0.0 and above, and child OUs are now traversed by default.
Arguments for Org block
See AWS org configuration for more information on all the arguments in the
Multi Account: Specific Accounts
CloudQuery can fetch from multiple accounts in parallel by using AssumeRole (You will need to use credentials that can AssumeRole to all other specified accounts). Below is an example configuration:
accounts: - id: <AccountID_Alias_1> role_arn: <YOUR_ROLE_ARN_1> # Optional. Local Profile is the named profile in your shared configuration file (usually `~/.aws/config`) that you want to use for this specific account local_profile: <NAMED_PROFILE> # Optional. Specify the Role Session name role_session_name: "" - id: <AccountID_Alias_2> local_profile: provider # Optional. Role ARN we want to assume when accessing this account role_arn: <YOUR_ROLE_ARN_2>
Arguments for Accounts block
See AWS accounts configuration for more information on all the arguments in the