In this blogpost you will learn how to build an open-source cloud asset inventory with CloudQuery and Grafana.
General architecture:
This is what you will get:
To try out the below dashboards (or build a custom one) you will need to connect the same PostgreSQL used by CloudQuery to Grafana as a datasource.
You can run out CloudQuery locally or in your cloud environment in a cron to update your PostgreSQL/RDS/Cloud SQL. See Quick Start here
You can try out some of our pre-made dashboards by importing them straight from our GitHub
One of the most common but a lot of times non-trivial tasks is to find a specific when the only identifier you have is one piece of information across accounts and regions. Just a few examples (as number of real-world scenarios is really indefinite ):
Some of those can be also solved by AWS Config but has the following limitations:
This is why we created open-source “basic inventory” grafana dashboards for 23 services that you are free to use, customize or build completely new ones (feel free to share back or suggest other):
Here is snippet from our EC2 dashboard:
Similar challenge exists in GCP though the situation is a bit better in some sense as you have a single view for each resource/asset type per project. Though, If you want to have a single view of all types of assets across multiple projects (which is very common), this would be clickops or bash magic again.
Here is an example of GCP Compute Asset inventory dashboard:
Some of that can be solved with the in-house GCP Cloud Asset Inventory but has similar limitations:
You can create your own security views and dashboards that you can then monitor and alert. Each company has its own security and compliance policies but we will share a basic one in this blog (more is coming…) we found useful.
Filterable dashboards by vpc, subnet, region including public, private ec2 instances.
Hopefully by now you are as excited as us for the future of open-source cloud asset inventory and we are looking to hear your feedback, either on GitHub or Discord. Also, feel free to contribute back or request additional Grafana dashboards here.