Introducing the new Splunk Destination Plugin

Our new Splunk Destination Plugin is the easiest way to sync data from any data source directly into your Splunk instance. Designed for cloud architects, security teams, and DevOps professionals, this plugin makes it easy to harness the full power of Splunk’s analytics by integrating real-time data from cloud environments, including AWS, Azure, GCP, and many others. Whether you’re monitoring cloud assets or tracking security threats, this integration enables you to centralize and visualize data like never before.

In this guide, we’ll walk through setting up the AWS Source Plugin with the Splunk Destination Plugin to create an AWS Cloud Asset Inventory Dashboard in Splunk. You’ll learn how to sync data from AWS to Splunk and build dynamic dashboards to track and analyze your cloud infrastructure efficiently.

Why Splunk + CloudQuery?

Integrating Splunk with CloudQuery significantly enhances your data analytics and security capabilities by providing deeper insights and real-time monitoring. CloudQuery brings in detailed cloud asset data that enriches Splunk’s analysis, helping contextualize incidents and reduce false positives. With CloudQuery’s connectors, you can build tailored security solutions like Cloud Asset Inventories, CSPM, CIEM, CNAPP, and FinOps tools, all customized to your needs. This integration allows you to detect new threats more effectively and unify data from multiple sources into a single view, enabling advanced threat detection, response, and compliance management.

CloudQuery’s flexible data collection framework and Splunk’s powerful querying and dashboard capabilities provide a centralized and real-time monitoring solution for your security and cloud infrastructure. By bringing your data to Splunk, you can proactively handle your security management through customizable dashboards and reports, enhancing visibility and control. Additionally, it enables the implementation of custom security policies, comprehensive policy enforcement, and cost optimization strategies, helping you manage your cloud environment more efficiently and securely.

• Enriched Incident Context: Detailed cloud data enhances incident analysis.
• Custom Security Solutions: Tailor asset inventories, CSPM, CIEM, and more.
• Unified View: Consolidate multi-cloud data for better monitoring.
• Real-Time Insights: Continuous monitoring for immediate threat detection.
• Custom Dashboards: Create insights tailored to your compliance and security needs.
• Cost Optimization: Integrated FinOps tools to optimize cloud spending.

This integration makes Splunk not just a data analysis tool but a comprehensive platform for security, compliance, and cloud management.

How to create an AWS Cloud Asset Inventory Dashboard in Splunk

Getting Started with the Splunk Destination Plugin

How to show S3 Buckets Per Region in Splunk

Now that you have loaded your AWS data into your Splunk instance with CloudQuery, you can create dashboards to display our AWS data.

How to Compare Resources Across Different Cloud Providers with Splunk

For this example, you will make a comparison of storage related resources across AWS and GCP using CloudQuery to fetch the data and Splunk for querying/visualization.

Which results in a new bar chart next to your GCP chart.

How to find resources that might need your attention?

Which results in these tables:

How to Find Non-Essential Cloud Resources

The CloudQuery Splunk Destination Plugin also allows you to get a personalized view of your data tailored to your needs. Let’s assume you’re after a more generic view of the situation. For example, you might want to see the total resources per cloud provider so you can try to find non-essential resources that you could potentially cut back on.

Taking into account:

If you followed the example above, you already have the bucket storage data for both AWS and GCP cloud providers so that they will be omitted in the next sync. To get additional resources, you will need to update the table value on your CloudQuery configuration files with these additional tables:

AWS:

tables: ['aws_ec2_instances', 'aws_rds_instances', 'aws_lambda_functions', 'aws_iam_users']

GCP:

tables:
  [
    'gcp_compute_instances',
    'gcp_sql_instances',
    'gcp_functionsv2_functions',
    'gcp_iam_service_accounts',
  ]

After syncing, you will need to add these searches to your Splunk dashboard:

GCP:

index="storage" | search cq_source_name=gcp AND cq_table_name IN (gcp_storage_buckets, gcp_compute_instances, gcp_sql_instances, gcp_functionsv2_functions, gcp_iam_service_accounts)
| stats count by cq_table_name
| sort count

AWS:

index="storage" | search cq_source_name=aws AND cq_table_name IN (aws_s3_buckets, aws_ec2_instances, aws_rds_instances, aws_lambda_functions,aws_iam_users)
| stats count by cq_table_name
| sort count

This results in the following view:

Summary

The Splunk Destination Plugin simplifies data integration from any source directly into your Splunk instance. Integrating detailed asset data from AWS and other cloud providers into Splunk allows you to monitor, visualize, and analyze your infrastructure more effectively.

This tutorial showed you how to sync your AWS cloud data into Splunk using CloudQuery, set up a Cloud Asset Inventory dashboard, and create custom visualizations like pie charts and bar charts to track your AWS S3 buckets and compare resources across different cloud providers. With this setup, you can now leverage Splunk’s querying and dashboard capabilities to maintain visibility and control over your cloud infrastructure, ensuring optimized performance and security.

Want to see CloudQuery in action? Schedule a demo with our team or check out the platform documentation to learn more.

Got feedback or suggestions? Join the CloudQuery community to connect with other users and experts, or message our team directly here if you have any questions.

FAQs

Q: What is the Splunk Destination Plugin for CloudQuery?

The Splunk Destination Plugin allows you to sync data from any CloudQuery source plugin, such as AWS, directly into your Splunk instance for real-time analysis and monitoring.

Q: Do I need to authenticate my AWS instance before using the plugin?

Yes, you need to authenticate to your AWS instance. Follow our authentication guide to ensure proper setup.

Q: Can I use the Splunk Destination Plugin with data sources other than AWS?

Absolutely. While this guide uses the AWS plugin, CloudQuery supports various data sources. You can easily configure other sources like GCP, Azure, and more with the Splunk plugin.

Q: What are the key benefits of integrating CloudQuery data into Splunk?

Integrating CloudQuery with Splunk provides enriched incident context, reduces false positives, and allows for centralized data aggregation from multiple cloud sources, enabling real-time monitoring and customized security solutions.