Announcing Buildkite Integration

We have released a new Buildkite CloudQuery integration designed to help extract data from Buildkite. This integration allows your security and infrastructure teams to gain better insights into their build and deployment processes. Sync data from Buildkite into your preferred data warehouse, such as PostgreSQL, BigQuery, Snowflake, and much, much more.

What can you do with the Buildkite integration?

Analyze Pipeline Configurations: You can examine pipeline configurations, including the various stages, to ensure specific security and quality checks are being run.

Track Agent Activity: Monitor agent activity, find underutilized agents, and plan your capacity.

Understand Build Patterns: With Buildkite data in your data warehouse, you can identify build trends and optimize your pipelines.

Combine with other CloudQuery data: Integrate your Buildkite data with other CloudQuery integrations to get a more holistic view of your operations; correlate build failures with security scan results using the Snyk integration or test failures and incidents from PagerDuty.

Let’s walk through some specific scenarios on what you can do with the Buildkite integration.

Show Buildkite builds by status and day

This query analyzes Buildkite builds by grouping them by their status (state) and the day they were completed (finished_at) by:

Identifying Build Status: Counts the number of builds per status (e.g., passed, failed) for each day.

Visualizing Daily Trends: Aggregates data to highlight how builds are distributed across days.

Pipeline Insights: Includes pipeline repository information for potential filtering or extended analysis.

with builds_by_pipeline as (
  select pipeline->>'id' as pipeline_id, finished_at, state, buildkite_pipelines.repository from buildkite_builds
  join buildkite_pipelines on buildkite_pipelines.id = pipeline->>'id'
)
select date_trunc('day', finished_at) as day, state, count(*) from builds_by_pipeline
group by day, state

Show the repositories with most failed builds last month

This query identifies the repositories with the most failed builds in the last 30 days by:

Spotting Problematic Repositories: Highlights repositories with frequent build failures to prioritize investigation and fixes.

Monitoring Recent Trends: Focuses on failures within the last month to address current issues.

Pipeline-Level Analysis: Links failures to specific repositories for better accountability.

with builds_by_pipeline as (
  select pipeline->>'id' as pipeline_id, finished_at, state, buildkite_pipelines.repository from buildkite_builds
  join buildkite_pipelines on buildkite_pipelines.id = pipeline->>'id'
)
select repository, count(*) from builds_by_pipeline
where state = 'failed' and finished_at > current_timestamp - interval '30 day'
group by repository

Wrap up

CloudQuery’s new Buildkite integration allows teams to sync Buildkite data into their preferred data warehouses, enabling deeper insights into CI/CD pipelines. With this integration, you can analyze pipeline configurations, track agent activity, identify build patterns, and combine Buildkite data with other sources to enhance operational visibility.
To get started syncing Buildkite data, Start managing your CI/CD pipelines more effectively today. Contact us to start using CloudQuery today. You can also read the Buildkite source plugin documentation for instructions.

Have questions or need help? Join our growing CloudQuery Developer Community, where you can connect with fellow developers, share insights, and access a wealth of knowledge. Whether you’re exploring new use cases or solving tricky challenges, the community is here to support you.

Finally, let us know how your team is tackling cloud security! Share your thoughts and use cases with us on LinkedIn, X, or in our Community Forum.