Buildkite Integration
Pipeline and step duration, agent queue wait times, and flaky test detection for Buildkite. Understand exactly where time is lost in your build pipeline and which tests are costing your team velocity.
How It Works
Connect via Buildkite Webhooks
Add the TigerOps webhook URL in your Buildkite organization notification settings. Enable build, job, and agent events. TigerOps begins ingesting pipeline and step events immediately after your next build.
Deploy the Agent Metrics Exporter
Run the tigerops-buildkite-exporter alongside your agent pools. It polls the Buildkite Agent REST API and exports queue depth, wait time percentiles, and running job counts as Prometheus metrics.
Add the TigerOps Plugin to Pipelines
Add the tigerops-buildkite-plugin to your pipeline steps for enriched telemetry — test result counts, artifact sizes, and custom duration annotations that go beyond what webhooks provide.
Enable Flaky Test Detection
TigerOps ingests JUnit XML artifacts from your Buildkite steps and computes per-test pass rates across builds. Tests that fail intermittently without consistent reproduction are surfaced on the flaky test dashboard.
What You Get Out of the Box
Pipeline & Step Duration Tracking
End-to-end pipeline duration and per-step timing with p50/p95/p99 breakdowns. TigerOps tracks duration trends across branches, detects regressions, and identifies which steps are on the critical path.
Agent Queue Wait Time Monitoring
Track how long jobs wait in each agent queue before starting. TigerOps alerts when queue wait times breach SLOs, correlates wait spikes with concurrency limits, and recommends agent pool sizing adjustments.
Flaky Test Detection & Ranking
TigerOps computes test flakiness scores from JUnit results across all builds. Tests with high flakiness scores are ranked and surfaced in a dedicated dashboard, so your team can prioritize the tests causing the most disruption.
Agent Pool Utilization Metrics
Monitor agent busy rate, idle rate, and concurrent job counts per queue. TigerOps provides heatmaps of agent utilization to identify under-provisioned queues and over-provisioned queues wasting compute.
Build Artifact Size Tracking
Track artifact sizes uploaded to Buildkite Artifacts over time. TigerOps alerts when build artifacts grow unexpectedly, which can indicate dependency bloat or misconfigured bundle outputs before they impact deploy times.
Deployment Change Event Correlation
When a Buildkite pipeline triggers a production deployment, TigerOps records the change event and correlates it with APM and infrastructure metric anomalies, surfacing deploy-induced regressions automatically.
Buildkite Pipeline with TigerOps Plugin
Add the TigerOps Buildkite plugin to your pipeline for enriched metrics and flaky test detection.
# .buildkite/pipeline.yml — TigerOps plugin integration
# Set TIGEROPS_API_KEY in your Buildkite organization secrets
steps:
- label: ":test_tube: Run Tests"
command: npm test -- --reporter junit --output-file test-results.xml
plugins:
- tigerops/tigerops#v1.2.0:
api_key: "${TIGEROPS_API_KEY}"
service: my-api
environment: "${BUILDKITE_BRANCH == 'main' ? 'production' : 'staging'}"
# Upload JUnit results for flaky test detection
junit_glob: "test-results.xml"
artifact_paths: "test-results.xml"
- label: ":rocket: Deploy"
command: ./scripts/deploy.sh
branches: "main"
plugins:
- tigerops/tigerops#v1.2.0:
api_key: "${TIGEROPS_API_KEY}"
service: my-api
environment: production
event_type: deployment
env:
TIGEROPS_API_KEY: "${TIGEROPS_API_KEY}"
# notify block fires on build completion (no plugin required)
notify:
- webhook: "https://ingest.atatus.net/api/v1/buildkite/events"
if: "build.state == 'failed' || build.state == 'passed'"Common Questions
Does TigerOps support Buildkite Pipelines running on self-hosted agents?
Yes. The Buildkite webhook integration works regardless of where agents run — on-premises, in your own cloud VPC, or on Buildkite-hosted agents. The agent metrics exporter can also run inside your private network and push metrics out to TigerOps.
How does TigerOps ingest JUnit test results from Buildkite?
The tigerops-buildkite-plugin automatically uploads JUnit XML artifacts from configured test steps to TigerOps. Alternatively, you can use the TigerOps API to POST JUnit results directly from your pipeline script after running tests.
Can TigerOps monitor Buildkite Test Analytics in addition to JUnit results?
Yes. TigerOps can ingest Buildkite Test Analytics API data alongside its own JUnit processing. This gives you both Buildkite native test analytics and TigerOps cross-pipeline flakiness correlation in a unified view.
What Buildkite agent queue metrics does TigerOps track?
TigerOps tracks jobs waiting count, jobs running count, agent count (idle vs. busy), average and p95 queue wait time, and scheduled-to-started latency — all broken down by queue name so you can optimize each agent pool independently.
How long does historical build data retention last in TigerOps?
TigerOps retains build event data and metrics according to your plan — 90 days on the standard plan and up to 1 year on the enterprise plan. Flaky test scores are computed on a rolling 30-day window by default, which is configurable per pipeline.
Stop Losing Velocity to Slow Pipelines and Flaky Tests
Agent queue analytics, step duration histograms, and flaky test rankings. Connect Buildkite in under 5 minutes.