AWS Kinesis Integration
Monitor shard iterator age, throughput metrics, and enhanced fan-out consumer health across your Kinesis data streams. Get predictive iterator age alerts and AI root cause analysis before consumers fall critically behind.
How It Works
Enable CloudWatch Metric Streams
Deploy the TigerOps CloudFormation template to your AWS account. It provisions a Kinesis Firehose delivery stream and a CloudWatch Metric Stream scoped to the AWS/Kinesis namespace, forwarding all shard and stream metrics in real time.
Configure IAM & Delivery
The CloudFormation stack creates the required IAM roles with least-privilege permissions. Metrics are delivered via Firehose to TigerOps in OpenTelemetry format with sub-minute latency — no polling, no scrape intervals.
Set Shard-Level Alert Thresholds
Define alert thresholds for GetRecords.IteratorAgeMilliseconds and WriteProvisionedThroughputExceeded per stream or per shard. TigerOps predicts iterator age growth before consumers fall behind.
Correlate with Producers & Consumers
TigerOps automatically links Kinesis iterator age spikes with Lambda consumer errors, Firehose delivery failures, or upstream producer PutRecord throttling — giving full context in a single incident view.
What You Get Out of the Box
Shard Iterator Age Tracking
Per-stream and per-shard GetRecords.IteratorAgeMilliseconds with trend analysis. TigerOps alerts when iterator age growth rate predicts a consumer falling critically behind before it causes data loss.
Throughput & Throttle Monitoring
Track WriteProvisionedThroughputExceeded, ReadProvisionedThroughputExceeded, and PutRecords success rates per stream. Identify which shards are hot and when automatic scaling is needed.
Enhanced Fan-Out Consumer Metrics
Monitor SubscribeToShard.RateExceeded, SubscribeToShardEvent.MillisBehindLatest, and SubscribeToShardEvent.Bytes per registered consumer. Get individual consumer health at a glance.
PutRecord Latency Percentiles
p50, p95, and p99 PutRecord and PutRecords latency by stream and shard. Detect pathological batching patterns and correlate producer latency spikes with downstream consumer lag.
Shard Count & Scaling Events
Track OpenShardCount over time and correlate resharding events with throughput anomalies. TigerOps records when splits and merges occur and their impact on iterator continuity.
AI Root Cause Analysis
When iterator age spikes, TigerOps AI cross-references Lambda concurrent execution limits, DynamoDB Streams consumer errors, and producer PutRecord failure rates to surface the root cause instantly.
CloudFormation Stack for Kinesis Metric Streams
Deploy the TigerOps CloudFormation template to start streaming Kinesis metrics in under 5 minutes.
# Deploy via AWS CLI:
# aws cloudformation deploy \
# --template-file tigerops-kinesis-stack.yaml \
# --stack-name tigerops-kinesis-monitoring \
# --capabilities CAPABILITY_IAM \
# --parameter-overrides TigerOpsApiKey=${TIGEROPS_API_KEY}
AWSTemplateFormatVersion: '2010-09-09'
Description: TigerOps - Kinesis CloudWatch Metric Streams
Parameters:
TigerOpsApiKey:
Type: String
NoEcho: true
Description: Your TigerOps API key
Resources:
TigerOpsFirehoseRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: firehose.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: TigerOpsFirehosePolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:PutLogEvents
Resource: '*'
TigerOpsDeliveryStream:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamType: DirectPut
HttpEndpointDestinationConfiguration:
EndpointConfiguration:
Url: https://ingest.atatus.net/api/v1/cloudwatch-metrics
AccessKey: !Ref TigerOpsApiKey
RequestConfiguration:
ContentEncoding: GZIP
BufferingHints:
IntervalInSeconds: 60
SizeInMBs: 4
RetryOptions:
DurationInSeconds: 300
TigerOpsMetricStream:
Type: AWS::CloudWatch::MetricStream
Properties:
Name: tigerops-kinesis-stream
FirehoseArn: !GetAtt TigerOpsDeliveryStream.Arn
RoleArn: !GetAtt TigerOpsFirehoseRole.Arn
OutputFormat: opentelemetry1.0
IncludeFilters:
- Namespace: AWS/Kinesis
- Namespace: AWS/Firehose
StatisticsConfigurations:
- AdditionalStatistics:
- p50
- p95
- p99
IncludeMetrics:
- Namespace: AWS/Kinesis
MetricName: GetRecords.IteratorAgeMilliseconds
- Namespace: AWS/Kinesis
MetricName: PutRecord.SuccessCommon Questions
How does TigerOps collect Kinesis metrics without an agent?
TigerOps uses CloudWatch Metric Streams with an AWS Firehose delivery stream. The CloudFormation template you deploy creates the stream scoped to the AWS/Kinesis namespace and forwards metrics in near real time via HTTP to the TigerOps ingest endpoint — no polling, no agents required on your instances.
Can TigerOps monitor enhanced fan-out (EFO) consumers separately?
Yes. Enhanced fan-out consumers expose their own CloudWatch metrics under the Kinesis namespace with a ConsumerName dimension. TigerOps ingests these automatically and lets you build per-consumer dashboards and alerts on MillisBehindLatest independently from standard GetRecords consumers.
What is the latency from a Kinesis metric change to an alert firing?
CloudWatch Metric Streams deliver metrics with approximately 2–3 minutes of end-to-end latency. TigerOps evaluates alert conditions as metrics arrive, so your alert fires within roughly 3 minutes of the underlying condition occurring — significantly faster than polling-based CloudWatch alarms.
Does TigerOps support Kinesis Data Firehose metrics as well?
Yes. The TigerOps CloudFormation template can be configured to include the AWS/Firehose namespace alongside AWS/Kinesis. This gives you DeliveryToS3.Success rates, DataFreshness, and IncomingBytes for your Firehose delivery streams in the same dashboard.
How do I monitor Kinesis consumers running in Lambda?
TigerOps correlates Kinesis iterator age metrics with Lambda function metrics (errors, duration, concurrency, throttles) using the function name and event source mapping. When a Lambda consumer causes rising iterator age, TigerOps surfaces the Lambda error logs and traces in the same incident context.
Stop Discovering Kinesis Iterator Age After the Fact
Predictive iterator age alerts, shard throughput monitoring, and AI root cause analysis. Deploy in 5 minutes with CloudFormation.