Amazon Bedrock Monitoring in ObserveOps
Overview
The Amazon Bedrock integration with ObserveOps (formerly known as AIOps) collects operational telemetry from Amazon Bedrock, AWS's fully managed foundation model service. It monitors provisioned throughput utilization, guardrail invocations and interventions, inference profile states, and model invocation performance including latency, token consumption, and error rates.
These metrics help AI platform teams track model usage costs, detect throttling and errors, and ensure guardrails are operating as expected across generative AI workloads.
Prerequisites
- Amazon Bedrock is enabled in the target AWS account and region.
- The IAM role or user used for integration has read access to Bedrock resources (
bedrock:ListProvisionedModelThroughputs, bedrock:GetProvisionedModelThroughput, bedrock:ListGuardrails, bedrock:GetGuardrail, cloudwatch:GetMetricData). - Required Bedrock and CloudWatch API endpoints are reachable from ObserveOps.
- The AWS account is added in discovery with correct credentials and region configuration.
List of Supported KPIs
Provisioned Throughput
| Metric | Description | Type |
|---|
| aws.bedrock.provisioned.state | Current state of the provisioned throughput configuration. | String |
| aws.bedrock.provisioned.model.unit | Number of model units allocated in the provisioned throughput. | Count |
| aws.bedrock.provisioned.name | Name of the provisioned throughput configuration. | String |
| aws.bedrock.provisioned.arn | Amazon Resource Name of the provisioned throughput. | String |
| aws.bedrock.provisioned.model.arn | ARN of the foundation model associated with provisioned throughput. | String |
| aws.bedrock.provisioned.commitment.duration | Commitment duration of the provisioned throughput agreement. | String |
| aws.bedrock.provisioned.utilization.percent | Utilization of the provisioned throughput capacity as a percentage. | Percent |
Guardrails
| Metric | Description | Type |
|---|
| aws.bedrock.guardrails | Total number of guardrails configured in the account. | Count |
| aws.bedrock.guardrail | Identifier of an individual guardrail. | String |
| aws.bedrock.guardrail.state | Current state of the guardrail. | String |
| aws.bedrock.guardrail.version | Version of the guardrail configuration. | String |
| aws.bedrock.guardrail.invocations | Total number of times the guardrail was invoked. | Count |
| aws.bedrock.guardrail.interventions | Number of times the guardrail intervened on a request. | Count |
Inference Profiles
| Metric | Description | Type |
|---|
| aws.bedrock.inference.profile | Identifier of an inference profile. | String |
| aws.bedrock.inference.profile.id | Unique ID assigned to the inference profile. | String |
| aws.bedrock.inference.profile.state | Current state of the inference profile. | String |
| Metric | Description | Type |
|---|
| aws.bedrock.invocations | Total number of model invocations. | Count |
| aws.bedrock.invocation.client.errors | Number of invocations that failed due to client-side errors. | Count |
| aws.bedrock.invocation.server.errors | Number of invocations that failed due to server-side errors. | Count |
| aws.bedrock.invocation.throttles | Number of invocations throttled due to rate limits. | Count |
| aws.bedrock.invocation.latency.ms | Average end-to-end latency of model invocations in milliseconds. | Milliseconds |
| aws.bedrock.first.byte.latency.ms | Time from request submission to first response byte in milliseconds. | Milliseconds |
| aws.bedrock.token.input | Total number of input tokens processed across all invocations. | Count |
| aws.bedrock.token.output | Total number of output tokens generated across all invocations. | Count |
| aws.bedrock.legacy.model.invocations | Number of invocations using legacy foundation models. | Count |