AWS Lambda Implementation Guide

Overview

Use this path to make AWS Lambda functions operationally visible in Tsuga. Start with native AWS metrics, add the Lambda layer for logs and application metrics, then instrument traces and propagation. The layer runs a Collector extension inside each execution environment. Do not install a host agent or expose a long-lived receiver.

Level 1: CloudWatch Metric Stream ──> Firehose ──> Tsuga Metrics

Level 2: Lambda runtime + stdout ──Telemetry API──> Lambda Collector extension ──> Tsuga Logs
                                              └──OTLP metrics──────────────────────> Tsuga Metrics

Level 3: Handler SDK / auto-instrumentation ──OTLP traces──> Lambda Collector extension ──> Tsuga Traces

Level 3 reuses application-instrumentation practices for resource identity, span-metric dimensions, and service-level monitors and SLOs. Level 2 requires structured logging and trace correlation. A Lambda execution environment has no host agent or long-lived Collector, so use the collection routes below.

The Layer path can send logs, metrics, and traces directly to Tsuga. The levels separate what is enabled, not three different Collectors. At level 2, keep the trace pipeline available but do not call the function trace-instrumented until its handler creates useful spans and those spans propagate to its downstream work.

Before starting

Agree the production AWS accounts, regions, Lambda functions, environments, owners, and any functions that must not send data outside the account.
Create an ingestion key. Keep it in the platform secret store and out of source code, deployment packages, and checked-in Collector configuration.
Confirm outbound HTTPS from the Lambda function to the Tsuga intake endpoint. A VPC-attached function needs suitable NAT or another approved egress path.
Decide whether CloudWatch remains the system of record for raw Lambda logs. During rollout, retaining CloudWatch is normally useful. If the Lambda layer sends logs directly to Tsuga, do not also forward the same CloudWatch log group to Tsuga.
Set the service-identity convention before enabling more than a pilot function. All signals for one logical service must share service.name and deployment.environment.name.
Choose a layer ARN that exactly matches the runtime, AWS region, and function architecture. Follow the upstream layer instructions for the language-specific wrapper or bootstrap setting.

References: OpenTelemetry language SDKs, resource semantic conventions, and context propagation.

Service identity for Lambda

Lambda function names are AWS infrastructure identity. They are not automatically the right service boundary. Choose one of the following models deliberately. Never encode an environment, mutable alias, or request-specific value in service.name.

Model	Use it when	Advantages	Trade-offs
One function = one service	A function is independently owned, deployed, alerted on, and meaningful to operate by itself.	Clear per-function service health, ownership, SLOs, and dependencies.	A workflow made of many small functions creates a noisy Services list and fragments the customer journey.
Several functions = one bounded service	Functions jointly implement one product capability and are owned/released as one operational unit.	One service view for the capability. Service-level traffic, errors, latency, and deployments stay together.	A function-level regression can be less obvious unless dashboards and queries group by the Lambda function attribute.

Use the second model by default for functions that form one bounded service. Keep one function per service when it is genuinely independent. In either model, preserve the automatic Lambda resource attributes, especially the function name (faas.name), to distinguish execution units. Do not make a unique service.name from a function invocation, alias, or log stream.

Set deployment-specific identity with environment variables:

OTEL_SERVICE_NAME=<LOGICAL_SERVICE_NAME>
OTEL_RESOURCE_ATTRIBUTES=service.namespace=<TEAM_OR_PRODUCT>,service.version=<IMMUTABLE_BUILD_VERSION>,deployment.environment.name=<ENVIRONMENT>

Use an immutable build identifier for service.version, such as the commit SHA that produced the deployment. A Lambda published version is also suitable only when it identifies that exact build. $LATEST and mutable aliases are not version identity.

References: Resource semantic conventions and OpenTelemetry semantic conventions.

Level 1 - AWS-native metrics

Start with the native AWS view. Stream the AWS/Lambda CloudWatch namespace to Tsuga with a CloudWatch Metric Stream in OpenTelemetry 1.0 format, a Firehose delivery stream, and an S3 failed-delivery backup. Restrict the metric stream to the namespaces that are in scope. The Lambda-only starting point is AWS/Lambda.

AWS/Lambda CloudWatch metrics ──> CloudWatch Metric Stream (OTel 1.0)
                                     │
                                     ▼
                         Firehose HTTP delivery stream ──> Tsuga
                                     │
                                     └──> S3, failed deliveries only

This level needs no application code, Lambda layer, or execution-role change. It establishes the operational baseline:

Invocations, errors, duration, throttles, and concurrent executions.
Dead-letter, destination, and asynchronous-invocation signals where those features are used.
Provisioned-concurrency and iterator-age signals where the function’s trigger model exposes them.

Build the first Lambda dashboard and monitors from these native metrics. Treat throttles, sustained errors, and unexpected duration growth as starting candidates, but agree traffic floors, windows, owners, and routing before paging. An error count without invocation context is not enough for a reliable error-rate alert.

Validate the path in order: CloudWatch Metric Stream targets the intended Firehose stream, Firehose reports deliveries or retains failures in its S3 backup, and recent Lambda metrics are visible in Tsuga with the expected AWS dimensions.

Exit criteria: a known production function has recent AWS-native invocation, error, duration, and capacity metrics in Tsuga. Failed delivery is recoverable from S3. The team can identify an unhealthy or throttled function without changing its code.

Level 2 - Lambda layer, metrics, and logs

Add the upstream OpenTelemetry Lambda layer and configure its embedded Collector extension. The extension receives Lambda platform and function logs through the AWS Telemetry API, then exports them directly to Tsuga. It also receives OTLP metrics from the function when the runtime or application deliberately emits them.

Keep CloudWatch metrics for invocation, concurrency, throttles, and runtime health. Use OTLP metrics only for application measures such as domain outcomes or work-item duration. Avoid sending the same AWS-native metric through both routes.

Add the layer and Collector configuration

Attach the layer that matches the runtime, region, and architecture without replacing existing layers. Set OPENTELEMETRY_COLLECTOR_CONFIG_URI to a packaged file, S3 URI, or HTTP URI. A packaged file has the fewest runtime dependencies. Shared S3 or HTTP configuration adds cold-start fetches and access requirements.

Store the Tsuga endpoint and ingestion key in the function’s secret-management path and expose them as runtime environment variables. Use this Collector configuration as the common level-2 baseline:

receivers:
  telemetryapi:
  otlp:
    protocols:
      http:
      grpc:

processors:
  batch:
  decouple:

exporters:
  otlp_http/tsuga:
    endpoint: ${env:TSUGA_OTLP_ENDPOINT}
    headers:
      Authorization: Bearer ${env:TSUGA_API_KEY}

service:
  pipelines:
    logs:
      receivers: [telemetryapi]
      processors: [batch, decouple]
      exporters: [otlp_http/tsuga]
    metrics:
      receivers: [otlp]
      processors: [batch, decouple]
      exporters: [otlp_http/tsuga]
    traces:
      receivers: [otlp]
      processors: [batch, decouple]
      exporters: [otlp_http/tsuga]

Keep decouple: it lets the extension finish an export after the handler returns and before Lambda freezes the environment. Keep batch unless a tested latency requirement outweighs the reduction in outbound requests.

Make log ownership explicit

During rollout, use one Tsuga log route: the Telemetry API extension or CloudWatch Logs/Data Firehose. After validation, customers that no longer need CloudWatch may deny logs:CreateLogGroup, logs:CreateLogStream, and logs:PutLogEvents. Leave that deny out of the initial rollout.

Make application logs structured JSON where possible. Include a timestamp, level, stable message, and bounded diagnostic fields. Keep secrets, authorization headers, full payloads, and unbounded identifiers out of logs. At this stage, logs will have Lambda platform context and service identity. Trace IDs are expected only after level 3.

Validate with one invocation that emits a unique, harmless application log line. Confirm the record appears once in Tsuga with the agreed context.service.name, environment, Lambda function context, and timestamp. If custom metrics are in scope, emit one known metric and verify it separately after its collection interval.

Exit criteria: Lambda platform and application logs arrive directly in Tsuga exactly once. Their service and environment identity are correct. CloudWatch retention and Tsuga forwarding are intentional. Any custom metric is visibly distinct from the native AWS metric baseline.

Level 3 - Application traces

Instrument the function handler with the OpenTelemetry Lambda layer’s language-specific auto-instrumentation path, then add manual spans only around meaningful business work that automatic instrumentation cannot see. Capture each invocation, its downstream dependencies, its failures, and its user-visible latency.

Set the runtime-specific wrapper or bootstrap configuration exactly as documented by the upstream OpenTelemetry Lambda layer. It must load before the handler and framework initialize. Do not create a second global SDK provider if the Lambda auto-instrumentation already owns one.

Use the following progression:

Prove one root invocation span. Invoke the function through its normal trigger and verify its handler/root span has the agreed service and environment identity.
Verify automatic dependency spans. Check AWS SDK, HTTP, database, and supported framework calls. Do not add manual spans around work already represented by a useful automatic span.
Add small, purposeful manual spans. Use them for business operations such as validate-order, calculate-price, or a custom external integration. Name spans after the operation, not request IDs or raw payloads.
Propagate context across boundaries. Set OTEL_PROPAGATORS=tracecontext,baggage when the runtime supports environment-based propagation. Verify API Gateway/HTTP and synchronous downstream calls share one trace ID. For SQS, EventBridge, SNS, and other delayed delivery, create a consumer span linked to the producer context rather than making later work a child that inflates request latency.
Correlate logs. Configure the runtime logger bridge or structured logger to emit the active trace_id and span_id. The Lambda Collector preserves them, but cannot recreate context that the application did not emit.

Avoid recording sensitive values in span names or attributes. Keep dimensions bounded. Use route, operation, AWS service, outcome, and status. Do not use user IDs, full ARNs containing tenant input, payloads, request IDs, or message bodies as metric dimensions.

Validate a normal invocation and a controlled failure:

Open the trace in Tsuga and confirm a meaningful root span, service identity, and child spans.
Confirm a synchronous downstream call shares the trace ID.
For an asynchronous path, confirm the producer and consumer have the intended causal link rather than a misleading parent-child duration.
Open a log from an active span and confirm the exact trace and span IDs are present.
Confirm the log appears from the span detail and the trace opens from the log.

Exit criteria: known Lambda invocations produce searchable traces with meaningful dependency and business-operation spans. Agreed cross-service and messaging paths preserve causality. Logs pivot to their trace. Traces, logs, and metrics share service and environment identity.

Operational coverage after level 3

Once the signals are reliable, use native AWS metrics for Lambda platform failure and capacity coverage, and trace-derived service metrics for user-facing error rate and latency. Keep monitors scoped to the owner, environment, and function or logical service boundary chosen above. A practical first set is:

sustained Lambda throttles or concurrency exhaustion.
elevated Lambda error rate with an invocation floor.
high handler or end-to-end request latency for a critical operation.
failed asynchronous processing or growing iterator age where applicable.
no-data only when absence of traffic or telemetry is actionable.

Start SLOs from customer-visible outcomes, not a generic function success percentage. For example, an order-processing service may have a success-rate SLO spanning several functions, while a public authentication Lambda might justify its own service-level SLO.

Troubleshooting path

For native metrics, check the Metric Stream, Firehose delivery or backup, then Tsuga. For direct telemetry, check the layer and configuration URI, extension exporter errors, then a unique log or trace. Duplicate logs indicate two Tsuga routes. Missing traces with present logs usually indicate the wrapper or local OTLP export path.

If signal identity is wrong, first check OTEL_SERVICE_NAME, OTEL_RESOURCE_ATTRIBUTES, and whether the same values reach every function in the bounded service. Do not solve a service-name decision by renaming individual log records or individual spans after collection.

Completion criterion

A known invocation appears in Tsuga as a trace with the agreed service.name and environment, its logs arrive exactly once through the chosen route, and the same function’s AWS-native invocation, error, and duration metrics are visible for the same period.

Validate with the Tsuga CLI

# Confirm the function emitted spans through the Lambda layer.
tsuga traces search \
  --query "context.service.name:<service>" \
  --from -15m \
  --to now \
  --max-results 10

# Confirm one known invocation log arrived once with the expected identity.
tsuga logs search \
  --query "context.service.name:<service> <unique-log-token>" \
  --from -15m \
  --to now \
  --max-results 10

The first command returns recent spans with the agreed service.name. An empty result while logs arrive points at the language wrapper or bootstrap setting, not at the export path. The second returns the test log once. Two copies mean both the Telemetry API extension and a CloudWatch route are delivering it. For the AWS-native metric baseline from level 1, check the metric in Tsuga through Metrics rather than a CLI aggregation query, because the CloudWatch-derived names and dimensions differ from the source namespace.