AI Observability with OpenTelemetry, Langfuse, and Azure AI Foundry

As generative AI applications transition from simple scripts to complex, multi-step agentic workflows, visibility becomes critical. You need to know exactly what prompt was sent, how many tokens were used, how long the model took to respond, and how much it cost. In this tutorial, we will set up OpenTelemetry (OTel) to instrument an AI workflow built in TypeScript. We'll use the gpt-5.4-mini model deployed via Azure AI Foundry, and route our telemetry data to a local, Dockerized instance of Langfuse backed by PostgreSQL.

Steps

The Local Infrastructure (Langfuse + Postgres)

Instead of relying on a managed cloud service for testing, we will spin up Langfuse and PostgreSQL locally. To make this seamless, we will use Langfuse’s Headless Initialization feature. This automatically provisions an organization, a project, and API keys the moment the container starts, so you don't have to click through a UI to generate credentials.

Create a docker-compose.yml file in a new directory:

version: "3.8"
 
name: langfuse-foundry
services:
  langfuse-server:
    image: langfuse/langfuse:3
    depends_on:
      - db
      - clickhouse
      - minio
      - redis
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/langfuse
      - NEXTAUTH_URL=http://localhost:3000
      - NEXTAUTH_SECRET=supersecret_nextauth_key
      - SALT=supersecret_salt
 
      # V3 Storage & Queue Dependencies
      - CLICKHOUSE_URL=http://clickhouse:8123
      - CLICKHOUSE_MIGRATION_URL=clickhouse://default:clickhouse123@clickhouse:9000
      - CLICKHOUSE_USER=default
      - CLICKHOUSE_PASSWORD=clickhouse123
      - CLICKHOUSE_CLUSTER_ENABLED=false
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_AUTH=
      - LANGFUSE_S3_EVENT_UPLOAD_BUCKET=langfuse
      - LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT=http://minio:9000
      - LANGFUSE_S3_EVENT_UPLOAD_REGION=us-east-1
      - LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID=minioadmin
      - LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=minioadmin
      - LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE=true
 
      # Headless Initialization
      - LANGFUSE_INIT_ORG_ID=local-org
      - LANGFUSE_INIT_PROJECT_ID=local-project
      - LANGFUSE_INIT_PROJECT_PUBLIC_KEY=pk-lf-1234567890
      - LANGFUSE_INIT_PROJECT_SECRET_KEY=sk-lf-1234567890
      - LANGFUSE_INIT_USER_EMAIL=admin@localhost.com
      - LANGFUSE_INIT_USER_PASSWORD=password123
 
  langfuse-worker:
    image: langfuse/langfuse-worker:3
    depends_on:
      - db
      - clickhouse
      - minio
      - redis
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/langfuse
      - SALT=supersecret_salt
      - CLICKHOUSE_URL=http://clickhouse:8123
      - CLICKHOUSE_MIGRATION_URL=clickhouse://default:clickhouse123@clickhouse:9000
      - CLICKHOUSE_USER=default
      - CLICKHOUSE_PASSWORD=clickhouse123
      - CLICKHOUSE_CLUSTER_ENABLED=false
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_AUTH=
      - LANGFUSE_S3_EVENT_UPLOAD_BUCKET=langfuse
      - LANGFUSE_S3_EVENT_UPLOAD_ENDPOINT=http://minio:9000
      - LANGFUSE_S3_EVENT_UPLOAD_REGION=us-east-1
      - LANGFUSE_S3_EVENT_UPLOAD_ACCESS_KEY_ID=minioadmin
      - LANGFUSE_S3_EVENT_UPLOAD_SECRET_ACCESS_KEY=minioadmin
      - LANGFUSE_S3_EVENT_UPLOAD_FORCE_PATH_STYLE=true
 
  db:
    image: postgres:15
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=langfuse
    ports:
      - "5432:5432"
    volumes:
      - <YOUR_LOCAL_PATH>:/var/lib/postgresql/data
 
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    ports:
      - "8123:8123"
      - "9001:9000"
    environment:
      - CLICKHOUSE_DB=default
      - CLICKHOUSE_USER=default
      - CLICKHOUSE_PASSWORD=clickhouse123
    volumes:
      - <YOUR_LOCAL_PATH>:/var/lib/clickhouse
 
  redis:
    image: redis:7
    ports:
      - "6379:6379"
 
  minio:
    image: minio/minio
    # Creates the 'langfuse' bucket automatically on startup
    entrypoint:
      [
        "sh",
        "-c",
        "mkdir -p /data/langfuse && minio server /data --console-address ':9002'",
      ]
    ports:
      - "9090:9000"
      - "9091:9002"
    environment:
      - MINIO_ROOT_USER=minioadmin
      - MINIO_ROOT_PASSWORD=minioadmin
    volumes:
      - <YOUR_LOCAL_PATH>:/data

Note:You can change <YOUR_LOCAL_PATH> to point the volumes to any path you want (IF that is what you want to do).

Now you can start your infra by running: docker compose up -d.

Wait a minute or two for the database to initialize and the migrations to run. You can verify it is working by navigating to http://localhost:3000 and logging in with admin@localhost.com and password123.

Set Up the TypeScript Project

Initialize a new Node.js project and install the required dependencies. We will use the standard openai package (which fully supports Azure AI Foundry) and @traceloop/node-server-sdk, which is the open-source OpenLLMetry standard for automatically injecting GenAI semantic conventions into OpenTelemetry spans.

# Install dependencies
npm install dotenv openai @traceloop/node-server-sdk
npm install -D typescript @types/node ts-node

And don't forget to init your TypeScript npx tsc --init and you can modify the cofig. Mine is for this example:

{
  "compilerOptions": {
    "target": "es2022",
    "module": "CommonJS",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "verbatimModuleSyntax": false,
    "strict": true,
    "skipLibCheck": true
  }
}

Configure Environment Variables

OpenTelemetry securely exports data using OTLP (OpenTelemetry Protocol). Langfuse’s OTLP endpoint requires basic authentication using your public and secret keys encoded in Base64. Our keys from the docker-compose.yml are pk-lf-1234567890 and sk-lf-1234567890. Formatted as public_key:secret_key, the Base64 encoding of that string is cGstbGYtMTIzNDU2Nzg5MDpzay1sZi0xMjM0NTY3ODkw. Create a .env file in the root of your project:

# 1. Azure AI Foundry Configuration
AZURE_OPENAI_ENDPOINT="https://<YOUR_RESOURCE_NAME>.openai.azure.com/"
AZURE_OPENAI_API_KEY="<YOUR_AZURE_API_KEY>"
 
# 2. OpenTelemetry Exporter Configuration
TRACELOOP_BASE_URL="http://localhost:3000/api/public/otel"
TRACELOOP_HEADERS="Authorization=Basic cGstbGYtMTIzNDU2Nzg5MDpzay1sZi0xMjM0NTY3ODkw"

(Make sure to replace the Azure endpoint and API key with your actual Azure AI Foundry credentials).

Write the Instrumentated Code

Create an index.ts file. Notice how we must initialize Traceloop before importing the Azure OpenAI client. This allows the SDK to properly wrap and auto-instrument the network calls.

import "dotenv/config";
import * as traceloop from "@traceloop/node-server-sdk";
 
// Init OpenTelemetry tracing
// Disable batching so traces appear instantly in our local Langfuse dashboard
traceloop.initialize({
  appName: "azure-foundry-agent",
  disableBatch: true,
});
 
// Import OpenAI after init traceloop
import { AzureOpenAI } from "openai";
 
// Init Azure OpenAI client
const client = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT,
  apiKey: process.env.AZURE_OPENAI_API_KEY,
  apiVersion: "2024-02-15-preview",
  deployment: "gpt-5.4-mini",
});
 
async function main() {
  console.log("Starting agent workflow...");
 
  // Wrap the logic in an OTel Span (Workflow)
  // This groups all underlying LLM calls into a single traceable transaction
  await traceloop.withWorkflow({ name: "typescript_fun_fact" }, async () => {
    try {
      const response = await client.chat.completions.create({
        model: "gpt-5.4-mini",
        messages: [
          {
            role: "user",
            content: "Tell me a brief, on-sentence fun fact about TypeScript.",
          },
        ],
        temperature: 0.7,
      });
 
      console.log("\Model Response:");
      console.log(response.choices[0].message.content);
    } catch (error) {
      console.error("Error calling Azure OpenAI:", error);
    }
  });
 
  console.log("\nWorkflow complete. Check Langfuse for traces.");
}
 
main();

Run and Validate

Run your script using ts-node npx ts-node index.ts.

You should see something like this:

Traceloop exporting traces to http://localhost:3000/api/public/otel
Starting agent workflow...
Model Response:
TypeScript was originally created at Microsoft and is often called a “superset of JavaScript” because every valid JavaScript program is also valid TypeScript.
 
Workflow complete. Check Langfuse for traces.

You can now go to your browser and open http://localhost:3000 and check everything you need :).

You now have a fully standard, vendor-agnostic OpenTelemetry pipeline. If you ever decide to switch from Langfuse to another OTel-compatible backend (like Datadog or LangSmith), you simply change the .env variables—your TypeScript code doesn't need to change at all!