LangChain Tracing

This quickstart shows how to instrument your LangChain application using OpenLLMetry and Scorecard for observability, debugging, and evaluation.

Looking for general tracing guidance? Check out the Tracing Quickstart for an overview of tracing concepts and alternative integration methods.

Steps

Install dependencies

Install the Traceloop SDK and the LangChain instrumentation package.

pip install traceloop-sdk opentelemetry-instrumentation-langchain

Set up environment variables

Configure the Traceloop SDK to send traces to Scorecard. Get your Scorecard API key from Settings.

export TRACELOOP_API_KEY="<your_scorecard_api_key>"
export TRACELOOP_BASE_URL="https://tracing.scorecard.io/otel"
export SCORECARD_PROJECT_ID="<your-project-id>"

Replace <your_scorecard_api_key> with your actual Scorecard API key (starts with ak_).

Initialize tracing

Initialize the Traceloop SDK with LangChain instrumentation before importing LangChain modules.

Import order matters! You must initialize Traceloop before importing any LangChain modules to ensure all calls are properly instrumented.

import os
from traceloop.sdk import Traceloop
from traceloop.sdk.instruments import Instruments

# Set scorecard.project_id to route traces to a specific project (defaults to oldest project)
Traceloop.init(
    disable_batch=True,
    instruments={Instruments.LANGCHAIN},
    resource_attributes={
        "scorecard.project_id": os.getenv("SCORECARD_PROJECT_ID")
    }
)

# Now import your LangChain modules
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

Run your LangChain application

With tracing initialized, run your LangChain application. All LLM calls, chain executions, and agent actions are automatically traced.Here’s a full example:

example.py

import os
from traceloop.sdk import Traceloop
from traceloop.sdk.instruments import Instruments

# Set scorecard.project_id to route traces to a specific project (defaults to oldest project)
Traceloop.init(
    disable_batch=True,
    instruments={Instruments.LANGCHAIN},
    resource_attributes={
        "scorecard.project_id": os.getenv("SCORECARD_PROJECT_ID")
    }
)

# Then import LangChain
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Create a simple chain
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}")
])

model = ChatOpenAI(model="gpt-4o-mini")
chain = prompt | model

# Run the chain - this will be traced
response = chain.invoke({"input": "What is the capital of France?"})
print(response.content)

You may see Failed to export batch warnings in the console. These can be safely ignored - your traces are still being captured and sent to Scorecard successfully.

View traces in Scorecard

Navigate to the Records page in Scorecard to see your LangChain traces.

It may take 1-2 minutes for traces to appear on the Records page.

Records page showing LangChain traces — Records page showing LangChain application traces

Click on any record to view the full trace details, including chain execution, LLM calls, and token usage.

Trace details view — Trace details with chain execution and LLM call data

What Gets Traced

OpenLLMetry automatically captures comprehensive telemetry from your LangChain applications:

Trace Data	Description
LLM Calls	Every LLM invocation with full prompt and completion
Chains	Chain executions with inputs, outputs, and intermediate steps
Agents	Agent reasoning steps, tool selections, and action outputs
Retrievers	Document retrieval operations and retrieved content
Token Usage	Input, output, and total token counts per LLM call
Errors	Any failures with full error context and stack traces

Introduction

Quickstarts

Core features

Advanced features

Governance, Risk, and Compliance

Steps

What Gets Traced

Next Steps

Records

Metrics

​Steps

​What Gets Traced

​Next Steps

Records

Metrics

Steps

What Gets Traced

Next Steps