Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
Langtrace is an open source observability software which lets you capture, debug and analyze traces and metrics from all your applications that leverages LLM APIs, Vector Databases and LLM based Frameworks.
The traces generated by Langtrace adhere to Open Telemetry Standards(OTEL). We are developing semantic conventions for the traces generated by this project. You can checkout the current definitions in this repository. Note: This is an ongoing development and we encourage you to get involved and welcome your feedback.
To use the managed SaaS version of Langtrace, follow the steps below:
npm i @langtrase/typescript-sdk
import * as Langtrace from '@langtrase/typescript-sdk' // Must precede any llm module imports
Langtrace.init({ api_key: <your_api_key> })
OR
import * as Langtrace from "@langtrase/typescript-sdk"; // Must precede any llm module imports
LangTrace.init(); // LANGTRACE_API_KEY as an ENVIRONMENT variable
pip install langtrace-python-sdk
from langtrace_python_sdk import langtrace
langtrace.init(api_key=<your_api_key>)
OR
from langtrace_python_sdk import langtrace
langtrace.init() # LANGTRACE_API_KEY as an ENVIRONMENT variable
To run the Langtrace locally, you have to run three services:
[!IMPORTANT] Checkout documentation for various deployment options and configurations.
Requirements:
Feel free to modify the .env
file to suit your needs.
docker compose up
The application will be available at http://localhost:3000
.
[!NOTE]
if you wish to build the docker image locally and use it, run the docker compose up command with the--build
flag.
[!TIP] to manually pull the docker image from docker hub, run the following command:
docker pull scale3labs/langtrace-client:latest
To delete containers and volumes
docker compose down -v
-v
flag is used to delete volumes
The following are some commands that may come handy during setup and debugging.
docker exec -it langtrace-postgres psql --dbname=langtrace --username=ltuser --password
docker exec -it langtrace-clickhouse clickhouse-client
docker logs langtrace
If you want to follow the logs
docker logs -f langtrace
docker exec -it langtrace npm run create-tables
docker exec -it langtrace npm run create-tables
docker exec -it langtrace npm run create-tables
Certain docker compose schema used in this project are only supported in newer versions of docker-compose.
Either you update the docker compose version OR remove the depends_on property that is causing the error.
Langtrace automatically captures traces from the following vendors:
Vendor | Type | Typescript SDK | Python SDK |
---|---|---|---|
OpenAI | LLM | :white_check_mark: | :white_check_mark: |
Anthropic | LLM | :white_check_mark: | :white_check_mark: |
Azure OpenAI | LLM | :white_check_mark: | :white_check_mark: |
Cohere | LLM | :white_check_mark: | :white_check_mark: |
Groq | LLM | :x: | :white_check_mark: |
Langchain | Framework | :x: | :white_check_mark: |
LlamaIndex | Framework | :white_check_mark: | :white_check_mark: |
Pinecone | Vector Database | :white_check_mark: | :white_check_mark: |
ChromaDB | Vector Database | :white_check_mark: | :white_check_mark: |
QDrant | Vector Database | :x: | :white_check_mark: |
We welcome contributions to this project. To get started, fork this repository and start developing. To get involved, join our Slack workspace.
To report security vulnerabilites, email us at [email protected]. You can read more on security here.
1. Can I self host and run Langtrace in my own cloud? Yes, you can absolutely do that. Follow the self hosting setup instructions laid out above.
2. What is the pricing for Langtrace cloud? Currently, we are not charging anything for Langtrace cloud and we are primarily looking for feedback so we can continue to improve the project. We will inform our users when we decide to monetize it.
3. What is the tech stack of Langtrace? Langtrace uses NextJS for the frontend and APIs. It uses PostgresDB as a metadata store and Clickhouse DB for storing spans, metrics, logs and traces.
4. Can I contribute to this project? Absolutely! We love developers and welcome contributions. Get involved early by joining our slack workspace.
5. What skillset is required to contribute to this project? Programming Languages: Typescript and Python. Framework knowledge: NextJS. Database: Postgres and Prisma ORM. Nice to haves: Opentelemetry instrumentation framework, experience with distributed tracing.