Langfuse

You add the Langfuse SDK or an integration to an existing application, and traces start flowing on the next request. A trace nests the spans that make up one agent run, so a retrieval step that returned the wrong chunk is visible next to the model call that then produced the wrong answer. From there you filter production traces to find a failure pattern, capture the offending inputs into a dataset, and run an experiment against that dataset to see whether a change helps. Evaluators run on production data or on experiment runs, and human annotations feed the same scores. Prompts live in Langfuse rather than in code, so you change one and deploy it with one click, or roll it back the same way. The playground lets you replay a real production input against a different model side by side before you commit to anything.

• •Hierarchical Tracing - Every LLM call, tool invocation and retrieval step captured with cost and latency, filterable by user, session, cost, latency or custom metadata
• •Evaluation - LLM-as-a-judge, heuristic scorers and human review, running either against production data or inside an experiment
• •Prompt Management - Prompts separated from code with one-click deployment and rollback, so changing a prompt is not a code release
• •Datasets and Experiments - Turn production traces and user feedback into versioned datasets, then run repeatable experiments against them
• •Playground - Test prompts on real production inputs and compare models side by side
• •Genuinely Self-Hostable - MIT core with maintained Docker Compose, Helm and Terraform paths for AWS, GCP and Azure, rather than a token open source repository

Teams past the prototype stage who have an agent in production and no idea why it fails the way it does, and who would rather own their trace data than send it to a vendor. The self-hosted path suits organisations with data residency or privacy constraints that rule out a hosted observability service. The prompt management piece is the strongest reason to choose Langfuse specifically: it is the part that lets non-engineers change behaviour safely. It is worth noting that this directory already lists Laminar for open source agent observability, so if all you need is tracing, that is a smaller tool for the same job and you should compare them rather than assume. Langfuse is also more than you need for a single-prompt feature with no evaluation story. And if the ClickHouse acquisition makes you uneasy about a long-term dependency, self-host it, which is exactly what the MIT licence is for.