Evaluation and observability

OpenTelemetry

Vendor-neutral traces, metrics, and logs.

made by
Cloud Native Computing Foundation

Magic Ship is one shop in Vancouver, BC, working remotely with clients worldwide. We are not a partner, reseller, or certified vendor of Cloud Native Computing Foundation - we just build with this.

What OpenTelemetry is

OpenTelemetry is a CNCF project defining an API, SDKs, and a wire protocol for traces, metrics, and logs. You instrument once against the SDK and export to whichever backend the collector is pointed at. Semantic conventions for model and agent spans exist but are still settling.

How we use it

We instrument services so one trace covers the HTTP request, the retrieval query, and every model call underneath it, with token counts and latency per step. Exporting through a collector means the backend can change without touching application code. When an agent is slow, the trace is what tells you whether the time went to the model, the tools, or the database, which is usually a surprise.

Where it is the wrong choice

The model-specific conventions are not stable yet, so span attributes need revisiting, and a purpose-built tool like Langfuse gives a better prompt-level view today. It also adds a collector to operate and a sampling policy to decide, which is real operational surface for a small project.

Building something on OpenTelemetry?

Send the problem rather than a job spec. You get an answer on scope, on fit, and on whetherOpenTelemetry is even the right call for it.

Start a project

OpenTelemetry and Cloud Native Computing Foundation are trademarks of their respective owners, used here to say what we work with.