Grace L. Christenbery's Blog

Three places to put a translator

2026-09-17T16:20:00-04:00

Every previous post here argued about what a translated span should say. This one is about a duller question that decides whether any of it gets used: where does the translator actually run?

The objection I keep hearing, and which is correct, goes like this. A platform team has settled on otelcol-contrib. It is in their base image, it is pinned, it is in their upgrade rota, and somebody has already argued about which processors are enabled. Asking them to maintain a custom Collector build so one family of spans can be renamed is asking for a permanent maintenance obligation in exchange for a dashboard. The answer is no, and it should be no.

So there are three places this can run, and the architecture page draws all three. Two are ordinary. The third is the one worth talking about.

A: in the path

Build processor/genaiinterlingua into your Collector distribution and put it between the receiver and the exporter. This is the version with the maintenance obligation, and it is the only one that gets to normalize spans the moment they arrive. It is a separate Go module, so building it in does not drag the CLI’s dependencies along, and the core has none.

B: beside the path

interlingua over captured OTLP JSON. This is how fixtures get built, how the conformance census is regenerated, and how you find out what a library actually emits rather than what its documentation claims. Five of seven captures found something the docs-derived version had wrong, which is the entire reason the fixtures are captured and not written.

C: not in the path at all

-emit ottl prints a config for transformprocessor. -emit schema-file prints a Telemetry Schema File for schemaprocessor. Both of those processors are already in the binary that platform team is running.

So the mapping runs, in their pipeline, on their spans, under their upgrade policy, and my code is not there. It ran once, on a laptop, to produce a text file they read before applying.

That is not a trick. It is what happens when the thing you have to distribute is a decision rather than an implementation.

Why it works

normalize.Span does not mutate a span. It returns a Result: attributes to set, attributes to remove, which dialect claimed the span, how big the winner’s margin was, and what the translation could not carry. A description of an edit, not the edit.

The Collector processor takes that description and applies it to pdata. The CLI applies the same description to OTLP JSON. The emitters render it as OTTL statements or as schema-file transformations for somebody else to apply. Three appliers, one decision, and a disagreement between them would be a bug rather than a configuration difference.

The same property is why the mapping can be golden tested at all. You cannot diff a mutation. You can diff a Result.

What C gives up

I would rather measure this than wave at it. A Telemetry Schema File carries less than the processor does, because the schema-file format can express a rename and cannot express reassembling a nested message document out of indexed keys. The size of that shortfall is measured on the conformance page, per dialect, rather than described as “some limitations apply.”

The honest summary is that row C is right for the renames, which is most of the volume, and row A is right when you need the parts of the mapping that are more than a rename. Knowing which one you are in is a question about your own telemetry, and the CLI in row B is how you answer it without deploying anything.

The part I actually care about

There is a general rule hiding in here, and it took me a while to see it.

A component that can only run as itself has to win a deployment argument before it can be evaluated. A component that can emit its decision as somebody else’s configuration gets evaluated first and deployed only if it earns it. The second kind is much harder to say no to, and it is also much harder to build, because it forces the decision to be data all the way through rather than a pile of if statements with side effects.

It is also why the differences between this and the upstream genainormalizer are going upstream rather than into a competing component. If the useful part is the decision and not the binary, then the decision belongs where the most people can run it, which is not here.

There is no model in here

2026-09-11T20:30:00-04:00

Three posts ago I argued that GenAI instrumentation libraries each spell the same facts differently, that the mappings should be data but not a new language, and that a translated span should say what the translation cost. Those were arguments. This one is the machine: what actually happens to a span, stage by stage, in pseudocode.

You can follow it without reading any Go, and there is no machine learning in it anywhere. That is worth saying at the top, because “figure out which library wrote this” sounds like a classification problem and gets built like one depressingly often. It is a counting problem. Everything below follows from that.

Two pages run the real thing in your browser if you would rather poke at it than read: the normalizer, the inspector for one span attribute at a time, and what translation costs for the measurements across six libraries.

The problem, in four lines

A model call produced 412 input tokens. Here are four of the ways it gets written down:

gen_ai.usage.prompt_tokens   = 412     OpenLLMetry
llm.token_count.prompt       = 412     OpenInference
ai.usage.promptTokens        = 412     Vercel AI SDK
gen_ai.usage.input_tokens    = 412     the conventions themselves

One fact, four spellings, and none of them can be summed with the others. So the job is to rewrite all of them into one vocabulary. To rewrite a span you first have to know which library wrote it, because nobody stamps their own name on it.

Stage 1: which library wrote this?

Each library gets to look at the span and say how much evidence it sees that only it would have written. Not “does this look like an LLM call” — everyone looks like an LLM call. Only the fingerprints.

Braintrust’s entire answer is two lines, and it is the clearest one:

score(span):
    if any attribute starts with "braintrust."   →  3
    otherwise                                   →  0

Most are a little richer. A whole namespace is worth 2 or 3; an individual telltale key is worth 1:

score_openllmetry(span):
    points = 0
    if span has any "traceloop.*"             points += 2
    for key in [gen_ai.usage.prompt_tokens,
                gen_ai.usage.completion_tokens,
                gen_ai.usage.total_tokens,
                llm.request.type]:
        if span has key                       points += 1
    if span has "gen_ai.prompt.*"
            or  "gen_ai.completion.*"         points += 1
    return points

Then you pick the winner:

detect(span):
    high, second, winner = 0, 0, none
    for each dialect:
        n = dialect.score(span)
        if n > high:     winner, high, second = dialect, n, high
        elif n > second: second = n

    confidence = high - second
    return winner, confidence

high - second is the whole confidence model. If one library scored 7 and the next best scored 0, the margin is 7. If they scored 5 and 3, the margin is 2 — same winner, much less comfortable. That integer gets written onto the span as interlingua.dialect.confidence, and a human or a query can use it to find the spans where the answer was close.

It is not a probability, and it cannot be turned into one. There is no denominator. The scores are unbounded evidence counts on scales that differ per library — one library’s 3 is a namespace, another’s 3 is three separate keys — so there is nothing to divide by. A percentage here would be a number invented to look like a measurement, which is why the page that draws these measurements does not publish one.

The highest score usually loses

Here is the part I did not expect when I built it. Five of the six dialects are named libraries. The sixth, raw, recognizes hand-rolled instrumentation by shape — it counts attributes the conventions themselves define, plus a table of common folk spellings:

score_raw(span):
    conformant = keys the conventions define        # worth 2 each
    folk       = keys in the folk-spelling table    # worth 1 each

    if conformant == 0 and folk < 2:  return 0      # a lone `model` attribute
    return conformant*2 + folk                      # is evidence of nothing

Because it counts every conventions-defined key at 2, raw scores enormously on spans that belong to somebody else. On a real LangChain span:

span	raw	openllmetry	winner	margin
`ChatOpenAI.chat`	32	3	openllmetry	3
`litellm_request`	24	3 (litellm 5)	litellm	2
`ai.generateText`	0	0 (vercel 7)	vercel	7
`chat gpt-4o-mini`	26	0	raw	0

raw out-scores the winner on eight of the fifteen spans in the corpus and wins none of the ones it competes for. It is marked as a fallback and excluded from the race entirely — consulted only when nothing that knows what it is looking at has claimed the span:

detect(span):
    contenders = dialects that are not fallbacks
    ...argmax over contenders, margin = high - second...

    if nobody claimed it:
        winner = best-scoring fallback
        confidence = 0            # it did not beat anything.
                                  # it was the only thing left.

The reason this belongs in detection rather than in the scoring is precise, and it is the nicest small design argument in the codebase: a fallback that scores on every GenAI span would not win those spans, but the points it took would come straight out of the winner’s margin. A positive identification would read as less confident merely because a fallback also exists. So the fallback tier is separate, and a span labeled raw with a confidence of 0 is telling the truth about how it was identified.

Note the last row. Margin 0 does not mean “coin flip”. It means “claimed from the fallback tier”.

Stage 2: translating it

Now the winner reads the span. About half of that work is a lookup table. A rule is three things:

Rule:
    field      the concept being produced        (not an attribute name)
    keys       source keys, in precedence order  (first one present wins)
    transform  what happens to the value on the way through

The field is deliberately not a name. gen_ai.usage.cache_write.input_tokens in the current draft of the conventions is spelled gen_ai.usage.cache_creation.input_tokens in the released version, and it is the same idea both times. So the middle of the pipeline traffics in concepts, and names are applied last, on the way out.

The transform is a closed set — six operations, applied in a fixed order:

transform:
    lower         lowercase it            "OpenAI" → "openai"
    cut_after     keep the part before a separator
    trim_suffix   strip a known suffix
    map           look it up in a table   (+ what to do if it's not in the table)
    divide        unit conversion         milliseconds → seconds
    to_list       wrap a scalar in a one-element list

Here is a real rule, and it uses three of them. The Vercel AI SDK writes its provider as <provider>.<api surface> — openai.chat, amazon-bedrock.messages, google.generative-ai — and only the first segment names the provider:

{ field: provider_name,
  keys:  [ai.model.provider, gen_ai.system],
  transform: lower, cut_after ".", map(vercel_providers) }

Every one of those six operations is a total function of one value. It cannot look at another attribute, iterate the span, or branch on anything but its own input. That constraint is the whole reason the table is a table: a rule that can be stated can also be printed, audited, and exported — into an OpenTelemetry Collector config that runs without this binary at all.

And it is why the other half of the mapping is not a table. Reassembling gen_ai.prompt.0.tool_calls.1.name-style indexed attributes into one nested document, or deciding what traceloop.entity.name means by looking at a sibling attribute, are not transforms of a value. They are readings of a span. Those stay as code, and the tool reports them as the part an export cannot carry. I tried making them data and that is the post about why it failed.

Stage 3: what could not be carried

This is the part other normalizers do not do, and it is the only reason this project is interesting.

When a rule cannot carry something, the reason is recorded on the span rather than dropped. Five reasons, and they are five genuinely different problems:

reason	what it means
`no_field`	the conventions have no concept for this, at any version
`unstructured`	a provider-shaped blob the translator declined to parse
`flattened`	indexed attributes collapsed into one field; per-index structure lost
`coerced`	the value survived but its type did not
`ambiguous`	it could have meant two different fields, so nothing was guessed

coerced has the best small example. The divide operation exists because the Vercel SDK reports milliseconds where the conventions specify seconds. If the value is not a number, it is not passed through unchanged:

divide(value, by):
    if value is not a number:
        return loss(coerced, "value is not a number")
    return value / by

Passing it through would put a millisecond count under an attribute the conventions define in seconds — wrong by exactly a thousand, and it looks fine. Refusing is the more useful answer.

(A pedantic detail I enjoy: it divides by 1000 rather than multiplying by 0.001. Over the first 200,000 millisecond values those two disagree in the last bit 26,651 times, because 0.001 has no exact float64 representation and 1000 does.)

The counter that counts what you forgot

Recording losses as you find them has an obvious failure: you only record the ones you thought to name. So after the rules run, there is a sweep. For every attribute under a namespace this dialect claimed as its own, that it neither read nor already reported, the sweep names it and records no_field.

Without that, lossy.count = 0 means “I recognized none of the things I know I drop,” which is very nearly the opposite of what any reader would take it for. A loss counter that only counts remembered losses is not a measurement. The shape-matching fallback had this right first and the five purpose-built dialects did not, which is an embarrassing way round for it to be.

What that measures, across six real dialects

Six dialects, nine span fixtures — seven captured from the libraries running, two hand-built — and thirty distinct fields. 105 source attributes have nowhere in the conventions to go:

dialect	fields carried (of 30)	attributes with no home
openllmetry	21	24
vercel	19	8
openinference	17	7
litellm	13	53
raw	13	6
braintrust	9	7

Ninety of the 105 are no_field — not a missing name, a missing concept.

LiteLLM’s 53 is not noise and it is not sloppiness. It is an entire cost model — gen_ai.cost.input_cost, output_cost, margin_percent, discount_amount — plus every user_api_key_* field recording who spent what. The conventions price nothing, at any version. That is a finding about the conventions, not about LiteLLM, and costing is a large part of why anyone puts a proxy in front of a model in the first place.

One more thing about the 105, because it is the question everyone asks: nothing was deleted. Under the default, every one of those attributes is still on the span under the name its library gave it. What it lacks is a gen_ai.* name, so a query written in the conventions’ vocabulary will not find it. The tool has a mode that removes them and it is not the default, because that is a choice somebody should make on purpose.

Stage 4: pinning a target costs something too

There is a second translation, and a second kind of loss. Concepts have to be written out under some specific version of the conventions, and the versions differ:

render(field, value, target):
    key = target.name_for(field)
    if no key:            return loss(no_attribute)   # this release has no such attribute
    if not target.accepts(field, value):
        return loss(no_value)                          # it has the attribute, not this value
    write key = value

The released v1.41.0 can express 27 of the 30 fields; the current draft can express all 30. Concretely, pinning the released version costs you gen_ai.prompt.version and both audio-token fields. And no_value bites on both targets: OpenLLMetry states an operation name and a provider name that are outside the conventions’ permitted value sets, so they are recorded as refused rather than written.

Both choices are defensible. Neither is correct. What is not defensible is making the choice silently, which is what a normalizer with a hardcoded table does — so the target is a flag.

What ends up on the span

The translation writes its own account of itself, in attributes you can group by:

interlingua.dialect            = openllmetry
interlingua.dialect.confidence = 7
interlingua.target             = v1.41.0
interlingua.lossy              = [gen_ai.completion.*, gen_ai.prompt.version, ...]
interlingua.lossy.count        = 4
interlingua.hops               = 1
interlingua.mapping            = 9f2c1ab4e07d5c83

Three choices there that took a while to get right.

lossy.count is written even when it is zero, because “this span lost nothing” and “this span was never normalized” are different facts and several backends store an array attribute as an opaque string you cannot count.

hops is an integer rather than a chain of translations. That is a concession to reality: Honeycomb types every array-valued attribute as a string column — including the conventions’ own gen_ai.input.messages — so a chain would arrive as a blob you cannot group by, filter into, or count. An integer you can alert on beats a structure you cannot query.

mapping is the one I would defend hardest. It is a digest over the rule tables themselves — every rule, every transform field including the ones left at their defaults, every target’s key map — and not a commit hash. A commit hash moves on every commit, including the ones that change nothing a span could possibly see, so it cannot answer “was this span translated under the same rules as that one?” The criterion is worth stating as a general rule, because it is the same one that makes me refuse a conformance percentage:

A number is a measurement only if it moves when the measured thing moves, and not otherwise.

The digest’s honest limit, stated rather than papered over: rewriting how a method reassembles messages, while changing nothing about which fields come out, leaves the digest where it was.

Go and look

Everything above is running in your browser, compiled from the same Go the command line and the Collector processor run, so the pages cannot disagree with the tool about what a span becomes. Nothing you paste is uploaded; there is no server to upload it to.

Normalize a span — paste one in, or pick a real capture, and watch it rewritten.
The inspector — one attribute at a time: which source key produced this value, what evidence the reading turned on, and what it could not carry.
What translation costs — the measurements above, drawn: the scoring race per span, and where all 105 losses go.

The source is here. Two of its documents are generated by the test suite rather than written, which is the only reason I trust the numbers in this post: the census is regenerated from the fixtures and CI fails when it and the fixtures disagree, and what capturing the real libraries found is the log of the five defects that turned up when I stopped writing fixtures from documentation and started recording what the libraries actually emit. Five of seven captures found something wrong. Every one of them was invisible to a completely green test suite.

What this is not

There is already a first-party component for this. processor/genainormalizer has been in opentelemetry-collector-contrib since February and ships in the otelcol-contrib binary you are probably already running. It maps OpenInference and OpenLLMetry and needs no custom build. If those are the libraries you have, reach for it first.

So the claim here is not that translating GenAI telemetry is novel. It is that none of the translators record what the translation cost. That component’s own source says callers must drop the attribute, and with remove_originals: true the source attribute goes too — so the evidence leaves with the data. Those 105 attributes are what that silence is hiding, and the differences are going upstream rather than into a competing component, because a second one would be worse for everyone than one good one.

Normalization you cannot audit is just another assertion. Everything in this post exists to make the assertion checkable.

Your telemetry was rewritten on the way here

2026-09-08T02:08:31-04:00

Added 2026-09-09. When I wrote this I was arguing from my own normalizer. It turns out the strongest evidence for the argument is in someone else’s: processor/genainormalizer, which ships in otelcol-contrib, drops what it cannot carry — by design, and it says so in its own source. From internal/otelsemconv/coerce.go: “src cannot be safely coerced; callers must drop the attribute” and “Map / Slice / Bytes: do not stringify. Caller drops the rename.” With its remove_originals: true the source attribute is deleted too, so the evidence goes with the data. That is not a criticism of a component I am now contributing to. It is the point of this post, written by someone else, in a comment, where no query can reach it.

Here is a span attribute: gen_ai.usage.input_tokens: 412.

You cannot tell, from that, whether the library emitted it that way. It might have been renamed from gen_ai.usage.prompt_tokens by a collector. It might have been translated from llm.token_count.prompt by something that also dropped three attributes it had nowhere to put. It might have been rewritten at ingest by your vendor, into or out of a vocabulary you never chose. All four produce a span that looks exactly like this one.

That is not a hypothetical about a thing that might start happening. It is a description of a Tuesday.

Everything rewrites telemetry

Four layers, all doing it now, none of them exotic:

A collector normalizes vendor attributes. This is the recommended practice. It is what the transform processor is for and what I spent the last two posts building.

A backend remaps at ingest. Arize AX converts inbound gen_ai.* attributes into OpenInference before your spans are stored. That is documented and defensible, and it means the attribute names you query are not the attribute names you sent.

A schema file converts between versions. OpenTelemetry has a whole mechanism for this: telemetry schemas, applied in the collector, renaming attributes as conventions evolve.

A migration is bridged. When the HTTP conventions renamed half their attributes, everybody ran something in the middle for a year.

Every one of these is a good idea. Together they mean the telemetry arriving in your backend has been through an unknown number of rewrites, by an unknown set of implementations, losing an unknown amount on the way — and there is no standard way for any of them to say so.

`schema_url` answers a different question

The nearest existing mechanism is schema_url, and it is worth being precise about why it does not cover this.

schema_url says which version of a schema the telemetry claims to conform to. That is useful. It is not the same as saying a conversion happened, which direction it went, what the source vocabulary was when the source was not a schema version at all, or what the conversion could not carry.

And it is absent exactly when it would help most. The GenAI semantic conventions currently have no released version to point at — the attributes were deprecated out of the main repository and moved to one that has never tagged anything — so there is no URL to put there. That is not an edge case I went looking for; it is the situation that made me build a normalizer in the first place.

The failure this produces

A translation layer that drops data silently is worse than no translation layer, because you will trust the result.

That sentence is the whole argument and I would like to defend it with something other than assertion.

While building the thing, I recorded on every rewritten span the list of keys that span was not a faithful carrier of. Not a log line — an attribute, next to the data, queryable. It is the least glamorous feature in the repository.

Five of seven captures of real instrumentation libraries turned up a mapping bug that a completely green test suite could not see. Every one was found by reading that list and asking why something was on it. The suite was green because the tests encoded the same misunderstanding the code did. The loss list was the only artifact that disagreed with me.

Then it did it again, in the export path. A generated OTTL config that parsed cleanly in a real collector turned out to carry less than its own header claimed, and I only found out by running both implementations over the same spans and diffing the results — which is, again, just the question “what did this translation cost” asked mechanically.

I am not claiming the idea is clever. I am claiming it is the only part of the system that has ever told me I was wrong.

What to record

Six attributes. No file format changes, no SDK work, no new parser — a namespace and a convention:


source	The vocabulary the telemetry arrived in. A schema URL when there is one; otherwise an identifier, because the common case is that the source declared nothing.
source confidence	How decisive the identification was, when it was inferred rather than declared.
target	What it was translated into.
lossy	The keys this telemetry is not a faithful carrier of.
lossy count	The length of that list, written even when zero.
by	Which implementation did it, when several implementations of one mapping exist and they do not all carry the same amount.

Two of those deserve defending, because both look redundant.

The count, when it is zero. “This translation lost nothing” and “nobody checked” are different claims and an absent array cannot tell them apart. Writing zero is an assertion. It is also the difference between a dimension your backend will aggregate over and one it will not — “which service is losing the most in translation” should be a query, not an afternoon.

The confidence. In practice the source is almost never declared. Instrumentation libraries do not stamp their own name on spans, so anything handling more than one has to infer which it is looking at from the shape of the attributes. That inference is usually easy and occasionally a coin flip. An inference recorded without its confidence is indistinguishable from a fact.

What this deliberately is not

It is not a way to describe the mapping. That is a much larger proposal, it belongs to a different argument, and it is considerably less likely to go anywhere.

This one is only: telemetry that has been translated should be able to say so. Worth doing even if no transformation format ever changes, because the translations are happening right now, in collectors and OTTL configs and vendor ingest pipelines, and not one of them can leave a trace.

Where it goes

OpenTelemetry, and specifically the Semantic Conventions SIG — this is an attribute convention, not a protocol change. Not W3C, who own trace context, the propagation format on the wire, and not attribute semantics. Not IETF.

I have drafted it and not filed it, which I would rather say plainly than let a directory called oteps/ imply otherwise. The order that works for this kind of thing is: ship it, get a couple of real users, open an issue describing the problem rather than your solution, show up to the SIG call twice before proposing anything, file small useful patches first. The document comes last, if it comes at all. An OTEP filed cold by somebody with no history in a project is a document that gets politely queued, and that is a reasonable thing for a project to do with it.

What exists today is the running version, under a namespace named after my own tool — which is exactly what a shared convention must not be, and why the draft proposes a neutral one. It is published as a Weaver registry that weaver registry check validates, so it can be resolved and depended on without adopting a line of my code.

That is a different conversation from a proposal that exists only as prose. It is not yet the conversation, and I would rather be at the beginning of it honestly than describe it as further along.

Don’t invent a language for this

2026-09-08T00:51:03-04:00

Last post I argued that GenAI instrumentation libraries each spell the same facts differently, that you should normalize them at the pipeline, and that whatever your normalizer cannot carry should be written onto the span rather than dropped in silence.

Two questions came back, and they are the right two. Should the mappings become a format — JSON, a small DSL, something SPL-shaped — instead of code? And should any of this be standardized, by somebody other than me?

No, and yes. The no is more interesting than it sounds: I built the format before deciding against it, and what killed it was not taste. And the yes turns out not to be about the mappings at all, which is where this ends up and the part I would keep if you only read one section.

Three layers that keep getting conflated

Normalizing telemetry is three separable things, and almost every conversation about it slides between them mid-sentence.

The rules. llm.token_count.prompt and gen_ai.usage.input_tokens are the same fact. That is data. It is a table.

The execution. Something has to actually rewrite the span, in a pipeline, at volume, without dropping anything.

The claim. An assertion that the two names are the same fact, that translating between them costs nothing — or costs exactly this much — and that the assertion is auditable by somebody who was not there when it was made.

The industry has quietly settled the second layer several times over, privately, inside each vendor’s ingest. It has never published the third. Arize AX rewrites inbound gen_ai.* attributes into OpenInference before your spans are stored. That is a defensible product decision and I am not picking on them; the point is that you cannot read the mapping, cannot version it, and cannot find out what it discarded. Every backend is doing some version of this and none of them will show you the table.

So: which layer would a new language be for?

The rules should be data. That is not the same as a new format.

I went and did this to my own code, which is the only honest way to find out what it costs.

genai-interlingua had two halves that both looked like tables and only one of which was. The schema half — which attribute name each convention version uses, which values it admits — is now generated from the registries OpenTelemetry publishes, rather than transcribed by me. That was straightforwardly correct and I should have done it first. The derivation turned out to be a rule with one exception: the attribute key is gen_ai. plus the field name, except for the token-cache field that v1.41.0 spells cache_creation and the new repository spells cache_write. Fifty keys and seventy-two keys, reproduced exactly. The hand-typed tables had been right, which I would not have bet on.

The mapping half is where it gets interesting. I converted five of the six dialects from Go calls into declared tables — 86 mappings state cleanly as data against 38 that do not, so roughly seven in ten. Stating one looks like: read this key, lowercase it, cut it at the first dot, run it through this lookup, write it there.

The ratio is an average over emitters that are nothing like each other, which matters more than the average. LiteLLM is 20 against 4. OpenInference is 16 against 16, because it packs every sampling parameter — temperature, top_p, the penalties, the seed, nine others — into one JSON string, and no table reaches inside a string. The sixth dialect is not migratable at all: it is the fallback for spans nobody designed, and it works by resolving each attribute against the entire semantic-convention registry at runtime, so writing it as data would either duplicate the registry or admit it carries nothing.

The three in ten that do not state are not waiting for a better format.

Consider reassembling gen_ai.prompt.0.tool_calls.1.arguments — two levels of indexing — into one nested JSON document. Or deciding what traceloop.entity.name means by looking at whether a sibling attribute says this span is an agent or a tool. Or refusing to map a Braintrust span carrying three evaluation scores, because the conventions model one evaluation per span and picking one silently would be worse than admitting the mismatch.

Those are not transformations of a value. They are readings of a span — they need the whole span in hand, not one attribute, because what they produce depends on what else is there. A format expressive enough to state them has conditionals, loops and a JSON parser, at which point you have written a programming language, and a worse one than the several already available.

And one of those already exists

OTTL — the OpenTelemetry Transformation Language — is a DSL for exactly this, governed by OpenTelemetry, shipped in every Collector distribution, running in production at a scale mine never will. Proposing a new config format for telemetry rewriting in 2026 means proposing to compete with it. That is a bad trade for a mapping table.

The better move is the opposite one: emit OTTL. So genai-interlingua now does.

$ interlingua -emit ottl -dialect litellm -target v1.41.0

Out comes a transform processor you paste into your own Collector. No custom build, no Go, no dependency on my repository continuing to exist.

A stock Collector accepts it; I checked, in both target schemas, because a generated config that has only ever been compared against a golden file has not been tested, it has been photographed.

Then I checked the thing that actually matters, which is whether it means the same thing. Every captured span goes through the Go processor and through a real Collector running the exported config, and the two outputs are compared attribute by attribute. It found four bugs on its first run, and every one of them had been sitting behind a config that parsed cleanly and looked right:

A scalar-to-list conversion that wrapped unconditionally, where the Go code wraps only a string. On the one emitter whose value is already an array, the attribute came out as a list containing a list.
A lookup table that never fired on array values at all. Go maps over the elements; OTTL has no iteration, so the generated statements compare the whole array against each table key and match nothing. That one is not fixable — it is now declared instead, which is the point.
An enum guard that was more destructive than the processor, deleting a non-conformant value the processor deliberately preserves.
And a header claiming coverage it did not have.

The third and fourth are the ones I would have shipped. An export that quietly does less than it advertises is the exact failure this post is about, and I wrote it into my own tool while writing the argument against it.

And because the export is a subset, it says so in its own header:

# WHAT THIS CARRIES (20 attributes)
# ...
# WHAT THIS CANNOT CARRY (4 attributes)
#
#   gen_ai.input.messages
#   gen_ai.output.messages
#   gen_ai.response.finish_reasons
#   gen_ai.tool.definitions

Three things do not survive the trip. The unstated mappings, above. Per-span loss accounting, because interlingua.lossy is computed from what a given span turned out to carry and a static config can only describe itself. And detection: picking a dialect means scoring every candidate across the whole span and taking the maximum, and OTTL has neither a loop nor an argmax. An exported config is pinned to one library and scoped by the spellings only that library uses.

That last one is a genuine limit, not a gap to fill later. It is also the smallest of the three.

The rule I would give anyone else: if your mapping can be stated as data, state it as data, then hand the data to a language somebody else maintains. Keep code for the rules that are readings rather than transformations, and be loud about which is which. My compiler cannot tell those apart, so a test does — it runs every captured span through the parser and fails the build if any field comes out that is neither declared as a rule nor admitted as unstateable. Without it, the next mapping I add as a method quietly disappears from the export while the header still claims completeness.

So what would you standardize?

Not the mapping table. That is data with a shelf life. OpenInference instrumentations already dual-emit gen_ai.*; the conventions are stabilizing; in two years the dialect tables are archaeology. A standards body should not be asked to ratify a compatibility shim for a problem that is actively dissolving.

The durable thing is the third layer — the claim — and it is genuinely missing. Here is what exists today and what each artifact can say:

	can express	cannot express
Telemetry Schema File 1.1.0	`rename_attributes`, `rename_events`, `rename_metrics`, metrics-only `split`	value rewriting, unit or type change, flattening, translation between vendors
Weaver `schema-changes`	added / renamed / obsoleted, within one registry’s own lineage	value-level transforms, cross-registry mappings
Weaver registries	third-party registries, dependencies, layering	equivalence claims between sibling registries
OTTL	all of it, imperatively	a portable, reviewable claim about equivalence and fidelity

The gap is the same shape in every row. You can rename an attribute. You cannot say that bedrock and aws.bedrock are the same provider, that milliseconds and seconds are the same duration, or — the one I care about most — that a particular translation was not faithful, and in exactly which respect.

I stopped asserting that and measured it. genai-interlingua now emits schema files too, and the resulting gap table is generated from the rule tables rather than written by me. The blunt version: for LiteLLM at v1.41.0, a schema file expresses zero of the mappings. At genai-main it manages one. Not because the format is bad — because every transformation it has changes a name, and the two mappings LiteLLM actually needs change a value. Eighteen of the rest need no rename at all, since LiteLLM already writes the conventions’ own names, and counting those as coverage would credit the format for work nobody did.

That measurement also cost me half my argument, which is the useful part. Most of what a schema file cannot carry, it cannot carry because the work is a reading of a span — and no declarative format should try to express that. Adding value transforms would not make schema files sufficient for normalizing GenAI telemetry. It would make them able to describe migrations the conventions have already made, which is a narrower claim and the only one the evidence supports.

Then it cost me the correction too, which is the more useful part. I migrated a second dialect and the ratio inverted: the gaps a format change would fix went from a minority to a clear majority, and I rewrote the conclusion to say the fixable share grows as you sample more emitters. Then I migrated three more and it went back.

sample	fixable by a format change	not expressible in any format	fixable share
1 dialect	4	8	33%
2 dialects	24	16	60%
5 dialects	44	76	37%

The middle row was one unusually-shaped emitter at n=2. The Vercel AI SDK writes the same fact under an ai.* name on the span your code creates and a gen_ai.* name on the span its provider adapter creates, so nearly every mapping it has carries two spellings — and twelve of those twenty-four fixable gaps were that one property of that one library. My correction was worse than the thing it corrected.

So the first answer was right, and I only know that because the table regenerates. If I had written “a minority” as prose in September I would have had no way to discover in October that I had briefly and confidently believed otherwise. Generate the number that your argument turns on. Not because generated numbers are true — these moved three times — but because a number that moves in front of you is one you can still be wrong about out loud.

Two things are worth writing down:

Cross-registry equivalence with value transforms. Not just “this key became that key” but the lookup, the unit, the coercion.

A fidelity vocabulary. A way to record, on the telemetry itself, which schema it was translated from, which it was translated to, what could not be carried, and — when the source convention was inferred rather than declared — how confident that inference was. I have been emitting a private version of this for months. It has no business being private. It generalizes far past GenAI: every semantic-convention migration, every vendor’s ingest pipeline, the whole HTTP attribute rename wave, has the same unanswerable question sitting under it.

And the ground is explicitly reserved. OTEP 0152, which introduced telemetry schemas in the first place, says the transformation set was held to “the bare minimum … with more types of transformations potentially proposed in the future,” and that changing the file format version must go through the OTEP process. That is not a wall. That is a door with a sign on it.

Who you submit it to

OpenTelemetry. Three different places, depending on which piece:

The GenAI mapping work, and the missing schema URL, go to semantic-conventions-genai — whose README still reads Schema URL: TODO, which is a fair summary of the situation.
Cross-registry composition is Weaver’s territory, and Weaver is actively building multi-registry layering right now.
A change to the schema file format is an OTEP, filed in the specification repository’s oteps/ directory. Note that the standalone oteps repo was archived in November 2025; proposals moved.

Not W3C — they own trace context, the propagation format on the wire, not attribute semantics. Not IETF. Not CNCF directly, which hosts OpenTelemetry but does not review its specs. And not a bilateral deal with the instrumentation vendors, who have entirely rational product reasons to keep their own vocabularies.

One more thing, which is the part nobody puts in the “how to contribute” guide: the order matters more than the document. An OTEP filed cold by someone with no history in the project is a document that gets politely queued. The sequence that works is to ship the thing, get a couple of real users, open an issue describing the problem rather than your solution, show up to the SIG call twice before proposing anything, and file small useful patches first. The spec comes last, if it comes at all. I am at the beginning of that, not the end of it, and I would rather say so than write this post as though the standard were already in flight.

The short version

Don’t build a language; one exists, and half your rules were never going to be data anyway. Do make the statable half statable, and export it into the language somebody else maintains. Do not try to standardize your mapping table.

Do standardize the thing nobody currently can say: this telemetry was translated, here is from what and to what, and here is precisely what did not survive.

Both proposals are drafted in the open and neither is filed, which is the honest state of them. The second one is visibly weaker than the first, and its own draft says so.

A translation layer that drops data silently is worse than no translation layer, because you will trust the result. That is true of my normalizer, it is true of your vendor’s ingest pipeline, and right now neither of us has a standard way to tell you which.

LLM telemetry has no standard. Here’s how to normalize it.

2026-09-07T12:00:00-04:00

Added 2026-09-09. This post doesn’t mention processor/genainormalizer, which has been in opentelemetry-collector-contrib since February and ships in the otelcol-contrib binary you are probably already running. If you have this problem, start there: it needs no custom build. It maps OpenInference and OpenLLMetry, hardcodes the version it normalizes to, and does not record what a mapping dropped — which is why the four points under What to do about it below still stand, and why I have taken them upstream rather than maintaining a second one. I did not know it existed when I wrote this, which is its own lesson, and the one I would add as a fifth point: before you build a Collector component, go and read the list of Collector components.

Here is the moment you notice. Finance wants the model spend broken down by provider, you have OpenTelemetry traces from every service, and you write the obvious query: sum the input tokens, group by provider. The number comes back and it is far too small. Not zero — small. One team’s worth.

The query is correct. It matched one service, because that service is the only one whose library spells the token count the way your query does.

This is structural, not a bug

Four services, four teams, four reasonable choices made at four different times. Here is the same fact — how many tokens went into a model call — as each of them records it.

	prompt tokens	provider
OpenLLMetry, before its migration	`gen_ai.usage.prompt_tokens`	`gen_ai.system`
OpenInference	`llm.token_count.prompt`	`llm.provider`, or `llm.system`
Vercel AI SDK	`ai.usage.promptTokens`	`ai.model.provider`
the conventions	`gen_ai.usage.input_tokens`	`gen_ai.provider.name`

None of these is wrong. Each library picked names before there was a standard to pick, or picked them from a version of the standard that has since changed. The data is all there. It is just not addressable as one column, and no dashboard can sum a column that does not exist.

So you do the obvious thing: translate every dialect into one vocabulary, at the pipeline, where you can do it once instead of in every service.

What that looks like

I built genai-interlingua for this. It recognizes the dialect a span is written in, rewrites it into one gen_ai.* schema, and records on the span whatever the translation could not carry.

The fastest way to see what it does to your own data is the CLI, which reads OTLP/JSON on stdin and writes it on stdout:

go install github.com/Grace/genai-interlingua/cmd/interlingua@v0.2.0
cat captured-span.json | interlingua -target v1.41.0

An OpenLLMetry span goes in with gen_ai.usage.prompt_tokens and traceloop.workflow.name on it. It comes out still carrying those — nothing is deleted by default — plus the normalized keys:

gen_ai.provider.name           = openai
gen_ai.usage.input_tokens      = 412
gen_ai.usage.output_tokens     = 27

interlingua.dialect            = openllmetry
interlingua.target             = v1.41.0
interlingua.lossy              = [gen_ai.usage.total_tokens, ...]
interlingua.lossy.count        = 5

That last pair is the part I care about most, and I will come back to it.

If you would rather not install anything, the same thing runs in your browser — it is this compiled to WebAssembly, with the captured spans from all seven libraries as presets. Switching the target there is the fastest way to see what the rest of this post is about.

The CLI is for looking, not for running. It is how you check what the mapping does to a payload you have captured, before you trust it. In a real pipeline you would not shell out per span — the same logic ships as an OpenTelemetry Collector processor, which is the integration that actually matters:

processors:
  genaiinterlingua:
    target: v1.41.0

One processor in the collector, and every service behind it starts speaking one vocabulary with nothing re-instrumented.

Which raises the question the rest of this post is about: normalize to what, exactly?

There is no version to normalize to

The gen_ai.* attributes used to live in open-telemetry/semantic-conventions, alongside HTTP and database and everything else. At v1.42.0 they were deprecated there and moved to a dedicated repository, open-telemetry/semantic-conventions-genai.

That is a reasonable decision. GenAI conventions change much faster than HTTP conventions, and a domain moving at a different speed deserves its own release cadence.

Here is the state of that decision today:


`semantic-conventions` v1.41.0	2026-04-28 — last release with live `gen_ai.*`
`semantic-conventions-genai` created	2026-05-05
`semantic-conventions` v1.42.0	2026-06-12 — `gen_ai.*` deprecated and moved out
`semantic-conventions` v1.43.0	2026-07-03
`semantic-conventions` v1.44.0	2026-08-04
`semantic-conventions-genai` releases	zero
`semantic-conventions-genai` tags	zero

The main repository has tagged three releases since the move. The repository that actually owns these attributes has tagged nothing, while committing to main steadily — twenty-one commits in the thirty days before I last checked.

The definitions are moving faster than they were before the split, and there is now no version number anywhere on them.

What that costs you, specifically

For most semantic conventions this question is boring. You pick a version, you put it in schema_url, and a consumer reading your span in three years can look up exactly what you meant by every key on it. Schema URLs are also what lets a backend apply transformations to carry old data forward.

There is no such string for GenAI. You cannot pin main. You can pin a commit, but nothing in the ecosystem will resolve it and no consumer will know what to do with it.

So “normalize to the GenAI conventions” is an underspecified instruction, and whoever writes the normalizer has to answer it. There are two answers and neither is correct:

Pin the last tagged cut, v1.41.0. It is frozen, it is deprecated at its source, and it is missing everything added in the months since. But it is a version, and a reader six months from now can reconstruct exactly what it meant.

Track main. You get the current definitions, and “conformant” becomes a claim with no version attached, which may quietly stop being true on any given Tuesday.

What is not defensible is making the choice silently, which is what a normalizer with a hardcoded attribute table does. Whoever wrote that table picked a version. They just did not write down which one, and neither did the spans.

The measurement that made me feel better

I expected pinning the frozen cut to be expensive. It is not, yet.

OpenTelemetry ships first-party GenAI instrumentation (opentelemetry-instrumentation-openai-v2). I captured a span from it and normalized that span to both targets — the frozen v1.41.0 and current main. The entire difference between the two outputs was one string: the label recording which target had been used. Every attribute the reference implementation emits today is expressible at the frozen cut.

So for spans from the official instrumentation, pinning costs nothing at all right now. That is a much better argument for the frozen default than “it is the only version you can name.”

Two caveats, and they are the point rather than footnotes. Those packages are versioned 2.4b0 and 1.1b0 — beta code tracking an untagged specification. The gap can open at any time with no version number changing to warn anyone. And even that span carries openai.response.system_fingerprint, which no target expresses at any version, so even the reference implementation emits data the conventions have no home for.

What to do about it

Four things, none of which require the upstream situation to resolve.

Make the target a parameter, not a constant. If your normalizer has a table of attribute names in it, that table is a snapshot of one version of a moving schema, and the version it snapshotted is not written down anywhere. Make it a flag with an enum, and make the default a decision you can defend out loud.

Put the target on the span. There is no schema_url to carry it, so the span itself is the only durable record of which vocabulary its gen_ai.* keys belong to. One attribute. A reader six months from now should not have to find the collector config that produced the data.

Record what the translation cost. Normalizing is lossy: some fields have no home at your target, some values are outside its enum, some emitters pack three facts into one JSON blob. A normalizer that drops those silently is worse than no normalizer, because you will trust the result. Put the list of keys the span is not a faithful carrier of on the span, next to the data. Then “which library is losing us the most” is a query rather than an afternoon.

And check the schema you copied from. I transcribed both attribute tables by hand from upstream YAML, which is a fine way to build them and a terrible way to keep them. Upstream publishes machine-readable registries; a test that compares your tables against them turns silent staleness into a build failure. Mine passes today, which I would not have bet on before writing it.

genai-interlingua does all four, across six instrumentation dialects. There is a browser demo if you want to see it work before installing anything; source and Collector configuration are in the repository, and there are binaries on the releases page for the usual six platforms.

Two things in it go further than this post does. The moving-target write-up is the long version of the schema argument, with the full timeline and what happens to everyone the day that repository finally tags something. And what capturing the real libraries found is what happened when I stopped trusting my own fixtures and recorded spans from the libraries actually running: five of seven captures turned up a mapping bug that a completely green test suite could not see, including one in OpenTelemetry’s own instrumentation output.

Duration is why every service on the path looks guilty

2026-09-05T18:00:00-04:00

Here is the moment you notice. Something is slow, you pull up the services on the request path sorted by how much worse they got, and five of them are at the top. Frontend: worse. The service it called: worse. The service that called: worse, by almost exactly the same amount. The list is correct and it has told you nothing.

This is structural, not a bug

A span’s duration is wall clock from start to end, and that includes every moment its children were running. So when a leaf slows down by 1.9 seconds, its parent’s duration grows by 1.9 seconds, and its parent’s parent’s duration grows by 1.9 seconds, all the way up to the entry point. Every one of those services genuinely did take longer. None of them except the last did anything differently.

One real request from the adManualGc window, trace 5be7c5ada59b. Every span on the path lasts about 3.08 seconds — left column. Only the last one is doing anything — right column, same scale. The seven above it are waiting on it.

Ranking by duration, or by any dimensional diff over durations, cannot separate them, because on that measurement they are not different. The signal you want isn’t in the numbers being compared. It’s in the shape of the tree they came from.

Self time is the measurement that separates them

Take a span’s duration and subtract the wall-clock time its children covered. What’s left is the work that span did itself.

When the leaf slows down, only the leaf’s self time moves. Its ancestors’ self times don’t move at all — they were waiting, and waiting is not work. The ranked list collapses from five services to one.

Two details matter more than they look.

Merge the children as intervals; don’t sum them. A fan-out to three services at once overlaps in wall clock. Summing three concurrent 80ms children against a 100ms parent gives you 240ms of “children” and a self time of negative 140ms, which reads as the parent having gotten faster while the system got slower. The question is how much of the parent’s wall clock was not covered by anything below it, and that’s a union.

Clamp children to their parent’s window. Clock skew across hosts is normal, and a child that reports starting before its parent will manufacture self time out of nothing if you let it.

What it does to a real incident

I ran ranger against the OpenTelemetry Demo — commit 8c47d47 — with the adManualGc flag on, which triggers full manual garbage collections in the ad service. Five minutes of baseline, five minutes with the flag on, 88 operations ranked.

The top of the list:

ad · oteldemo.AdService/GetAds     self 4.22ms → 2.10s     z 742

The runner-up scored 1.7. Not a close call.

The same window, ranked two ways. By duration the service that actually stalled is fourth, behind three frontend spans that were only waiting for it — all four within 30ms of each other. By self time it is first, by a factor of eight over anything else in the window.

And the span that makes the point is the one that didn’t rank. The frontend’s client span for that same call — the outbound side of the identical request — moved like this:

duration   +2125.1ms
self time      +9.0ms

Its duration moved roughly two hundred times more than its own work did. On a duration ranking it’s the second-worst thing in the system. On self time it’s a service sitting patiently on a socket, which is what it actually was.

What real traces taught that my own tests hadn’t

I had a test for this. Synthetic traces, a caller whose duration moved and whose self time didn’t, asserting the caller gets labelled as waiting. It passed.

Then real traces arrived and the caller got labelled as the cause.

The classifier asked two questions in the wrong order. It checked an absolute floor first — did this operation’s self time move by at least a couple of milliseconds — and only asked about proportion if the answer was no. In my synthetic fixtures the caller’s own noise was fifteen microseconds, comfortably under the floor, so the proportion question always got asked and the test always passed.

Real callers are not that quiet. The frontend picked up 3.6ms of its own jitter during a window where the thing underneath it was pausing for whole seconds. 3.6ms clears any floor worth having. So the operation was called slower, and the list handed an on-call engineer the wrong service.

Waiting is a question about proportion and it has to be asked first: did this operation’s own work account for a real share of its own slowdown, or did something below it? An operation whose self time explains less than a fifth of its duration shift didn’t cause it, however many milliseconds that fifth happens to be.

The fix didn’t change the answer — the ad service ranked first either way, at z=742 against 1.7. It changed a label on the row underneath, which is the row someone reads when the top one doesn’t look right. The demo’s actual numbers are the regression test now.

The number, and what’s wrong with it

Four labeled failures from the demo, five minutes of baseline and five minutes per incident, scored two ways:

ranking	top-1	top-3	wrong	declined
deviation (robust-z)	25%	25%	50%	25%
effect size	25%	50%	25%	25%

Four cases is not a benchmark. It is enough to say the tool names the wrong service more often than the right one, and that’s the number to carry rather than the one case it nailed.

The second row is there because measuring this exposed a mistake in the ranking itself. Robust-z asks how far a shift falls outside an operation’s own history — that’s a significance test — and I was reading it as importance. Across tiers those come apart badly: a browser page-load timing moving 30% of a 2.4-second baseline outscores a gRPC handler that tripled 3.5ms, and only one of those is a cause. Scoring the shift as a fraction of the operation’s own baseline instead — an effect size — halves the wrong rate. Everything with no instrumented children made it worse, because a span with no children has self time equal to its duration by construction, so it can never be called a waiter and always looks like its own cause. Browser and load-generator spans are almost all leaves.

The bigger finding isn’t in the tool. I captured one baseline at the start of the run and compared it against windows up to 32 minutes later, and the ad service ends the run about 2.5× slower than it began with nothing injected into it:

baseline                     GetAds p50   4.33ms
productCatalogFailure        GetAds p50  11.19ms   ← ad untouched
recommendationCacheFailure   GetAds p50  10.42ms   ← ad untouched

That drift gets attributed to whichever flag happened to be on. It’s why the ad service tops the ranking in a window about a recommendation-service cache leak — a false positive my harness manufactured, not the ranking. So 25% is a floor on the error rate, not an estimate of it, and the fix is interleaving a fresh baseline between injections instead of reusing one.

Three other things the harness got wrong before the tool got a chance to. cartFailure only fires inside EmptyCart, which sees about three calls a minute — 14 samples against a floor of 20, so it can’t clear the bar in five minutes and it’s out of the set. intlShippingSlowdown looked like the ideal first case and is unusable: it only delays non-US addresses, and exactly one of the demo’s nine load-generator personas is Canadian. And productCatalogFailure couldn’t fire at all for two runs, because the demo ships it with a targeting rule whose branches are both "off" — and a targeting rule overrides the default variant, so flipping the default left the flag disabled while appearing to work. It scored as a decline. That was me, not the tool, and it’s why every case now verifies its own injection and writes the evidence next to the result.

The scoring keeps wrong and declined in separate columns throughout. A localizer that’s right 60% of the time and quiet the rest is usable at 3am; one that’s right 60% and confidently wrong the rest is not, and both score 60% if you only count hits. A service that only ever appeared as “waiting on something below it” counts as a miss, never a hit — anything else is marking your own homework on the one distinction the tool claims to make.

The code is at github.com/Grace/ranger. Pre-alpha, and the README says so.

Grace L. Christenbery's Blog

Three places to put a translator

A: in the path

B: beside the path

C: not in the path at all

Why it works

What C gives up

The part I actually care about

There is no model in here

The problem, in four lines

Stage 1: which library wrote this?

The highest score usually loses

Stage 2: translating it

Stage 3: what could not be carried

The counter that counts what you forgot

What that measures, across six real dialects

Stage 4: pinning a target costs something too

What ends up on the span

Go and look

What this is not

Your telemetry was rewritten on the way here

Everything rewrites telemetry

schema_url answers a different question

The failure this produces

What to record

What this deliberately is not

Where it goes

Don’t invent a language for this

Three layers that keep getting conflated

The rules should be data. That is not the same as a new format.

And one of those already exists

So what would you standardize?

Who you submit it to

The short version

LLM telemetry has no standard. Here’s how to normalize it.

This is structural, not a bug

What that looks like

There is no version to normalize to

What that costs you, specifically

The measurement that made me feel better

What to do about it

Duration is why every service on the path looks guilty

This is structural, not a bug

Self time is the measurement that separates them

What it does to a real incident

What real traces taught that my own tests hadn’t

The number, and what’s wrong with it

`schema_url` answers a different question