I have an issue with Open Telemetry distributed tracing and Service Bus Message-triggered Logic App Workflows. We have OTEL and distributed tracing in Azure involving Web APIs, Azure Functions and Logic Apps.
If a logic app workflow is HTTP-triggered, traceparent and diagnostic-id headers (with the same value) are passed in the HTTP headers and traces are produced correctly. The trace id is being extracted and used.
If instead a logic app workflow is Azure Service Bus message-triggered, the traceparent and diagnostic-id values are passed as message properties (UserProperties in the message). Here is where we have the problem. In this case the logic app workflow does not extract and does not use the trace id that is passed. It creates and uses new trace ids for each step in the workflow.
My hypothesis is that this is either some logic app limitation or we have some missing configuration.
Here is an example of logged json message that triggers the logic app workflow.
{
"headers": null,
"body": {
"contentData": {
"greeting": "hello"
},
"contentType": "application/json",
"userProperties": {
"traceparent": "00-3371dfda4f619ea823086e0945c089c3-ae3de3878ad08241-01",
"diagnostic-id": "00-3371dfda4f619ea823086e0945c089c3-ae3de3878ad08241-01",
"dataType": "test"
},
"messageId": "197ae0b969b54f17bb470ce017b83847",
"scheduledEnqueueTimeUtc": "1/1/0001 12:00:00 AM",
"timeToLive": "14.00:00:00",
"deliveryCount": 1,
"enqueuedSequenceNumber": 225,
"enqueuedTimeUtc": "2025-02-12T15:03:02.614Z",
"lockedUntilUtc": "2025-02-12T15:04:02.63Z",
"lockToken": "27133a78-4d53-4df0-b97b-1e406f788864",
"sequenceNumber": 5
}
}
The logic app Diagnostic Settings are the following:
cpq-dv-diagnostic-log
Datadog_DS_V2_700b9f7a
We have also Data Dog Environment variables defined (I am putting the most relevant ones)
DD_TRACE_DEBUG = true
DD_TRACE_OTEL_ENABLED = true
DD_TRACE_PROPAGATION_EXTRACT_FIRST = true
DD_TRACE_PROPAGATION_STYLE_EXTRACT = tracecontext,datadog
DD_TRACE_PROPAGATION_STYLE_INJECT = tracecontext,datadog
Can U please suggest a solution? Thanks in advance.