I'm ingesting log output from a Spring Boot application with Open Telemetry configured and using the open telemetry collector to import logs into loki (also into victoriametrics-logs for comparing).
This is my open telemetry collector configuration.
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
zipkin: # just for troubleshooting
#
processors:
attributes:
actions:
- key: team
action: insert
value: martinahrer
exporters:
debug:
verbosity: detailed
#otlphttp exporter
#.md
otlphttp/victoria-logs:
compression: gzip
endpoint: http://victoria-logs:9428/insert/opentelemetry
tls:
insecure: true
otlphttp/victoria-metrics:
compression: gzip
endpoint: http://victoria-metrics:8428/opentelemetry/
tls:
insecure: true
#otlp exporter
#
otlphttp/tempo:
# tempo:4317 #gRPC
endpoint: http://tempo:4318
tls:
insecure: true
# Added just to compare with victoria-metrics
prometheusremotewrite:
endpoint: http://prometheus:9090/api/v1/write
# Added just to compare with victorialogs
otlphttp/loki:
endpoint: http://loki:3100/otlp
service:
telemetry:
logs:
level: DEBUG
extensions:
- zpages
pipelines:
logs:
receivers: [otlp]
processors: []
exporters: [debug, otlphttp/victoria-logs, otlphttp/loki]
metrics:
receivers: [otlp]
processors: [attributes]
exporters: [debug, otlphttp/victoria-metrics, prometheusremotewrite]
traces:
receivers: [otlp, zipkin]
exporters: [debug, otlphttp/tempo]
extensions:
zpages:
endpoint: "0.0.0.0:55679"
loki is running with default configuration!
With the loki datasource certain fields like the spanId and the traceID appear twice (as span_id and trace_id) when quering for logs in grafana. These do not appear when querying my victoriametrics-logs datasource.
All of those fields appear as structured metadata!
S trace_id d4631c7e0ac527b81064282c3523a39f
S traceId d4631c7e0ac527b81064282c3523a39f
Is this some loki thing? Looking at the opentelemtery collector log output I can only find the attributes spanId and traceId.
How can I prevent these to appear twice or at least how can I remove those from the attributes displayed with loki.
I have been experimenting with limits_config as shown in but had no success with that.