Disclaimer: I haven't used the otel collector before.
I am trying out a system where I can export Otel to a file, then re-import that contents of the file into an Otel collector (otel/opentelemetry-collector-contrib image).
For troubleshooting purposes, I actually created my own image because the official version is so stripped down that it's very hard to troubleshoot. But essentially, I just downloaded and executed the otel-contrib executable. Something like this:
# syntax=docker/dockerfile:1.5
# Install collector
FROM MyFatUbuntuImage AS Builder
WORKDIR /App
RUN \
curl --proto '=https' --tlsv1.2 -fOL .118.0/otelcol-contrib_0.118.0_linux_amd64.tar.gz && \
tar -xvf otelcol-contrib_0.118.0_linux_amd64.tar.gz && \
rm otelcol-contrib_0.118.0_linux_amd64.tar.gz
# Final stage
FROM ubuntu
WORKDIR /App
COPY --from=Builder /App /App/
ENTRYPOINT [ "/App/otelcol-contrib", "--config", "/etc/otelcol/config.yaml" ]
I managed to successfully get configure the Otel collector to export incoming telemetry to a file using the following configuration.
My compose project:
services:
otel-exporter:
image: opentelemetry-collector
container_name: otel-exporter
restart: always
ports:
- "1100:4317" # OTLP gRPC
volumes:
- ./otel-config.yml:/etc/otelcol/config.yaml
- ./Data:/Data
otel-config.yml:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:
exporters:
file:
path: "/Data/Telemetry.json" # Path where telemetry will be stored
rotation:
max_megabytes: 100 # Rotate after 100MB
max_days: 7 # Keep files for 7 days
max_backups: 5 # Keep 5 rotated backups
debug:
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
logs:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
I got a Telemetry.json file that contains various traces, logs and metrics. All good here.
The problem I have is with the import. This is what I've tried.
compose file:
services:
aspire-dashboard:
image: mcr.microsoft/dotnet/aspire-dashboard:9.0
container_name: aspire-dashboard
restart: unless-stopped
ports:
- "18888:18888" # Dashboard UI
- "4317:18889" # OTLP gRPC
environment:
- LOGGING__LOGLEVEL__DEFAULT=Information
- LOGGING__LOGLEVEL__MICROSOFT=Information
- LOGGING__LOGLEVEL__MICROSOFT.ASPNETCORE.DATAPROTECTION=Information
otel-importer:
#image: otel/opentelemetry-collector-contrib:latest
image: opentelemetry-collector
container_name: otel-importer
restart: unless-stopped
depends_on:
- aspire-dashboard
volumes:
- "./otel-config.yml:/etc/otelcol/config.yaml:ro"
- "./Telemetry.json:/Data/Telemetry.json" # Read exported telemetry
otel-config.yml:
receivers:
otlpjsonfile:
include:
- "/Data/*"
processors:
batch:
exporters:
otlp:
endpoint: "http://aspire-dashboard:18889" # Send telemetry to another OTLP collector
tls:
insecure: true
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
logs:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
metrics:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
When I try to run it, the dashboard remains empty. I opened port 4317 for testing purposes, because I have been able to push telemetry to this collector before, and when I tried to do so again using this compose file, it worked fine.
I double checked permissions and that the telemetry file was properly mounted inside the importer container, and it was. No output in either container could explain what is happening.
Importer container:
2025-02-03T13:26:20.280Z info [email protected]/service.go:164 Setting up own telemetry...
2025-02-03T13:26:20.280Z info telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2025-02-03T13:26:20.280Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "debug"}
2025-02-03T13:26:20.281Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "debug"}
2025-02-03T13:26:20.281Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2025-02-03T13:26:20.281Z info [email protected]/service.go:230 Starting otelcol-contrib... {"Version": "0.118.0", "NumCPU": 16}
2025-02-03T13:26:20.281Z info extensions/extensions.go:39 Starting extensions...
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.282Z info [email protected]/service.go:253 Everything is ready. Begin running and processing data.
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "metrics", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "logs", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
Some startup errors, but after that, it seems fine.
Dashboard container:
info: Aspire.Dashboard.DashboardWebApplication[0]
Aspire version: 9.0.0+01ed51919f8df692ececce51048a140615dc759d
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
Storing keys in a directory '/home/app/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[62]
User profile is available. Using '/home/app/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
Creating key {50cd5d08-6625-41e8-a6af-6c34d194c0d4} with creation date 2025-02-03 13:43:40Z, activation date 2025-02-03 13:43:40Z, and expiration date 2025-05-04 13:43:40Z.
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
No XML encryptor configured. Key {50cd5d08-6625-41e8-a6af-6c34d194c0d4} may be persisted to storage in unencrypted form.
info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
Writing data to file '/home/app/.aspnet/DataProtection-Keys/key-50cd5d08-6625-41e8-a6af-6c34d194c0d4.xml'.
info: Aspire.Dashboard.DashboardWebApplication[0]
Now listening on: http://[::]:18888
info: Aspire.Dashboard.DashboardWebApplication[0]
Login to the dashboard at http://localhost:18888/login?t=47b568160c0b6f6a40fbf4ba20101010. The URL may need changes depending on how network access to the container is configured.
info: Aspire.Dashboard.DashboardWebApplication[0]
OTLP/gRPC listening on: http://[::]:18889
info: Aspire.Dashboard.DashboardWebApplication[0]
OTLP/HTTP listening on: http://[::]:18890
warn: Aspire.Dashboard.DashboardWebApplication[0]
OTLP server is unsecured. Untrusted apps can send telemetry to the dashboard. For more information, visit /?linkid=2267030
No errors here.
Any ideas what I'm doing wrong?
Disclaimer: I haven't used the otel collector before.
I am trying out a system where I can export Otel to a file, then re-import that contents of the file into an Otel collector (otel/opentelemetry-collector-contrib image).
For troubleshooting purposes, I actually created my own image because the official version is so stripped down that it's very hard to troubleshoot. But essentially, I just downloaded and executed the otel-contrib executable. Something like this:
# syntax=docker/dockerfile:1.5
# Install collector
FROM MyFatUbuntuImage AS Builder
WORKDIR /App
RUN \
curl --proto '=https' --tlsv1.2 -fOL https://github/open-telemetry/opentelemetry-collector-releases/releases/download/v0.118.0/otelcol-contrib_0.118.0_linux_amd64.tar.gz && \
tar -xvf otelcol-contrib_0.118.0_linux_amd64.tar.gz && \
rm otelcol-contrib_0.118.0_linux_amd64.tar.gz
# Final stage
FROM ubuntu
WORKDIR /App
COPY --from=Builder /App /App/
ENTRYPOINT [ "/App/otelcol-contrib", "--config", "/etc/otelcol/config.yaml" ]
I managed to successfully get configure the Otel collector to export incoming telemetry to a file using the following configuration.
My compose project:
services:
otel-exporter:
image: opentelemetry-collector
container_name: otel-exporter
restart: always
ports:
- "1100:4317" # OTLP gRPC
volumes:
- ./otel-config.yml:/etc/otelcol/config.yaml
- ./Data:/Data
otel-config.yml:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:
exporters:
file:
path: "/Data/Telemetry.json" # Path where telemetry will be stored
rotation:
max_megabytes: 100 # Rotate after 100MB
max_days: 7 # Keep files for 7 days
max_backups: 5 # Keep 5 rotated backups
debug:
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
logs:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug, file]
I got a Telemetry.json file that contains various traces, logs and metrics. All good here.
The problem I have is with the import. This is what I've tried.
compose file:
services:
aspire-dashboard:
image: mcr.microsoft/dotnet/aspire-dashboard:9.0
container_name: aspire-dashboard
restart: unless-stopped
ports:
- "18888:18888" # Dashboard UI
- "4317:18889" # OTLP gRPC
environment:
- LOGGING__LOGLEVEL__DEFAULT=Information
- LOGGING__LOGLEVEL__MICROSOFT=Information
- LOGGING__LOGLEVEL__MICROSOFT.ASPNETCORE.DATAPROTECTION=Information
otel-importer:
#image: otel/opentelemetry-collector-contrib:latest
image: opentelemetry-collector
container_name: otel-importer
restart: unless-stopped
depends_on:
- aspire-dashboard
volumes:
- "./otel-config.yml:/etc/otelcol/config.yaml:ro"
- "./Telemetry.json:/Data/Telemetry.json" # Read exported telemetry
otel-config.yml:
receivers:
otlpjsonfile:
include:
- "/Data/*"
processors:
batch:
exporters:
otlp:
endpoint: "http://aspire-dashboard:18889" # Send telemetry to another OTLP collector
tls:
insecure: true
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
logs:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
metrics:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
When I try to run it, the dashboard remains empty. I opened port 4317 for testing purposes, because I have been able to push telemetry to this collector before, and when I tried to do so again using this compose file, it worked fine.
I double checked permissions and that the telemetry file was properly mounted inside the importer container, and it was. No output in either container could explain what is happening.
Importer container:
2025-02-03T13:26:20.280Z info [email protected]/service.go:164 Setting up own telemetry...
2025-02-03T13:26:20.280Z info telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2025-02-03T13:26:20.280Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "debug"}
2025-02-03T13:26:20.281Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "traces", "name": "debug"}
2025-02-03T13:26:20.281Z info builders/builders.go:26 Development component. May change in the future. {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2025-02-03T13:26:20.281Z info [email protected]/service.go:230 Starting otelcol-contrib... {"Version": "0.118.0", "NumCPU": 16}
2025-02-03T13:26:20.281Z info extensions/extensions.go:39 Starting extensions...
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #3 SubChannel #4]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.282Z info [email protected]/service.go:253 Everything is ready. Begin running and processing data.
2025-02-03T13:26:20.282Z warn [email protected]/clientconn.go:1381 [core] [Channel #5 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "aspire-dashboard:18889", ServerName: "aspire-dashboard:18889", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.25.0.2:18889: connect: connection refused" {"grpc_log": true}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "metrics", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "traces", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
2025-02-03T13:26:20.483Z info fileconsumer/file.go:265 Started watching file {"kind": "receiver", "name": "otlpjsonfile", "data_type": "logs", "component": "fileconsumer", "path": "/Data/Telemetry.json"}
Some startup errors, but after that, it seems fine.
Dashboard container:
info: Aspire.Dashboard.DashboardWebApplication[0]
Aspire version: 9.0.0+01ed51919f8df692ececce51048a140615dc759d
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
Storing keys in a directory '/home/app/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[62]
User profile is available. Using '/home/app/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.
info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
Creating key {50cd5d08-6625-41e8-a6af-6c34d194c0d4} with creation date 2025-02-03 13:43:40Z, activation date 2025-02-03 13:43:40Z, and expiration date 2025-05-04 13:43:40Z.
warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
No XML encryptor configured. Key {50cd5d08-6625-41e8-a6af-6c34d194c0d4} may be persisted to storage in unencrypted form.
info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
Writing data to file '/home/app/.aspnet/DataProtection-Keys/key-50cd5d08-6625-41e8-a6af-6c34d194c0d4.xml'.
info: Aspire.Dashboard.DashboardWebApplication[0]
Now listening on: http://[::]:18888
info: Aspire.Dashboard.DashboardWebApplication[0]
Login to the dashboard at http://localhost:18888/login?t=47b568160c0b6f6a40fbf4ba20101010. The URL may need changes depending on how network access to the container is configured.
info: Aspire.Dashboard.DashboardWebApplication[0]
OTLP/gRPC listening on: http://[::]:18889
info: Aspire.Dashboard.DashboardWebApplication[0]
OTLP/HTTP listening on: http://[::]:18890
warn: Aspire.Dashboard.DashboardWebApplication[0]
OTLP server is unsecured. Untrusted apps can send telemetry to the dashboard. For more information, visit https://go.microsoft/fwlink/?linkid=2267030
No errors here.
Any ideas what I'm doing wrong?
Share Improve this question asked Feb 3 at 13:46 AthenaAthena 656 bronze badges1 Answer
Reset to default 0I think I may have an answer. I was under the impression that the otlpjsonfile receiver plugin didn't take any configuration options other than include, because no configurations was listed in the plugin's readme file (https://github/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/otlpjsonfilereceiver/README.md), except for an example. But that turned out to be a false assumption.
I can't be sure where the options for the plugin is documented, but I assume it's the as the filelog plugin (https://github/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/fileexporter/README.md).
I updated my otel-config.yml to the following and it worked:
receivers:
otlpjsonfile:
start_at: beginning
include:
- "/Data/*"
processors:
batch:
exporters:
otlp:
endpoint: "http://aspire-dashboard:18889" # Send telemetry to another OTLP collector
tls:
insecure: true
debug:
verbosity: detailed
service:
pipelines:
traces:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
logs:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
metrics:
receivers: [otlpjsonfile]
processors: [batch]
exporters: [debug, otlp]
But I am still new to the otel collector, so I would still love to hear feedback on my approach. All the contents of the telemetry file is exported exactly once to the dashboard.