I try to add observability support to my spring boot 3 application and I've found out that by default there only 10% of traces are collected. To collect all traces I've added following config
management:
tracing:
sampling:
probability: 1
But later I've thought about it and now I think that it spring(or micrometer) developers had some reasons why have they made those default to 10%.
Does tracing slowdown the application ? Should I disable it in production and enable only in case of some issue?
P.S.
Based on micrometer documentation:
Micrometer Tracing provides a simple facade for the most popular tracer libraries, letting you instrument your JVM-based application code without vendor lock-in. It is designed to add little to no overhead to your tracing collection activity while maximizing the portability of your tracing effort.
So it looks controversial for me. Little or no overhead but by default only 10% of traces are collected.
I try to add observability support to my spring boot 3 application and I've found out that by default there only 10% of traces are collected. To collect all traces I've added following config
management:
tracing:
sampling:
probability: 1
But later I've thought about it and now I think that it spring(or micrometer) developers had some reasons why have they made those default to 10%.
Does tracing slowdown the application ? Should I disable it in production and enable only in case of some issue?
P.S.
Based on micrometer documentation:
Micrometer Tracing provides a simple facade for the most popular tracer libraries, letting you instrument your JVM-based application code without vendor lock-in. It is designed to add little to no overhead to your tracing collection activity while maximizing the portability of your tracing effort.
So it looks controversial for me. Little or no overhead but by default only 10% of traces are collected.
Share Improve this question edited Mar 4 at 9:54 gstackoverflow asked Mar 3 at 14:03 gstackoverflowgstackoverflow 36.6k138 gold badges419 silver badges786 bronze badges1 Answer
Reset to default 0You inferred it correctly. The new observability feature of spring boot 3 is designed keeping system performance monitoring in mind and not Tracing/logging. Thus a small data set serves good enough for the assessment.
The feature of management.tracing.sampling.probability is offered to the devops/ developers to configure the probability value based on specific requirement for each app.
Setting the probability to 1 does negatively affect the application and eventually it does comes down to Trade-Off Between Observability and Performance.
High probability does affect Increased CPU usage, Higher memory consumption and Greater network I/O to send traces to the backend.
For Production, my recommendation would be to reduce the exposure gradually e.g. Keep the probability higher (less than 1) in first 24 hours and then gradually decreasing it. This way, you would have a good data set to see how your application is behaving without affecting performance.