I'm working on a C++ application where our custom object " service" is started in its own thread and a manager is dealing all the services. The main pro of this architecture is every service can access and interact with any other. But now almost each feature is done inside a service without thinking about ownership or lifespan between object.

Example:

We have a "config" service that reads a JSON file at the start of the execution. The config data is then accessed by other services throughout the system. While this setup works for sharing configuration data, I’m wondering if it's suboptimal to dedicate a separate thread to a service that does little more than read the file at the beginning and then sits idle while only providing data for others to access. My concern:

Is it overkill to create a separate thread for a service that only reads a file once and doesn’t perform any active processing afterward? I wonder if the system would perform better if such services simply owned their own data (like a class with a copy of the configuration data) instead of creating a service and managing a thread for it. Even though the service does nothing active after initialization, it feels like using a service with its own thread just for data sharing might be unnecessarily resource-intensive.

Questions:

• Is it suboptimal to create a separate thread for each service in a multi-service C++ application, especially when the service does minimal work like reading a file once?

• Could the thread-per-service model be contributing to the high load average, and if so, would it be better to simply have services own a copy of the required data instead of creating threads for every service? Would it create latency and CPU shortage?

I want to try to create a thread only when needed, but since it will refactor all my architecture I just want to understand correctly thread perfomance and cpu before :)

Edit: for more context, I work on an embedded system with 4 cores and have currently a load average over time of around 20 and not much more available cpu; I am looking for optimizations.

I'm working on a C++ application where our custom object " service" is started in its own thread and a manager is dealing all the services. The main pro of this architecture is every service can access and interact with any other. But now almost each feature is done inside a service without thinking about ownership or lifespan between object.

Example:

We have a "config" service that reads a JSON file at the start of the execution. The config data is then accessed by other services throughout the system. While this setup works for sharing configuration data, I’m wondering if it's suboptimal to dedicate a separate thread to a service that does little more than read the file at the beginning and then sits idle while only providing data for others to access. My concern:

Is it overkill to create a separate thread for a service that only reads a file once and doesn’t perform any active processing afterward? I wonder if the system would perform better if such services simply owned their own data (like a class with a copy of the configuration data) instead of creating a service and managing a thread for it. Even though the service does nothing active after initialization, it feels like using a service with its own thread just for data sharing might be unnecessarily resource-intensive.

Questions:

• Is it suboptimal to create a separate thread for each service in a multi-service C++ application, especially when the service does minimal work like reading a file once?

• Could the thread-per-service model be contributing to the high load average, and if so, would it be better to simply have services own a copy of the required data instead of creating threads for every service? Would it create latency and CPU shortage?

I want to try to create a thread only when needed, but since it will refactor all my architecture I just want to understand correctly thread perfomance and cpu before :)

Edit: for more context, I work on an embedded system with 4 cores and have currently a load average over time of around 20 and not much more available cpu; I am looking for optimizations.

• c++
• multithreading
• performance
• optimization
• architecture

• 6 This isn't really something with a clear answer. The best answer is to write test-programs that uses single threading, thread pooling, and one thread per task/service. Then benchmark to see which ones perform better on your target platform. – Some programmer dude Commented Feb 17 at 13:46
• Load average is computed from processes and threads in the R (wants to run) or D (uninterruptible sleep... usually I/O) states. If it truly "doesn’t perform any active processing afterward" and is sitting around in S state, the thread won't contribute to load. – teapot418 Commented Feb 17 at 13:46
• 3 the problem that i fear is that for example you have getX() that creates a request on some queue, that the thread picks up and then replies by putting another message on another queue for another thread to pick up etc ... now a simple memory access that should've taken 100 nanoseconds is like 40 microseconds, if that's the case then it will obviously be bad. microservices do it because they work on different computers, while for your application, a few concurrent containers could just do the trick sometimes, and not everything needs to be a service that runs on its own thread – Ahmed AEK Commented Feb 17 at 13:58

2 Answers 2

You cannot over-commit the CPU, you can make as many as you want, and threads will still be scheduled.

What you can get is :

• That each thread will consume system resources like memory, mostly stack space. Which in turn can lead to over-use of virtual memory which can cause swapping
• It typically also aversly affects the usage of your cached memory since each thread uses its own stack which might be swapped out by the time a thread is scheduled again
• It may cause extra latency because each thread needs to be scheduled for a slice of CPU time (assuming threads are not waiting for something like a condition variable which puts them to sleep).
• Adding more threads than cores won't speed up your program (it might improve responsiveness), so don't add more threads because you think it will speed up calculations (it doesn't).

So all in all design a system with some workerthreads (threadpool) where you can queue work and let them do their thing. Or have shorter lived threads that last for the duration of your activity (like reading a file) and then remove them.

Is it overkill to create a separate thread for a service that only reads a file once and doesn’t perform any active processing afterward?

If the "system" is contained within a single process?* Then absolutely! Yes. That is way overkill. If the configuration data are immutable after they've been read, then why not simply make a big, read-only "config" struct, and let any thread in the program directly access it?

But, what if the configuration can be changed on-the-fly? Well, if you're going to allow thread A to change the data while threads B, C, D, and E are trying to use the data, then you're going to have to implement a protocol to prevent thread A from invalidating or corrupting what all of the other threads are doing. Having an extra "config service" thread that they all talk to probably won't make it any easier for you to implement that protocol, but it certainly will make your program use more memory, and maybe make it use a bit more CPU.

*If you anticipate that some day, in the future, the "system" might become a collection of cooperating processes, maybe processes running on different hosts in a "cluster," then implementing a configuration service (or at least, a configuration service API) today might save you from having to rewrite a lot of code tomorrow when you're ready to scale it up.