I'm using reqwest library for request sending in my old service written on rocket. The service job contains of two parts:

1. Take incoming request body with serde_json library
2. Send this body to another N services using reqwest library

But there is "bottleneck" problem here. When my service once get more then 500 request per second, scheduler trying to switch between them and causes huge CPU usage (almost 800% in docker stats)

It's like a worker pool problem..?

If anyone has any ideas on how to solve this problem, I would be very grateful.

UPD: code example

pub async fn handler(data: Json<Value>) {
   let data = data.take().to_string();
   for url in urls {
       match Client::new().post(url)
       .header("Content-Type", "application/json")
       .body(data)
       .send().await{
           ...
       }
   }
}

I'm using reqwest library for request sending in my old service written on rocket. The service job contains of two parts:

1. Take incoming request body with serde_json library
2. Send this body to another N services using reqwest library

But there is "bottleneck" problem here. When my service once get more then 500 request per second, scheduler trying to switch between them and causes huge CPU usage (almost 800% in docker stats)

It's like a worker pool problem..?

If anyone has any ideas on how to solve this problem, I would be very grateful.

UPD: code example

pub async fn handler(data: Json<Value>) {
   let data = data.take().to_string();
   for url in urls {
       match Client::new().post(url)
       .header("Content-Type", "application/json")
       .body(data)
       .send().await{
           ...
       }
   }
}

• rust
• reqwest

• 1 Can you add more details, if possible a minimal reproducible example to your question, it's not clear to me what problem you're trying to solve, do you receive errors because you send requests out too fast? Is your service not able to handle sufficiently many requests in a specific timeframe? … – cafce25 Commented Jan 29 at 13:37
• @cafce25 the last one, i can't handle many requests, as I said before. I 'm waiting until response came back. I need solution that will improve service efficiency. Like, before I have created three async background tasks per request and scheduler became mad, and in docker container CPU usage was ~700%. It happenes again when I sending 100 request/sec. Ah, this server written using rocket framework. – Grimlock Commented Jan 29 at 13:49
• How did you benchmark it (i.e. how do you know it's a scheduler problem)? How is the service configured? – cafce25 Commented Jan 29 at 15:26
• @cafce25 when I tested it manually, requests came periodically and all works fine. I also saw docker stats cpu usage and it was like 50-60%. Then I used a script that sends a lot of requests (thousand). And usage became enormous. When I removed async tasks creation, usage became normal (hello scheduler), but when I send another thousand (at once) and then 10-100 requests per second, usage became enormous again. Idk where is bottleneck. – Grimlock Commented Jan 29 at 16:06
• But it's quite expected that CPU usage rises when you have to process more requests, when the usage is as high as 50-60% with just a couple of requests then I'd expect a 10 higher usage for 10x the requests, so the usage of 800% would line up quite nicely with just an increased workload. – cafce25 Commented Jan 29 at 16:25

1 Answer 1

We have had this problem in production before. We were using reqwest::get() instead, but the problem is the same: you are creating a single client per request. Connection reuse/pooling happens at the client level, so if you create a client for each request, you cannot reuse connections at all. This results in:

• A DNS lookup per request.
• A new TCP connection per request.
• Likely a TLS handshake per request (if you're using https URLs, which you should be).

All of this overhead was enough to bring one of our services to its knees when it got very busy.

The solution is to create a single reqwest::Client and share it around. Note that internally, clients have shared ownership of a pool. This means you can cheaply .clone() a client and all clones will share the same connection pool.

There's two straightforward ways to implement this strategy:

• Create a single client somewhere and .clone() it around to the workers.

• Create a global LazyLock holding a client:

  static REQWEST_CLIENT: LazyLock<Client> = LazyLock::new(Client::new);
  

Note that if you want to enable in-process DNS response caching, you need to add the hickory-dns feature to your reqwest crate dependency, and enable this feature when you create the client. For example:

static REQWEST_CLIENT: LazyLock<Client> =
    LazyLock::new(|| Client::builder().hickory_dns(true).build().unwrap());