The problem is I have a tasklist of APIs but the issue is I can't hit more than 3 APIs at once simultaneously from my endpoint to the server(it's the server restriction that it will not respond back to the same requested URL which has more than 3 API called in processing). So I have found a way around to achieve this task but there is a better approach that I know but have not any clue about that how to do but i found a simple solution which is sequentially hit API and wait for the response then hit another but this takes too much time

More optimized solution if N = 3, then I will execute 'task1', 'task2', and 'task3' in the taskList concurrently. Once task2 finishes, you then execute 'task4', and so on. This is the solution that I want to achieve without using any library.

const taskList = [
  "task1",
  "task2",
  "task3",
  "task4",
  "task5",
  "task6",
  "task7",
  "task8",
  "task9",
  "task10",
];

const dummyAPICalled = (task) => {
  console.log(task, "started");
  return new Promise(function (resolve, reject) {
    setTimeout(function () {
      console.log(task, "ended");
      resolve(true);
    }, Math.random() * 20000);
  });
};

Hit one by one API's called and wait for the responses.

for await (const task of taskList) {
    await dummyAPICalled(task);
}

Hit all APIs at once parallel but didn't get a response because of server requirement

const promises = taskList.map((task) => dummyAPICalled(task));

The problem is I have a tasklist of APIs but the issue is I can't hit more than 3 APIs at once simultaneously from my endpoint to the server(it's the server restriction that it will not respond back to the same requested URL which has more than 3 API called in processing). So I have found a way around to achieve this task but there is a better approach that I know but have not any clue about that how to do but i found a simple solution which is sequentially hit API and wait for the response then hit another but this takes too much time

More optimized solution if N = 3, then I will execute 'task1', 'task2', and 'task3' in the taskList concurrently. Once task2 finishes, you then execute 'task4', and so on. This is the solution that I want to achieve without using any library.

const taskList = [
  "task1",
  "task2",
  "task3",
  "task4",
  "task5",
  "task6",
  "task7",
  "task8",
  "task9",
  "task10",
];

const dummyAPICalled = (task) => {
  console.log(task, "started");
  return new Promise(function (resolve, reject) {
    setTimeout(function () {
      console.log(task, "ended");
      resolve(true);
    }, Math.random() * 20000);
  });
};

Hit one by one API's called and wait for the responses.

for await (const task of taskList) {
    await dummyAPICalled(task);
}

Hit all APIs at once parallel but didn't get a response because of server requirement

const promises = taskList.map((task) => dummyAPICalled(task));

• javascript
• multithreading
• concurrency

4 Answers 4

Concurrent Promises

Since you want concurrent API requests, you need to create multiple instances before the previous instances resolve. When each individual request resolves, you can make another API request per instance. You can group multiple promises together with Promise.all that resolves when all of the instances resolves.

It could look something like this:

function callTasks(concurrentLimit) {
  var currentTask = 0;
  async function createTaskInstance() {
    while (currentTask < taskList.length) {
      // currentTask is a variable that's shared across
      // all of the task instances
      await dummyAPICalled(taskList[currentTask++]);
    }
  }
  var tasks = [];
  for (let i = 0; i < concurrentLimit; i++) {
    tasks.push(createTaskInstance());
  }
  return Promise.all(tasks);
  // or alternatively:
  // return Promise.all([...Array(concurrentLimit)].map(createTaskInstance));
}

For example, using code from the question:

const taskList = [
  "task1",
  "task2",
  "task3",
  "task4",
  "task5",
  "task6",
  "task7",
  "task8",
  "task9",
  "task10",
];

const dummyAPICalled = (task) => {
  console.log(task, "started");
  return new Promise(function (resolve, reject) {
    setTimeout(function () {
      console.log(task, "ended");
      resolve(true);
    }, Math.random() * 10000);
  });
};

function callTasks(concurrentLimit) {
  var currentTask = 0;
  async function createTaskInstance() {
    while (currentTask < taskList.length) {
      await dummyAPICalled(taskList[currentTask++]);
    }
  }
  var tasks = [];
  for (let i = 0; i < concurrentLimit; i++) {
    tasks.push(createTaskInstance());
  }
  return Promise.all(tasks);
}

callTasks(3).then(()=>console.log('all done'));

Instead of reinventing the wheel i suggest using an existing library. I remend th elibrarhy p-limit. It does specifically what you asked for in that it can limit the concurrent number of any asynchronous function callbacks, that includes external APIs requests.

import pLimit from 'p-limit';

const limit = pLimit(CONCURRENT_REQUESTS);

const input = [
    limit(() => fetchSomething('foo')),
    limit(() => fetchSomething('bar')),
    limit(() => doSomething())
];

// Only $CONCURRENT_REQUESTS promises are run at once
const result = await Promise.all(input);

Firstly, we want to call the server several times. Say we have callAPI that would call the server. Without such limit we would do.

const params = [task1, task2, ... ]; // more than 3
const responses = await Promise.all(params.map(param => callAPI(param)));

Now we need an agent to keep count of concurrent calls being made and not exceed it.

const agent = limit(callAPI, 3);
const responses = await Promise.all(params.map(param => agent(param)));

We want agent to run callAPI as many as possible (but at most 3) concurrently. So

function limit(callAPI, max) {
  let keepCount = 0;
  return async function do(param) {
    if (keepCount < max) {
      keepCount++;
      const res = await callAPI(param);
      keepCount--;
      return res;
    } else {
      return new Promise((resolve, reject) => {
        setTimeout(async () => {
          try {
            const res = await do(param);
            resolve(res);
          } catch (err) {
            reject(err);
          }
        }, 0);
      });
    }
  };
}

We need that setTimeout and promise construction to not block the event-loop.

Bottleneck is another package that can be used to stay within the rate limit and allowed concurrency of an API. Just define a limiter for each API you use and set it up with the corresponding options.

import Bottleneck from "bottleneck";

const limiter = new Bottleneck({
    maxConcurrent: 3, // Max 3 concurrent calls
    minTime: 333      // Rate limit max 3 calls/second
});

// Add a job
limiter.schedule(() => client.get(url))
    .then(response => console.log(response.body));