I'm curious on how API retry and timeouts should be implemented. Sometimes simply awaiting an api call then catching any error that e up isn't sufficient. If there's a chain of async requests I need to make, like so:

await client
  .callA()
  .then(async () => await callB())
  .then(async () => await callC())
  .catch(err => console.error(err));

and if one of the promise fails midchain, I want to attempt to the request again after a few seconds until attempts run out.

This is my attempt at making a retry wrapper.

async function retry (fn, undo, attempts, wait = 5000) {
  await fn().catch(async (err) => {

    console.error(err.message + `\n retrying in ${wait/1000} seconds...`);

    if (attempts !== 0) {
      // async timeout
      await new Promise((resolve) => {
        setTimeout(() => resolve(retry(fn, undo, attempts - 1)), wait);
      })
    } else {
      await undo()
    }
  })
}

await retry(calls, undoCalls, 10)

callA -> callB -> callC

Say callA() succeeds, but callB() fails, I want the wrapper to retry callB() at an interval instead of starting over again. Then either:

1. callB() eventually succeeds within the allowed attempts, move onto callC().
2. callB() runs out of attempts, call undoCallA() to revert the changes previously made.

Repeat the above until end of chain.

I'd like some insight on how this is implemented or if there's a library that does something similar. Thanks!

I'm curious on how API retry and timeouts should be implemented. Sometimes simply awaiting an api call then catching any error that e up isn't sufficient. If there's a chain of async requests I need to make, like so:

await client
  .callA()
  .then(async () => await callB())
  .then(async () => await callC())
  .catch(err => console.error(err));

and if one of the promise fails midchain, I want to attempt to the request again after a few seconds until attempts run out.

This is my attempt at making a retry wrapper.

async function retry (fn, undo, attempts, wait = 5000) {
  await fn().catch(async (err) => {

    console.error(err.message + `\n retrying in ${wait/1000} seconds...`);

    if (attempts !== 0) {
      // async timeout
      await new Promise((resolve) => {
        setTimeout(() => resolve(retry(fn, undo, attempts - 1)), wait);
      })
    } else {
      await undo()
    }
  })
}

await retry(calls, undoCalls, 10)

callA -> callB -> callC

Say callA() succeeds, but callB() fails, I want the wrapper to retry callB() at an interval instead of starting over again. Then either:

1. callB() eventually succeeds within the allowed attempts, move onto callC().
2. callB() runs out of attempts, call undoCallA() to revert the changes previously made.

Repeat the above until end of chain.

I'd like some insight on how this is implemented or if there's a library that does something similar. Thanks!

• javascript
• typescript
• asynchronous
• async-await
• promise

• What is not working about your retry method? – Jamiec Commented May 20, 2021 at 9:20
• retry(fn, attempts - 1) is missing parameter undo, also you are awaiting the promise to resolve (await calls()) and then calling await fn(). Just pass calls, undoCalls as parameter and I guess it will work – stacj Commented May 20, 2021 at 9:30
• @Jamiec It works for a single call, but it restarts the call chain on fail. – Terry Commented May 20, 2021 at 9:32
• What if undoCallA fails? – trincot Commented May 20, 2021 at 9:41
• 1 .then(async () => await callB()) is an anti-pattern. it's the same as .then(_ => callB()) – Mulan Commented May 21, 2021 at 3:15

3 Answers 3

Functions should be simple and do just one thing. I would start with a generic sleep -

const sleep = ms =>
  new Promise(r => setTimeout(r, ms))

Using simple functions we can build more sophisticated ones, like timeout -

const timeout = (p, ms) =>
  Promise.race([ p, sleep(ms).then(_ => { throw Error("timeout") }) ])

Now let's say we have a task, myTask that takes up to 4 seconds to run. It returns successfully if it generates an odd number. Otherwise it rejects, "X is not odd" -

async function myTask () {
  await sleep(Math.random() * 4000)
  const x = Math.floor(Math.random() * 100)
  if (x % 2 == 0) throw Error(`${x} is not odd`)
  return x
}

Now let's say we want to run myTask with a timeout of two (2) seconds and retry a maximum of three (3) times -

retry(_ => timeout(myTask(), 2000), 3)
  .then(console.log, console.error)

Error: 48 is not odd (retry 1/3)
Error: timeout (retry 2/3)
79

It's possible myTask could produce an odd number on the first attempt. Or it's possible that it could exhaust all attempts before emitting a final error -

Error: timeout (retry 1/3)
Error: timeout (retry 2/3)
Error: 34 is not odd (retry 3/3)
Error: timeout
Error: failed after 3 retries

Now we implement retry. We can use a simple for loop -

async function retry (f, count = 5, ms = 1000) {
  for (let attempt = 1; attempt <= count; attempt++) {
    try {
      return await f()
    }
    catch (err) {
      if (attempt <= count) {
        console.error(err.message, `(retry ${attempt}/${count})`)
        await sleep(ms)
      }
      else {
        console.error(err.message)
      }
    }
  }
  throw Error(`failed after ${count} retries`)
}

Now that we see how retry works, let's write a more plex example that retries multiple tasks -

async function pick3 () {
  const a = await retry(_ => timeout(myTask(), 3000))
  console.log("first pick:", a)
  const b = await retry(_ => timeout(myTask(), 3000))
  console.log("second pick:", b)
  const c = await retry(_ => timeout(myTask(), 3000))
  console.log("third pick:", c)
  return [a, b, c]
}

pick3()
  .then(JSON.stringify)
  .then(console.log, console.error)

Error: timeout (retry 1/5)
Error: timeout (retry 2/5)
first pick: 37
Error: 16 is not odd (retry 1/5)
second pick: 13
Error: 60 is not odd (retry 1/5)
Error: timeout (retry 2/5)
third pick: 15
[37,13,15]

Expand the snippet below to verify the result in your browser -

const sleep = ms =>
  new Promise(r => setTimeout(r, ms))

const timeout = (p, ms) =>
  Promise.race([ p, sleep(ms).then(_ => { throw Error("timeout") }) ])

async function retry (f, count = 5, ms = 1000) {
  for (let attempt = 0; attempt <= count; attempt++) {
    try {
      return await f()
    }
    catch (err) {
      if (attempt < count) {
        console.error(err.message, `(retry ${attempt + 1}/${count})`)
        await sleep(ms)
      }
      else {
        console.error(err.message)
      }
    }
  }
  throw Error(`failed after ${count} retries`)
}

async function myTask () {
  await sleep(Math.random() * 4000)
  const x = Math.floor(Math.random() * 100)
  if (x % 2 == 0) throw Error(`${x} is not odd`)
  return x
}

async function pick3 () {
  const a = await retry(_ => timeout(myTask(), 3000))
  console.log("first", a)
  const b = await retry(_ => timeout(myTask(), 3000))
  console.log("second", b)
  const c = await retry(_ => timeout(myTask(), 3000))
  console.log("third", c)
  return [a, b, c]
}

pick3()
  .then(JSON.stringify)
  .then(console.log, console.error)

And because timeout is decoupled from retry, we can achieve different program semantics. By contrast, the following example not timeout individual tasks but will retry if myTask returns an even number -

async function pick3 () {
  const a = await retry(myTask)
  const b = await retry(myTask)
  const c = await retry(myTask)
  return [a, b, c]
}

And we could now say timeout pick3 if it takes longer than ten (10) seconds, and retry the entire pick if it does -

retry(_ => timeout(pick3(), 10000))
  .then(JSON.stringify)
  .then(console.log, console.error)

This ability to bine simple functions in a variety of ways is what makes them more powerful than one big plex function that tries to do everything on its own.

Of course this means we can apply retry directly to the example code in your question -

async function main () {
  await retry(callA, ...)
  await retry(callB, ...)
  await retry(callC, ...)
  return "done"
}

main().then(console.log, console.error)

You can either apply timeout to the individual calls -

async function main () {
  await retry(_ => timeout(callA(), 3000), ...)
  await retry(_ => timeout(callB(), 3000), ...)
  await retry(_ => timeout(callC(), 3000), ...)
  return "done"
}

main().then(console.log, console.error)

Or apply timeout to each retry -

async function main () {
  await timeout(retry(callA, ...), 10000)
  await timeout(retry(callB, ...), 10000)
  await timeout(retry(callC, ...), 10000)
  return "done"
}

main().then(console.log, console.error)

Or maybe apply timeout to the entire process -

async function main () {
  await retry(callA, ...)
  await retry(callB, ...)
  await retry(callC, ...)
  return "done"
}

timeout(main(), 30000).then(console.log, console.error)

Or any other bination that matches your actual intention!

I just wouldn't be trying to chain these together. It gets too plex too quickly, instead have a simpler retry logic, and use try...catch to determine if something went wrong and call the appropriate undo method. Kind of in this pattern

await retry(callA, 10, 5000)
try{
  await retry(callB, 10, 5000);
}
catch {
   await undoCallA()
}

The retry method for this pattern can be much simpler, just retrying for the number of attempts allowed and rejecting if that number is exhausted. This will then raise an error back to the caller allowing it to be catched.

A simple example:

const delay = (ms) => new Promise(resolve => setTimeout(resolve,ms));

async function retry(fn, attempts, wait = 1000) {
  let count=0;
  while(count++<attempts) {
     try{        
        const result = await fn();
        return result;
     }
     catch(e){
       if(count == attempts){
          console.log("throw");
          throw e;
       }
       await delay(wait);
       console.log("retry");
     }         
  }
}


function callA(){
  return new Promise( (resolve,reject) => {
    if(Math.random() < 0.6) reject("random fail A");
    else resolve("CallA")
  })
}

async function undoCallA(){ console.log("Undo A") }

function callB(){
  return new Promise( (resolve,reject) => {
    if(Math.random() < 0.99) reject("random fail B");
    else resolve("CallB")
  })
}

(async function(){
  console.log(await retry(callA,10));
  
  try{
    console.log(await retry(callB,10))  
  }
  catch(e) {
    console.log(e);
    await undoCallA();  
  }
  
})()

Here's a Live Demo that uses a custom Promise:

import { CPromise } from "c-promise2";

const callA = async () => console.log(`Call A`);
const callB = async () => {
  console.log(`Call B`);
  throw Error("ooops");
};
const callC = async () => console.log(`Call C`);

const undoA = async () => console.log(`Undo A`);
const undoB = async () => console.log(`Undo B`);
const undoC = async () => console.log(`Undo C`);

const promise = CPromise.retry(() => callA())
  .finally((v, isRejected) => isRejected && undoA())
  .then(() =>
    CPromise.retry(() => callB()).finally(
      (v, isRejected) => isRejected && undoB()
    )
  )
  .then(() =>
    CPromise.retry(() => callC()).finally(
      (v, isRejected) => isRejected && undoC()
    )
  )

  .then(
    (v) => console.log(`Done: ${v}`),
    (err) => console.error(`Fail: ${err}`)
  );

// promise.pause()
// promise.resume()
// promise.cancel()

Console log:

Call A 
Call B 
Call B 
Call B 
Undo B 
Error: ooops

One more example with retrying the axios request::

import { CPromise } from "c-promise2";
import cpAxios from "cp-axios";

CPromise.retry(
  (attempt) => {
    console.log(`Attempt [${attempt}]`);
    return cpAxios(url).timeout(attempt * 1000 + 500);
  },
  { retries: 3, delay: (attempt) => attempt * 1000 }
).then(
  (response) => console.log(`Response: ${JSON.stringify(response.data)}`),
  (err) => console.warn(`Fail: ${err}`)
);