I'm using this bit of code that contains four asynchronous functions.
I need them to execute in a strict order.
How do I go about doing that where they will execute in the order given in the sample?
My use case is in a Lambda and I have access to async.
function scanProducts() {
dynamoClient.scan(productParams, function (err, data) {
});
}
function scanCoupons() {
dynamoClient.scan(couponsParams, function (err, data) {
});
}
function scanRetailers() {
dynamoClient.scan(retailerParams, function (err, data) {
});
}
function sendEmail(ses) {
var email = {
"Source": "[email protected]",
"Template": "test-template",
"Destination": {
"ToAddresses": ["[email protected]"]
},
"TemplateData": `{}`
}
ses.sendTemplatedEmail(email);
}
I'm using this bit of code that contains four asynchronous functions.
I need them to execute in a strict order.
How do I go about doing that where they will execute in the order given in the sample?
My use case is in a Lambda and I have access to async.
function scanProducts() {
dynamoClient.scan(productParams, function (err, data) {
});
}
function scanCoupons() {
dynamoClient.scan(couponsParams, function (err, data) {
});
}
function scanRetailers() {
dynamoClient.scan(retailerParams, function (err, data) {
});
}
function sendEmail(ses) {
var email = {
"Source": "[email protected]",
"Template": "test-template",
"Destination": {
"ToAddresses": ["[email protected]"]
},
"TemplateData": `{}`
}
ses.sendTemplatedEmail(email);
}
- use Continuation-Passing Style? – Robert Harvey Commented Jul 29, 2019 at 23:02
- Step 1: Convert your asynchronous operations to return promises so you can use promises to manage the sequencing of your operations. Step 2: Learn how to sequence operations tracked with promises. A reference on the topic: How to synchronize a sequence of promises. – jfriend00 Commented Jul 30, 2019 at 0:01
3 Answers
Reset to default 5I'd convert the dynamoClient.scan to a Promise-based function, and then await each call of it, eg:
const dynamoClientScanProm = (params) => new Promise((resolve, reject) => {
dynamoClient.scan(params, function (err, data) {
if (err) reject(err);
else resolve(data);
});
});
// ...
// in an async function:
try {
await dynamoClientScanProm(productParams);
await dynamoClientScanProm(couponsParams);
await dynamoClientScanProm(retailerParams);
// promisify/await this too, if it's asynchronous
ses.sendTemplatedEmail(email);
} catch(e) {
// handle errors
}
It's not clear if you need to use the result of the calls, but if you do need the result and don't just need to wait for the Promise to resolve, assign to a variable when awaiting, eg
const productResults = await dynamoClientScanProm(productParams);
That said, if the results aren't being used by the other calls of dynamoClientScanProm, it would make more sense to run all calls in parallel (using Promise.all), rather than in series, so that the whole process can be pleted sooner.
Answer:
You can use the Symbol.iterator in accordance with for await to perform asynchronous execution of your promises. This can be packaged up into a constructor, in the example case it's called Serial (because we're going through promises one by one, in order)
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
What is the above?
- It's a constructor called
Serial. - It takes as an argument an array of Functions that return Promises.
- The functions are stored in
Serial.promises - It has an empty array stored in
Serial.resolved- this will store the resolved promise requests. - It has two methods:
addPromise: Takes a Function that returns a Promise and adds it toSerial.promisesresolve: Asynchronously calls a customSymbol.iterator. Thisiteratorgoes through every single promise, waits for it to be pleted, and adds it toSerial.resolved. Once this is pleted, it returns a map function that acts on the populatedSerial.resolvedarray. This allows you to simply callresolveand then provide a callback of what to do with the members in the response. If you return the promise to that function, you can pass athenfunction to be given the whole array.
An Example:
promises.resolve((resolved_request) => {
//do something with each resolved request
return resolved_request;
}).then((all_resolved_requests) => {
// do something with all resolved requests
});
The below example shows how this can be used to great effect, whether you want something to happen on each individual resolution, or wait until everything is pleted.
Notice that they will always be in order. This is evident by the fact that the first timer is set with the highest ms count. The second Promise will not even begin until the first has pleted, the third won't begin before the second finishes, etc.
That brings me to an important point. Though having your Promises Serialized in order is effective, it's important to realize that this will delay your responses for your data if they take any one of them takes any amount of time. The beauty of Parallel is that, if all goes well, all requests take a shorter amount of time to plete. Something like Serialization is great for if an application has multiple required requests, and the whole thing will fail if one is not available, or if one item relies on another(pretty mon).
//helpers
let log = console.log.bind(console),
promises = Serial(),
timer = (tag, ms) => () => new Promise(res => {
setTimeout(() => {
res("finished " + tag);
}, ms) });
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
promises.addPromise(timer(1, 3000));
promises.addPromise(timer(2, 1000));
promises.addPromise(timer(3, 2000));
promises
.resolve(msg => ( log(msg), msg) )
.then((plete) => log("everything is plete: " + plete));How does it work?
By using an iterator that calls each promise one by one, we can be certain that they are received in order.
Although many people don't realize this Symbol.iterator is much more powerful than standard for loops. This is for two big reasons.
The first reason, and the one that is applicable in this situation, is because it allows for asynchronous calls that can affect the state of the applied object.
The second reason is that it can be used to provide two different types of data from the same object. A.e. You may have an array that you would like to read the contents of:
let arr = [1,2,3,4];
You can use a for loop or forEach to get the data:
arr.forEach(v => console.log(v));
// 1, 2, 3, 4
But if you adjust the iterator:
arr[Symbol.iterator] = function* () {
yield* this.map(v => v+1);
};
You get this:
arr.forEach(v => console.log(v));
// 1, 2, 3, 4
for(let v of arr) console.log(v);
// 2, 3, 4, 5
This is useful for many different reasons, including timestamping requests/mapping references, etc. If you'd like to know more please take a look at the ECMAScript Documentation: For in and For Of Statements
Use:
It can be used by calling the constructor with an Array of functions that return Promises. You can also add Function Promises to the Object by using
new Serial([])
.addPromise(() => fetch(url))
It doesn't run the Function Promises until you use the .resolve method.
This means that you can add promises ad-hoc if you'd like before you do anything with the asynchronous calls. A.e. These two are the same:
With addPromise:
let promises = new Serial([() => fetch(url), () => fetch(url2), () => fetch(url3)]);
promises.addPromise(() => fetch(url4));
promises.resolve().then((responses) => responses)
Without addPromise:
let promises = new Serial([() => fetch(url), () => fetch(url2), () => fetch(url3), () => fetch(url4)])
.resolve().then((responses) => responses)
Adjusting your Code:
Below is an Example of adjusting your code to do things in order. Thing is, you didn't really provide a whole lot of starter code so I substituted your scan function for the timer function I've used in previous examples.
To get this functioning using your code, all you would have to do is return a Promise from your scan function, and it will work perfectly :)
function Serial(promises = []) {
return {
promises,
resolved: [],
addPromise: function(fn) {
promises.push(fn);
},
resolve: async function(cb = i => i, err = (e) => console.log("trace: Serial.resolve " + e)) {
try {
for await (let p of this[Symbol.iterator]()) {}
return this.resolved.map(cb);
} catch (e) {
err(e);
}
},
[Symbol.iterator]: async function*() {
this.resolved = [];
for (let promise of this.promises) {
let p = await promise().catch(e => console.log("trace: Serial[Symbol.iterator] ::" + e));
this.resolved.push(p);
yield p;
}
}
}
}
const timer = (tag, ms) => new Promise(res => {
setTimeout(() => {
res("finished " + tag);
}, ms)
});
function scanProducts() {
return timer("products", 3000);
}
function scanCoupons() {
return timer("coupons", 1000);
}
async function scanRetailers() {
return timer("retailers", 2500);
}
function sendEmail(ses) {
var email = {
"Source": "[email protected]",
"Template": "test-template",
"Destination": {
"ToAddresses": ["[email protected]"]
},
"TemplateData": `{}`
}
ses.sendTemplatedEmail(email);
}
let promises = Serial([scanProducts, scanCoupons, scanRetailers]);
promises.resolve().then(resolutions => console.log(resolutions));Hope this helps! Happy Coding!
use async-series. It run a series of callbacks in sequence, as simply as possible.
series([
function(done) {
console.log('first thing')
done()
},
function(done) {
console.log('second thing')
done(new Error('another thing'))
},
function(done) {
// never happens, because "second thing"
// passed an error to the done() callback
}
], function(err) {
console.log(err.message) // "another thing"
})