Beginning with express and mongoose i often need to do some batch operations on collections. However it usually involves callbacks which is a pain given how concurrency is coded in nodejs. so basically
//given a collection C
var i = 0;
var doRecursive = function(i){
if(i<C.length){
C[i].callAsync(err,result){
i=+1;
return doRecursive(i);
}
}else{
return done();
}
}
doRecursive(i);
Now i dont remember what is the max stack before i get a stackover flow with node , but i guess with 10 000 elements , it wont do. I wonder if there are other ways to handle this, if yes , what are they? thanks
Beginning with express and mongoose i often need to do some batch operations on collections. However it usually involves callbacks which is a pain given how concurrency is coded in nodejs. so basically
//given a collection C
var i = 0;
var doRecursive = function(i){
if(i<C.length){
C[i].callAsync(err,result){
i=+1;
return doRecursive(i);
}
}else{
return done();
}
}
doRecursive(i);
Now i dont remember what is the max stack before i get a stackover flow with node , but i guess with 10 000 elements , it wont do. I wonder if there are other ways to handle this, if yes , what are they? thanks
Share Improve this question asked Jul 12, 2013 at 18:58 mpmmpm 20.2k7 gold badges51 silver badges56 bronze badges 2-
1
Well, asynchronous functions will help with stack overflows as the callback will be in a different call stack. But, by breaking out into a different stack, it renders
return
rather useless. – Jonathan Lonowski Commented Jul 12, 2013 at 19:09 - 1 take a look at async – go-oleg Commented Jul 12, 2013 at 19:16
2 Answers
Reset to default 5If the goal is to iterate an collection asynchronously, there are numerous control flow libraries available.
A good example is async
and its reduce
function:
async.reduce(C, 0, function (memo, item, callback) {
item.callAsync(function (err, result) {
if (err) {
callback(err);
} else {
callback(null, memo + result);
}
});
}, function (err, result) {
// ...
});
Note: It's not entirely clear what value you wanted to get from doRecursion
, so this just uses addition for an example.
i think you can simply self-iterate instead of true recursion, since you're not drilling into a deep object:
function doRecursive (C, i){
i=i||0;
if(i<C.length){
C[i].callAsync(err, function(result){
doRecursive(C, ++i);
});
}else{
done();
}
};
doRecursive(C);
this does not create a tall stack if the code functions as labeled. i localized C so that it executes faster and is potentially re-usable on other collections. the pattern also makes it easy to defer it for long-running operations, just by changing
doRecursive(C, ++i);
to
setTimeout( doRecursive.bind(this, C, ++i), 50 );