My app is fetching all of a User's Google Contacts from the Google Contacts API on the client side. This usually results in between 1 - 2000 different JSON objects. When those are received, my app iterates through them, reformats each Contact Object, and then attempts to save the reformatted Contact Object to my database via a POST request. The result of this is a large number ( 1 - 2000 ) of AJAX calls firing on the Client Side, but they stall out after 5-10 of them. What is the best way to handle all of these AJAX requests, or saving such a large amount of data all at once?

Here is a summarized version of my current code:

// gContacts.length = 722

$(gContacts).each(function(index, contact) {

         // Reformat each contact object to fit into my database
         var newContact = {}
         newContact.title = // String
         newContact.emails = // Object featuring different emails
         newContact.phone_numbers = // Object featuring different phonenumbers

         // Save to Database via Backbone
                    var newContact = new App.Collections.Contacts()
                    newContact.create({
                        title           : newContact.title,
                        emails          : newContact.emails,
                        phone_numbers   : newContact.phone_numbers
                    }, {
                        success: function (response) {

                        },
                        error: function (model, xhr) {
                            var errors = $.parseJSON(xhr.responseText).errors
                            console.log(errors)
                        }
                    }) // End .save
}); // End of .each()

My app is fetching all of a User's Google Contacts from the Google Contacts API on the client side. This usually results in between 1 - 2000 different JSON objects. When those are received, my app iterates through them, reformats each Contact Object, and then attempts to save the reformatted Contact Object to my database via a POST request. The result of this is a large number ( 1 - 2000 ) of AJAX calls firing on the Client Side, but they stall out after 5-10 of them. What is the best way to handle all of these AJAX requests, or saving such a large amount of data all at once?

Here is a summarized version of my current code:

// gContacts.length = 722

$(gContacts).each(function(index, contact) {

         // Reformat each contact object to fit into my database
         var newContact = {}
         newContact.title = // String
         newContact.emails = // Object featuring different emails
         newContact.phone_numbers = // Object featuring different phonenumbers

         // Save to Database via Backbone
                    var newContact = new App.Collections.Contacts()
                    newContact.create({
                        title           : newContact.title,
                        emails          : newContact.emails,
                        phone_numbers   : newContact.phone_numbers
                    }, {
                        success: function (response) {

                        },
                        error: function (model, xhr) {
                            var errors = $.parseJSON(xhr.responseText).errors
                            console.log(errors)
                        }
                    }) // End .save
}); // End of .each()

• javascript
• jquery
• ajax
• backbone.js
• express

• 1 Does this need to be done client-side? Seems like it would be better suited for the server. – Jason P Commented Nov 7, 2013 at 22:11
• 4 Why not perform a single request that incorporates all the data? – zerkms Commented Nov 7, 2013 at 22:13
• 1 Don't make 1000+ calls - it's wasteful of network resources and slow. Process the entire list into one JSON array and post the whole lot at once. – user1864610 Commented Nov 7, 2013 at 22:13
• 1 Seems like it should be made in a single request like zerkms said. The overhead of creating and destroying a connection is killing you. – Walt Commented Nov 7, 2013 at 22:15
• 1 Why are you iterating over all 700 contacts and creating a new contact for each? Why aren't you creating all 700 at once in a single request? – Kevin B Commented Nov 7, 2013 at 22:19

2 Answers 2

I would make a server-side action that takes a whole list of contact objects. Then on the client side, just format them and add them all to an array, and once that's done send the array. The way you're doing it incurs a lot of unnecessary network overhead.

Most modern browsers limit simultaneous requests to 6 per domain. Requests made over and above that are blocked until one of the active requests pletes, freeing up a connection for the next request in the queue. If the active requests take long enough (read, "too long"), the pending requests will timeout, like you're seeing.

To handle this properly, you really need to roll your own requests management code. Issue only as many requests at once as the browser can handle. As the requests plete, or error out, or time out, issue the next request. This requires maintaining a queue of pending requests and accurately detecting when requests plete or finish, but should do the trick.

One problem you may run into is that your POST requests will be peting with all other requests to your server. For example, you may or may not have noticed that if you open another page to your website while these POST requests are going over the wire, your website never loads, or loads very slowly.

As others have said, batching these updates into a single request is probably a better solution.