I have a scenario where I get thousands of records from the server as JSON and bind all records to the page. For each record, I am doing some calculations in jquery and binding data to UI. As record count is in 1000's the time take for calculation and binding data is more. Data on the page is bound at the end when all records calculations are done. Is there any option to bind data one by one or 10 by 10 and show binding on UI for that set. What I am trying to find is to execute $.each for 10 records at a time and append next set of 10 records to it and so on. any idea to make the page load faster? (Paging is not required for my requirement). Any clue can help.

<div id="keepFinalDataHere"></div>


$.each(data, function (i, record) {

 content += "<div>" + record.id + "</div><div>" + record.fromId + "</div><div>" + record.subject + "</div>";
        });

    $(content).appendTo('#keepFinalDataHere');

In above code, content is built by getting several thousands of records and once the content is built, then it is being bound to the div. I am looking for an option to get first 10 items bind the data to make sure that users feel like the page is loaded, and then APPEND remaining items in sets of 100 or so to existing list.

I have a scenario where I get thousands of records from the server as JSON and bind all records to the page. For each record, I am doing some calculations in jquery and binding data to UI. As record count is in 1000's the time take for calculation and binding data is more. Data on the page is bound at the end when all records calculations are done. Is there any option to bind data one by one or 10 by 10 and show binding on UI for that set. What I am trying to find is to execute $.each for 10 records at a time and append next set of 10 records to it and so on. any idea to make the page load faster? (Paging is not required for my requirement). Any clue can help.

<div id="keepFinalDataHere"></div>


$.each(data, function (i, record) {

 content += "<div>" + record.id + "</div><div>" + record.fromId + "</div><div>" + record.subject + "</div>";
        });

    $(content).appendTo('#keepFinalDataHere');

In above code, content is built by getting several thousands of records and once the content is built, then it is being bound to the div. I am looking for an option to get first 10 items bind the data to make sure that users feel like the page is loaded, and then APPEND remaining items in sets of 100 or so to existing list.

• javascript
• jquery

• Could you please share the bindDatasource ? – Ismail RBOUH Commented Jul 6, 2016 at 22:03
• Why paging is not good? Use live scrolling then. Show N rows, so user will think all is ready. When first data is ready render next pack etc. Use timeouts for example. – Sharikov Vladislav Commented Jul 6, 2016 at 22:04
• I am trying to code to show first 10 records so that user understand that page is loaded and slowly append next set to those 10 records. Any clue please to find out this logic. – Kurkula Commented Jul 6, 2016 at 22:06
• Ismail, I updated question with code. – Kurkula Commented Jul 6, 2016 at 22:08
• Another key point to consider. Instead of adding each item to the DOM individually, build up a buffer with the data for the chunk and add the chunk to the DOM as a whole. One of the slowest steps is the DOM manipulations. Reducing the number of times that you manipulate the DOM will speed up the process – Nick Harrison Commented Jul 15, 2016 at 14:28

5 Answers 5

In simple way you can do in chunks.

 <div id="keepFinalDataHere"></div>

<script>
//.../ 

var chunkSize = 50;//what ever you want or could be dynamic based on data size
var $keepFinalDataHere = $('#keepFinalDataHere');
$.each(data, function (i, record) {
  content += "<div>" + record.id + "</div><div>" + record.fromId + "</div><div>" + record.subject + "</div>";
     if(i % chunkSize === 0){ // content chunk is ready
          $keepFinalDataHere.append(content); // show records
          content = '';//reset the content
         }
    });
if(!(content === '')){//any leftOver records
   $keepFinalDataHere.append(content);
 }

If you want to keep the UI responsive and want to be able to execute code in between rendering a large amount of DOM elements, you'll have to use a timeout mechanism. You can do so by passing your render method to setTimeout.

Instead of adding the method to the stack and executing it immediately, setTimeout pushes the method to a task queue and only executes it once the current js stack has cleared.

The main steps of the method I propose:

1. Copy your data set to a temporary array
2. Use splice to remove the first n items from the array
3. Render the first n items to the DOM
4. if there are still items left, go to (2)

Here's the main part of the code, with ments, assuming:

• testData holds an array of data points
• createRow holds the logic to transform a data point to a rendered DOM element
• INITIAL_CHUNK_SIZE holds the number of rows you want to render without a timeout.
• DEFAULT_CHUNK_SIZE holds the number of rows each following loop has to render

The time out renderer (toRenderer):

var toRenderer = function(s) {
  // We need a copy because `splice` mutates an array
  var dataBuffer = [].concat(testData);

  var nextRender = function(s) {
    // Default value that can be overridden
    var chunkSize = s || DEFAULT_CHUNK_SIZE; 

    dataBuffer
      .splice(0, chunkSize)
      .forEach(createRow);

    if (dataBuffer.length) {
      setTimeout(nextRender);
    }
  };

  // Triggers the initial (not timed out) render
  nextRender(INITIAL_CHUNK_SIZE);
};

In the example below I've included a moving spinner to show how the render loop is able to hold a decent frame rate.

Note that the larger the DEFAULT_CHUNK_SIZE, the faster you'll have all your items rendered. The tradeoff: once one render chunk takes more than 1/60s, you'll loose your smooth frame rate.

// SETTINGS
var DATA_LENGTH = 10000;
var DEFAULT_CHUNK_SIZE = 100;
var INITIAL_CHUNK_SIZE = 10;

var list = document.querySelector("ul");
var createRow = function(data) {
  var div = document.createElement("div");
  div.innerHTML = data;
  list.appendChild(div);
};

// Blocking until all rows are rendered
var bruteRenderer = function() {
  console.time("Brute renderer total time:");
  testData.forEach(createRow);
  console.timeEnd("Brute renderer total time:");
}

// Pushes "render assignments" to the "task que"
var toRenderer = function(s) {
  console.time("Timeout renderer total time:");
  var dataBuffer = [].concat(testData);

  var nextRender = function(s) {
    var chunkSize = s || DEFAULT_CHUNK_SIZE;

    dataBuffer
      .splice(0, chunkSize)
      .forEach(createRow);

    if (dataBuffer.length) {
      setTimeout(nextRender);
    } else {
      console.timeEnd("Timeout renderer total time:");
    }
  };

  nextRender(INITIAL_CHUNK_SIZE);
};

// EXAMPLE DATA, EVENT LISTENERS:


// Generate test data
var testData = (function() {
  var result = [];
  for (var i = 0; i < DATA_LENGTH; i += 1) {
    result.push("Item " + i);
  }
  return result;
}());

var clearList = function() {
  list.innerHTML = "";
};

// Attach buttons
document.querySelector(".js-brute").addEventListener("click", bruteRenderer);
document.querySelector(".js-to").addEventListener("click", toRenderer);
document.querySelector(".js-clear").addEventListener("click", clearList);

button {
  display: inline-block;
  margin-right: .5rem;
}
.spinner {
  background: red;
  border-radius: 50%;
  width: 20px;
  height: 20px;
  animation-duration: 1s;
  animation-timing-function: linear;
  animation-direction: alternate;
  animation-name: move;
  animation-iteration-count: infinite;
}
@keyframes move {
  from {
    transform: translate3d(800%, 0, 0);
  }
  to {
    transform: translate3d(0, 0, 0);
  }
}
ul {
  height: 200px;
  overflow-y: scroll;
  background: #efefef;
  border: 1px solid #ccc;
}

<button class="js-brute">
  Inject rows brute force
</button>
<button class="js-to">
  Inject rows timeout
</button>
<button class="js-clear">
  clear list
</button>

<pre></pre>
<div class="spinner"></div>

<ul>

</ul>

If you have problems with the amount of data you had fetched from the server, you should found a way to limit here the array.

So your client code could handle just the proper amount of elements, just those you could show to the user.

If this is not possible, and you want to do all on client side, you should have a more plicated approach.

You have to save the pointer to the processed elements, and a variable with the amount of element to process (page num?).

And then use a for loop.

// Globally but not global
var cursor = 0

... 


for(var i = cursor; i < (cursor+pageNum); i++) {
    var element = myDataAsJsonFromApi[i];
    // ... do something here.
}

// check if pageNum elements is added..
cursor += pageNum

if (myDataAsJsonFromApi.length == cursor) {
 // load from server...
}

One option is to split your data buffer into chunks, so you can operate on some of the data at a time.

var data = [1,2,3,4,5,6,7,7,8,9,9,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,89];

(function () {

  var lastSliceStart = 0;

  function writeNext() {
    var length = 10;
    var chunk = $(data).slice(lastSliceStart, lastSliceStart+length);

    $(chunk).each((key, item) => {
      console.log(item);
    });

    lastSliceStart += length;

    if (lastSliceStart < data.length) {
      setTimeout(writeNext, 500); // Wait .5 seconds between runs
    }
  }

  writeNext();

})();

https://jsfiddle/bogfdmfb/1/

Build a queue, process queue for few items at a time, show progress, process next items in queue an so on.

//your app
App = {
    data: [] //set your JSON dataSource here
}

//define Task
Task = function () {        
    this.buildQueue(10);
};
Task.prototype = {
    buildQueue: function (size) {
        var data_count = App.data.length; //length of your datasource
        this.queue = [];                        
        // fill the queue
        var lastIndex = 0;
        var current_index = size;
        var c = true;
        while (c) {
            if (current_index >= data_count - 1) {
                current_index = data_count;
                c = false;
            }
            this.queue.push([lastIndex, current_index - 1]);
            lastIndex = current_index;
            current_index += size;
        }
        /* If size is 10, array would be [[0,9], [10,19], [20,29]....and so on], The smaller the size, better progress / percentage variation  / loading on ui display */
    },
    doNext: function () {
        if (this.queue.length == 0) {
            this.end();
            return;
        }            
        var row = this.queue.shift(); //stack is LIFO, queue is FIFO, this.queue.pop()
        try {
            this.processQueue(App.data, row[0], row[1]); //pass dataSource, and indexes of array, loop in processQueue function for indexes passed
        } catch (e) {                
            return;
        }
        this.incrementProgress(row[1] / App.data.length); //progress on ui
        // do next
        var _self = this;
        setTimeout(function () {
            _self.doNext();
        }, 1);
    },
    incrementProgress: function (percent) {
        var $progress = $('#percent');
        percent = Math.ceil(percent * 100);
        percent = percent + '%';
        $progress.text(percent);
    },
    start: function () {            
        $('#percent').show();
        this.doNext(); //initiate loop
    },
    end: function () {            
        $('#percent').hide();            
    },
    processQueue: function (data, start, end) {            
        for (var i = start; i <= end; i++) {                
            var dataObj = data[i];
            //use the data here, update UI so user sees something on screen
        }
    }
};

//initialize an instance of Task
var _task = new Task(task); 
_task.start();