I'm trying to understand a few things about RxJs. What I would like to do is consume some JSON data and immediately begin to render that data on the DOM as it's ing in. I've setup the stream request, response, and display. It's outputting every just fine but it's doing it all at once and not over time.

I want to start showing the data on the page as its ing in, instead of waiting for the whole file to plete then show at once which would create a long wait time.

//Cache the selector
var $resultList = $('.results');

//Gets the JSON (this will not be a static file,just for testing)
var requestStream = Rx.Observable.just("/results.json");

var responseStream = requestStream
    .flatMap(function(requestUrl) {
            return Rx.Observable.fromPromise(jQuery.getJSON(requestUrl))
             });

var displayStream = responseStream.subscribe(
    function(response) {
    //This maps to a Handlebars Template and puts it on the DOM
    $resultList.html(piledTemplate(response)); 
            },
            function(err) {
                    console.log('Error: %s', err);
             },
             function() {
                    console.log('Completed');
             });




//Sample of the data from the JSON file
Object{
    beginIndex: "1"
    catId: "111"
    endIndex: "1"
    products: Array[100]

}

I'm trying to understand a few things about RxJs. What I would like to do is consume some JSON data and immediately begin to render that data on the DOM as it's ing in. I've setup the stream request, response, and display. It's outputting every just fine but it's doing it all at once and not over time.

I want to start showing the data on the page as its ing in, instead of waiting for the whole file to plete then show at once which would create a long wait time.

//Cache the selector
var $resultList = $('.results');

//Gets the JSON (this will not be a static file,just for testing)
var requestStream = Rx.Observable.just("/results.json");

var responseStream = requestStream
    .flatMap(function(requestUrl) {
            return Rx.Observable.fromPromise(jQuery.getJSON(requestUrl))
             });

var displayStream = responseStream.subscribe(
    function(response) {
    //This maps to a Handlebars Template and puts it on the DOM
    $resultList.html(piledTemplate(response)); 
            },
            function(err) {
                    console.log('Error: %s', err);
             },
             function() {
                    console.log('Completed');
             });




//Sample of the data from the JSON file
Object{
    beginIndex: "1"
    catId: "111"
    endIndex: "1"
    products: Array[100]

}

• javascript
• json
• rxjs
• reactivex

• You have everything set up correctly, however, getJSON retrieves everything at once. In that sense, responseStream isn't a stream, its a fully formed object. And the promise resolved from jQuery.getJSON is just the entire response from the page. Your code would produce the actions you're looking for only if you were loading data from multiple sources at once (results.json, results2.json etc). – bstockwell Commented Dec 19, 2015 at 18:04
• Is there a way to begin converting the single JSON file into an object and start looping through the Array on that object as its ing in? I'm looking for a way to stream the data incrementally as it's ing in and being processed. – agdev Commented Dec 19, 2015 at 18:33
• You're really dependent on the behavior of the browser. The response es in chunks which have a uniform size, but could begin and end anywhere in the data objects: [chunk 1, data: "[{object 1},{[ob] [chunk 2, data: "[{ject2}{[object3...etc.... The broswer collect these and puts them in the right order. Now what you're asking about is getting into hacking the http protocol, which is overkill. If you want to parse this object by object, you'll want to connect to your server with a websocket and emit each member one by one, which is more streamy than a single get call. – bstockwell Commented Dec 19, 2015 at 18:43
• Strike that last answer, it's probably sending everything in 1 big chunk, which, for our purposes is going to arrive at an instant in time. You're still looking for a websocket like behavior. (eg: socket.io) – bstockwell Commented Dec 19, 2015 at 18:47
• So if streaming is not a good route, how about chunking? So take the first 10 in that array and send that off to be displayed. Then the next 10 and so on. So more like buffering/processing that in smaller chunks. – agdev Commented Dec 19, 2015 at 18:51

1 Answer 1

If I understand well, there are two relevant points to make:

1. you need to find a way to have a stream of objects from that file instead of one big object when you finish reading that file (I want to start showing the data on the page as its ing in). The mechanics of that would depend first on the structure of the source (file and file reading mechanism) than on Rxjs (is every line an object that can lead to information display etc.?). Once you have that 'minimum displayable unit of information' you can use Rxjs to buffer/process it if need be (do you want to display something for each object, or each 100 objects, or remove uncessary attributes etc.?)
2. you need to update your display incrementally as new data arrive. That means you need something like $resultList.html($resultList.html() + piledTemplate(response)); to append the new piled html to the old one.

UPDATE : for chunking an array, you can have a look at this jsfiddle : http://jsfiddle/429vw0za/

var ta_result = document.getElementById('ta_result');

function emits ( who, who_ ) {return function ( x ) {
 who.innerHTML = [who.innerHTML, who_ + " emits " + JSON.stringify(x)].join("\n");
};}

function fillArrayWithNumbers(n) {
        var arr = Array.apply(null, Array(n));
        return arr.map(function (x, i) { return {prop1: i, prop2:i, prop3:i} });
    }

var sampleObj = {
    beginIndex: "1",
    catId: "111",
    endIndex: "1",
    products: fillArrayWithNumbers(100)
}

console.log('sampleObj', sampleObj);

var result$ = Rx.Observable
  .from(sampleObj.products)
  .bufferWithCount(10)
  .map(function(mini_array){return {
  beginIndex: sampleObj.beginIndex,
  catId: sampleObj.catId,
  endIndex: sampleObj.endIndex,
  products: mini_array
  }})
  .do(emits(ta_result, 'result'));

result$.subscribe(function(){    });

You will then have a stream of objects with arrays of size 10 taken from the array of size 100.