I'm working with a dataset that's relatively large for web users, especially smartphone users. I'm worried about performance. Which is a bigger problem for users?

1. Forcing the client's browser to fetch/request a large data file (JSON).
2. Forcing the client's browser to reformat a smaller file (CSV) into a larger file (JSON) so it can be used.

When I pile the data as JSON, it's about 570KB – far larger than I would normally use. And that's stripped right down (e.g. I've reduced the keys to a single character each).

When I pile the data as CSV, it's about 220KB. However, I then need the browser to reformat it into JSON format anyway.

Here's a tiny example. A CSV file:

"year","birth","101","102","103","104","105"
1981,"Australia",5972,1099,573,747,667
1981,"China",141,4,3,2,2
1981,"India",139,5,4,6,2
1981,"Indonesia",371,9,14,5,6
1981,"Malaysia",838,72,42,11,14 

... pared with the same data as JSON:

[{"year":1981,"birth":"Australia","101":5972,"102":1099,"103":573,"104":747,"105":667},
{year":1981,"birth":"China","101":141,"102":4,"103":3,"104":2,"105":2},
{year":1981,"birth":"India","101":139,"102":5,"103":4,"104":6,"105":2},
{year":1981,"birth":"Indonesia","101":371,"102":9,"103":14,"104":5,"105":6},
{year":1981,"birth":"Malaysia","101":838,"102":72,"103":42,"104":11,"105":14}]

TLDR: What's more important for performance: (1) minimising the size of data files, or (2) minimising the amount of data processing the browser must do?

I'm working with a dataset that's relatively large for web users, especially smartphone users. I'm worried about performance. Which is a bigger problem for users?

1. Forcing the client's browser to fetch/request a large data file (JSON).
2. Forcing the client's browser to reformat a smaller file (CSV) into a larger file (JSON) so it can be used.

When I pile the data as JSON, it's about 570KB – far larger than I would normally use. And that's stripped right down (e.g. I've reduced the keys to a single character each).

When I pile the data as CSV, it's about 220KB. However, I then need the browser to reformat it into JSON format anyway.

Here's a tiny example. A CSV file:

"year","birth","101","102","103","104","105"
1981,"Australia",5972,1099,573,747,667
1981,"China",141,4,3,2,2
1981,"India",139,5,4,6,2
1981,"Indonesia",371,9,14,5,6
1981,"Malaysia",838,72,42,11,14 

... pared with the same data as JSON:

[{"year":1981,"birth":"Australia","101":5972,"102":1099,"103":573,"104":747,"105":667},
{year":1981,"birth":"China","101":141,"102":4,"103":3,"104":2,"105":2},
{year":1981,"birth":"India","101":139,"102":5,"103":4,"104":6,"105":2},
{year":1981,"birth":"Indonesia","101":371,"102":9,"103":14,"104":5,"105":6},
{year":1981,"birth":"Malaysia","101":838,"102":72,"103":42,"104":11,"105":14}]

TLDR: What's more important for performance: (1) minimising the size of data files, or (2) minimising the amount of data processing the browser must do?

• javascript
• json
• performance
• csv
• request

• 1 The network request is going to be asynchronous, so your user won't lose interactivity with the page if it takes a tiny bit more time. The csv parsing, on the other hand, will block interactivity if it is slow, assuming you are not using a service worker. Therefore the json is preferable from a user's perspective, assuming a worst case scenario. However, in practice, I doubt either solution is going to lead to a perceptible slow-down relative to the other. – M-N Commented Oct 8, 2019 at 3:38
• 1 @M-N You could do the CSV parsing in a web-worker and return the materialized data to the web-page via a message. That said, I don't remend processing CSV data in JavaScript simply because JavaScript does not have (widely supported) string and stream processing support (e.g. as JavaScript strings are immutable and there's no mutable string builder, building CSV strings and performing lots of string.split calls will waste memor with excessive string copies and allocations). – Dai Commented Oct 8, 2019 at 4:23
• 1 Feedback on why the question was down-voted would be appreciated. – Markus Commented Oct 8, 2019 at 5:04
• I’d like to add that V8 switches off its optimising piler when running on some mobile devices. That’s a reason to consider avoiding a lot of data processing. – user11390576 Commented Oct 25, 2019 at 21:33

1 Answer 1

Preface:

I'd argue that what you're wanting to do is a kind of premature microoptimization ( https://en.wikipedia/wiki/Program_optimization ), this is because most webservers will GZip HTTP responses anyway, so as far as actual transferred data is concerned both the CSV and expanded JSON representations will have roughly the same GZip size because they have the same Information Entropy.

Also, I remend reading this article from Google (dated June 2019): https://v8.dev/blog/cost-of-javascript-2019 - in short: JavaScript is cheap and you only need to worry about optimization on mobile devices, not desktops/laptops.

Anyway:

There are a few other alternatives besides CSV and JSON Objects.

JSON Arrays:

One option, which is probably the best of both worlds, is using JSON arrays, like so:

[
 [ "year","birth","101","102","103","104","105" ],
 [ 1981,"Australia",5972,1099,573,747,667 ],
 [ 1981,"China",141,4,3,2,2 ],
 [ 1981,"India",139,5,4,6,2 ],
 [ 1981,"Indonesia",371,9,14,5,6 ],
 [ 1981,"Malaysia",838,72,42,11,14 ]
]

You can use named const array indexes to access each data member:

const Idx = {
    YEAR: 0,
    BIRTH: 1,
    _101: 2,
    _102: 3,
    _104: 4,
    // etc
};

var data = JSON.parse( text ); // the array from above

for( var i = 1; i < data.length; i++ ) {
    var row = data[i];

    console.log( "Year: %d, Birth: %s", row[Idx.YEAR], row[Idx.BIRTH] );
}

You could also have your own materialization function to convert each row into a strongly-typed object:

function Item( row ) {
    this.year = row[Idx.YEAR];
    this.birth = row[Idx.BIRTH];
}

var data = JSON.parse( text ); // the array from above

var items = data.map( row => new Item( row ) );

Array of constructor calls

Another alternative to representing each record as an array within a parent array is to represent each record as a constructor call - however this will not work with JSON.parse - you must either use eval() (NOT RECOMMENDED), render data directly within a web-page inside your server-side generation script, or have the client load it into a <script> element (this is how JSONP works, but is dangerous).

I use this approach myself when rendering data to a webpage for consumption by third-party data-visualization ponents like D3 or various other charting libraries:

function Item( year, birth, _101, _102, _103, _104, _105 ) {
    this.year = year;
    this.birth = birth;
    this._101 = _101;
    this._102 = etc...
}

data = [
    new Item( 1981,"Australia",5972,1099,573,747,667 ),
    new Item( 1981,"China",141,4,3,2,2 ),
    new Item( 1981,"India",139,5,4,6,2 ),
    new Item( 1981,"Malaysia",838,72,42,11,14 ),
    // etc
];

renderChart( data );

I use this approach when I need to perform client-side transformation of data and I don't want to render two copies of the data in different formats to the response, for example. But as I said, this technique does not work with JSON.parse because json must be just static data and not a constructor call.