I am trying to build a network graph (like a network for brain) to display millions of nodes. I would like to know to what extent I can push the d3 js to in terms of adding more network nodes on one graph?

Like for example, / and :Cappuccino

I am not that familiar with d3.js (though I am a JS dev), I just want to know if d3.js is the right tool to build a massive network visualization (one million nodes +) before I start looking at some other tools.

My requirements are simply: build a interactive web based network visualization that can scale

I am trying to build a network graph (like a network for brain) to display millions of nodes. I would like to know to what extent I can push the d3 js to in terms of adding more network nodes on one graph?

Like for example, http://linkedjazz/network/ and http://fatiherikli.github.io/programming-language-network/#foundation:Cappuccino

I am not that familiar with d3.js (though I am a JS dev), I just want to know if d3.js is the right tool to build a massive network visualization (one million nodes +) before I start looking at some other tools.

My requirements are simply: build a interactive web based network visualization that can scale

• javascript
• d3.js
• neural-network
• scalability

3 Answers 3

Doing a little searching myself, I found the following D3 Performance Test.

Be Careful, I locked up a few of my browser tabs trying to push this to the limit.

Some further searching led me to a possible solution where you can pre-render the d3.js charts server side, but I'm not sure this will help depending on your level of interaction desired.

That can be found here.

"Scaling" is not really an abstract question, it's all about how much you want to do and what kind of hardware you have available. You've defined one variable: "millions of nodes". So, the next question is what kind of hardware will this run on? If the answer is "anything that hits my website", the answer is "no, it will not scale". Not with d3 and probably not with anything. Low cost smartphones will not handle millions of nodes. If the answer is "high end workstations" the answer is "maybe".

The only way to know for sure is to take the lowest-end hardware profile you plan to support and test it. Can you guarantee users have access to a 64GB 16 core workstation? An 8GB 2 core laptop? Whatever it is, load up a page with whatever the maximum number of nodes is and sketch in something to simulate the demands of the type of interaction you want and see if it works.

How much d3 scales is very dependent on how you go about using it.

If you use d3 to render lots of svg elements, browsers will start to have performance issues in the upper thousands of elements. You can render up to about 100k elements before the browser crashes, but at that point user interaction is basically useless.

It is possible, however, to render lots and lots of lines or circles with a canvas. In canvas, everything is rendered in a single image file. Rather than creating a new element for each node or line, you draw a line in the image file for it. The downside of this is that animation is a bit more difficult, since you can't move elements in a canvas, only draw on top of a canvas or redraw the whole thing. This isn't impossible, but would be putationally expensive with a million nodes.

Since canvas doesn't have nodes, if you want to use the enter/exit/update paradigm with it, you have to put placeholder elements in the DOM. Here's a good example of how to do that: DOM-to-canvas with D3.

Since the memory costs of canvas don't scale with the number of nodes, it makes for a very scalable solution for large visualizations, but workarounds are required to get it to be interactive.