I'm trying to recreate the awesome 'dot flow' visualizations from Bussed out by Nadieh Bremer and Shirely Wu.

I'm especially intrigued by the very circular shape of the 'bubbles' and the fluid-dynamics-like pression in the spot where the dots arrive to the bubble (black arrow).

My take was to create (three) fixed nodes by .fx and .fy (black dots) and link all the other nodes to a respective fixed node. The result looks quite disheveled and the bubbles even don't form around their center nodes, when I lower the forces so the animation runs a little slower.

  const simulation = d3.forceSimulation(nodes)
    .force("collide", d3.forceCollide((n, i) => i < 3 ? 0 : 7))
    .force("links", d3.forceLink(links).strength(.06))

Any ideas on force setup which would yield more aesthetically pleasing results?

I do understand that I'll have to animate the group assignment over time to get the 'trickle' effect (otherwise all the points would just swarm to their destination), but i'd like to start with a nice and round steady state of the simulation.

EDIT

I did check the source code, and it's just replaying pre-recorded simulation data, I guess for performance reasons.

I'm trying to recreate the awesome 'dot flow' visualizations from Bussed out by Nadieh Bremer and Shirely Wu.

I'm especially intrigued by the very circular shape of the 'bubbles' and the fluid-dynamics-like pression in the spot where the dots arrive to the bubble (black arrow).

My take was to create (three) fixed nodes by .fx and .fy (black dots) and link all the other nodes to a respective fixed node. The result looks quite disheveled and the bubbles even don't form around their center nodes, when I lower the forces so the animation runs a little slower.

  const simulation = d3.forceSimulation(nodes)
    .force("collide", d3.forceCollide((n, i) => i < 3 ? 0 : 7))
    .force("links", d3.forceLink(links).strength(.06))

Any ideas on force setup which would yield more aesthetically pleasing results?

I do understand that I'll have to animate the group assignment over time to get the 'trickle' effect (otherwise all the points would just swarm to their destination), but i'd like to start with a nice and round steady state of the simulation.

EDIT

I did check the source code, and it's just replaying pre-recorded simulation data, I guess for performance reasons.

• javascript
• d3.js
• data-visualization
• visualization
• force-layout

• Wow, yet another awesome data viz by Nadieh Bremer! I started digging into the article's sources only to find them being packed, which doesn't really help. Have you tried reaching out to her to at least get you started? – altocumulus Commented Jan 24, 2019 at 11:34
• After some research I believe the author of this particular piece is actually Shirley. I'll try to contact her.. – liborm Commented Jan 24, 2019 at 13:17
• 2 @liborm You are probably correct, she put that on her page: sxywu. – Gerardo Furtado Commented Jan 24, 2019 at 13:21

3 Answers 3

Building off of Gerardo's start,

I think that one of the key points, to avoid excessive entropy is to specify a velocity decay - this will help avoid overshooting the desired location. Too slow, you won't get an increase in density where the flow stops, too fast, and you have the nodes either get too jumbled or overshoot their destination, oscillating between too far and too short.

A many body force is useful here - it can keep the nodes spaced (rather than a collision force), with the repulsion between nodes being offset by positioning forces for each cluster. Below I have used two centering points and a node property to determine which one is used. These forces have to be fairly weak - strong forces lead to over correction quite easily.

Rather than using a timer, I'm using the simulation.find() functionality each tick to select one node from one cluster and switch which center it is attracted to. After 1000 ticks the simulation below will stop:

var canvas = d3.select("canvas");
var width = +canvas.attr("width");
var height = +canvas.attr("height");
var context = canvas.node().getContext('2d');

// Key variables:
var nodes = [];
var strength = -0.25;         // default repulsion
var centeringStrength = 0.01; // power of centering force for two clusters
var velocityDecay = 0.15;     // velocity decay: higher value, less overshooting
var outerRadius = 250;        // new nodes within this radius
var innerRadius = 100;        // new nodes outside this radius, initial nodes within.
var startCenter = [250,250];  // new nodes/initial nodes center point
var endCenter = [710,250];	  // destination center
var n = 200;		          // number of initial nodes
var cycles = 1000;	          // number of ticks before stopping.



// Create a random node:
var random = function() {
	var angle = Math.random() * Math.PI * 2;
	var distance = Math.random() * (outerRadius - innerRadius) + innerRadius;
	var x = Math.cos(angle) * distance + startCenter[0];
	var y = Math.sin(angle) * distance + startCenter[1];

	return { 
	   x: x,
	   y: y,
	   strength: strength,
	   migrated: false
	   }
}

// Initial nodes:
for(var i = 0; i < n; i++) {
	nodes.push(random());
}
	
var simulation = d3.forceSimulation()
    .force("charge", d3.forceManyBody().strength(function(d) { return d.strength; } ))
	.force("x1",d3.forceX().x(function(d) { return d.migrated ? endCenter[0] : startCenter[0] }).strength(centeringStrength))
	.force("y1",d3.forceY().y(function(d) { return d.migrated ? endCenter[1] : startCenter[1] }).strength(centeringStrength))
	.alphaDecay(0)
	.velocityDecay(velocityDecay)
    .nodes(nodes)
    .on("tick", ticked);

var tick = 0;
	
function ticked() {
	tick++;
	
	if(tick > cycles) this.stop();
	
	nodes.push(random()); // create a node
	this.nodes(nodes);    // update the nodes.

  var migrating = this.find((Math.random() - 0.5) * 50 + startCenter[0], (Math.random() - 0.5) * 50 + startCenter[1], 10);
  if(migrating) migrating.migrated = true;
  
	
	context.clearRect(0,0,width,height);
	
	nodes.forEach(function(d) {
		context.beginPath();
		context.fillStyle = d.migrated ? "steelblue" : "orange";
		context.arc(d.x,d.y,3,0,Math.PI*2);
		context.fill();
	})
}

<script src="https://cdnjs.cloudflare./ajax/libs/d3/5.7.0/d3.min.js"></script>
<canvas width="960" height="500"></canvas>

Here's a block view (snippet would be better full page, the parameters are meant for it). The initial nodes are formed in the same ring as later nodes (so there is a bit of jostle at the get go, but this is an easy fix). On each tick, one node is created and one attempt is made to migrate a node near the middle to other side - this way a stream is created (as opposed to any random node).

For fluids, unlinked nodes are probably best (I've been using it for wind simulation) - linked nodes are ideal for structured materials like nets or cloth. And, like Gerardo, I'm also a fan of Nadieh's work, but will have to keep an eye on Shirley's work as well in the future.

Nadieh Bremer is my idol in D3 visualisations, she's an absolute star! (correction after OP's ment: it seems that this datavis was created by Shirley Wu... anyway, that doesn't change what I said about Bremer).

The first attempt to find out what's happening on that page is having a look at the source code, which, unfortunately, is an herculean job. So, the option that remains is trying to reproduce that.

The challenge here is not creating a circular pattern, that's quite easy: you only need to bine forceX, forceY and forceCollide:

const svg = d3.select("svg")
const data = d3.range(500).map(() => ({}));

const simulation = d3.forceSimulation(data)
  .force("x", d3.forceX(200))
  .force("y", d3.forceY(120))
  .force("collide", d3.forceCollide(4))
  .stop();

for (let i = 300; i--;) simulation.tick();

const circles = svg.selectAll(null)
  .data(data)
  .enter()
  .append("circle")
  .attr("r", 2)
  .style("fill", "tomato")
  .attr("cx", d => d.x)
  .attr("cy", d => d.y);

<script src="https://cdnjs.cloudflare./ajax/libs/d3/5.7.0/d3.min.js"></script>
<svg width="400" height="300"></svg>

The real challenge here is moving those circles to a given simulation one by one, not all at the same time, as I did here.

So, this is my suggestion/attempt:

We create a simulation, that we stop...

simulation.stop();

Then, in a timer...

const timer = d3.interval(function() {etc...

... we add the nodes to the simulation:

const newData = data.slice(0, index++)
simulation.nodes(newData);

This is the result, click the button:

const radius = 2;
let index = 0;
const limit = 500;
const svg = d3.select("svg")
const data = d3.range(500).map(() => ({
  x: 80 + Math.random() * 40,
  y: 80 + Math.random() * 40
}));

let circles = svg.selectAll(null)
  .data(data);
circles = circles.enter()
  .append("circle")
  .attr("r", radius)
  .style("fill", "tomato")
  .attr("cx", d => d.x)
  .attr("cy", d => d.y)
  .style("opacity", 0)
  .merge(circles);

const simulation = d3.forceSimulation()
  .force("x", d3.forceX(500))
  .force("y", d3.forceY(100))
  .force("collide", d3.forceCollide(radius * 2))
  .stop();

function ticked() {
  circles.attr("cx", d => d.x)
    .attr("cy", d => d.y);
}

d3.select("button").on("click", function() {
  simulation.on("tick", ticked).restart();
  const timer = d3.interval(function() {
    if (index > limit) timer.stop();
    circles.filter((_, i) => i === index).style("opacity", 1)
    const newData = data.slice(0, index++)
    simulation.alpha(0.25).nodes(newData);
  }, 5)
})

<script src="https://cdnjs.cloudflare./ajax/libs/d3/5.7.0/d3.min.js"></script>
<button>Click</button>
<svg width="600" height="200"></svg>

Problems with this approach

As you can see, there is too much entropy here, particularly at the centre. Nadieh Bremer/Shirley Wu probably used a way more sofisticated code. But these are my two cents for now, let's see if other answers will show up with different approaches.

With the help of other answers here I went on experimenting, and I'd like to summarize my findings:

Disc shape

forceManyBody seems to be more stable than forceCollide. The key for using it without distorting the disc shapes is .distanceMax. With the downside that your visualization is not 'scale-free' any more and it has to be tuned by hand. As a guidance, overshooting in each direction causes distinct artifacts:

Setting distanceMax too high deforms the neighboring discs.

Setting distanceMax too low (lower than expected disc diameter):

This artifact can be seen in the Guardian visualization (when the red and blue dots form a huge disc in the end), so I'm quite sure distanceMax was used.

Node positioning

I still find using forceX with forceY and custom accessor functions too cumbersome for more plex animations. I decided to go with 'control' nodes, and with little tuning (chargeForce.strength(-4), link.strength(.2).distance(1)) it works ok.

Fluid feeling

While experimenting with the settings I noticed that the fluid feeling (ining nodes push boundary of accepting disc) depends especially on simulation.velocityDecay, but lowering it too much adds too much entropy to the system.

Final result

My sample code splits one 'population' into three, and then into five - check it on blocks. Each of the sinks is represented by a control node. The nodes are re-assigned to new sinks in batches, which gives more control over the visual of the 'stream'. Starting to pick nodes to assign closer to the sinks looks more natural (single sort at the beginning of each animation).