Interactive Chord Diagrams in D3

Image may be NSFW.
Clik here to view.

Authored by Steven Hall

In the last few weeks I've had the opportunity to explore how to create interactive chord diagrams that transition from one state to another.  It's tricky.  You need to do some caching of layout information in order to get a smooth animation that makes sense to the user.  You also need to think a little about how things get sorted on the screen by D3.  Keeping the order as stable as possible from transition to transition makes the diagram easier to follow.

If you have read my other article on chord diagrams, this code should be thought of as an refinement over that code.  It's easier to use and loses the underscore dependency. While the main idea is the same, the code here is more robust and, importantly, it can handle transitions by managing a cache of layout information.  I would work from this code even if you are just trying to make some basic chord diagrams.

In this article we're going to look at making the demo below and zoom in on the code to manage the chord diagram transitions...

VIEW THE DEMO HERE

The code is also available on GitHub...

DOWNLOAD THE CODE FROM GITHUB

This particular demo turned out to be a good way to work through how to do the transitioning for chord diagrams.  By tweaking the data processing a little (basically dumping more dyads into the "other trade" category) I could turn up and down the number of chords that get displayed at any one time.  In that way I was able to try it out on different browsers and tablet devices and see how it handled the transitioning with differing numbers of elements. 

The Basic Idea

Here's the basic idea.  We are going to build a custom matrix object that has some special methods.  We give this object the data, build an index for our matrix, add some configuration information and it will handle the constructing of the matrix from the raw data and all the layout caching stuff.  We just call the methods on the object to get our chords, groups, custom tween functions, and meta data about the matrix.

Image may be NSFW.
Clik here to view.

You are building a matrix each time the data changes which makes it easy to reason about, but you have to try to economize where you can to make it performant.  If you can provide a way for the user to limit the number of elements, that's really helpful.  On a desktop machine I may dial it up to hundreds of chords, but I can still use the display (with less elements) on a smaller devices.   That way people can maximize their experience on faster machines.

As it is, this demo is passable on an old iPad Mini but this is about the limit without doing some optimization tweaks on the code.  On a fast laptop machine this demo works awesome with virtually no jitter.  I was pretty impressed with the performance and the smoothness of the transitions given the large number of elements.

Need to just get into the details? Take look at the code for the matrix object on GitHub here.

Do I Need Angular?

You really don't need Angular to use this code.  It just provides a nice wrapper.  If you are familiar with D3, you can just look at the matrixFactory file and pull out the chordMatrix function.  That's really the whole show right there.  You can see how to use it by looking at the chordDirective file.  Although it's in a directive, the D3 code should look pretty familiar to anyone who has done a few D3 projects.  You just need to pull out the link function and port it over.

Looking at the Code

There's really three parts to this code: a controller,  a directive, and the chordMatrix object:

• mainController - The main controller loads the CSV data, stores it by year, and then listens for events on the years selection and filters.  When those change, the controller sends the filtered data to the drawChords function in the chordDirective and a new matrix is created and the display updates.
• chordDirective - The directive is mainly just standard D3 code, but it creates a chordMatrix object which survives across all transitions and manages the layout data for updating the chords and groups.
• matrixFactory - The factory returns chordMatrix objects.  The constructor function is in this file and is where most of the action is.

First, let's take a look at how the chordMatrix object gets configured in the chordDirective. 

var chord = d3.layout.chord()
   .padding(0.02)
   .sortSubgroups(d3.ascending);

var matrix = matrixFactory.chordMatrix()
  .layout(chord)
  .filter(function (item, r, c) {
    return (item.importer1 === r.name && item.importer2 === c.name) ||
           (item.importer1 === c.name && item.importer2 === r.name);
  })
  .reduce(function (items, r, c) {
    var value;
    if (!items[0]) {
      value = 0;
    } else {
      value = items.reduce(function (m, n) {
        if (r === c) {
          return m + (n.flow1 + n.flow2);
        } else {
          return m + (n.importer1 === r.name ? n.flow1: n.flow2);
        }
      }, 0);
    }
    return {value: value, data: items};
  });

Above we create new chordMatrix object and pass it a D3 chord layout, filter function, and a reduce function.  When you do an update, you pass in an array of objects as your data to the chordMatrix so the filter function will be called on every entry in the matrix to specify how to filter the data array down to the item (or items) of interest for the matrix entry.  If you are loading data from a CSV this structure will seem very natural.  You have an array of objects (i.e. rows) with properties to filter down and you need to find a particular row or set of rows in the file.  

The reduce function will be given the resulting item or items from the filter function.  The idea is "reduce" the result set down to an object with a "value" property and a "data" property.  The reason is that we are building a matrix, not of numbers, but of objects.  A matrix of numbers is fine for simple diagrams, but you need to know other information about the matrix (and the items in it) to make more complex displays, so we plan ahead and build our matrix out of objects.

The Update Process

The demo has a drawChords function in the chordDirective file that I call each time the data changes and I need to construct an updated matrix and transition the chart.  In the drawChords function, if we just pull out the key items happening we get this...

var arc = d3.svg.arc()
  .innerRadius(innerRadius)
  .outerRadius(innerRadius + 20);

var path = d3.svg.chord()
  .radius(innerRadius);

// CODE FOR CREATING THE MATRIX OBJECT - HERE

// THEN ON UPDATE
matrix.data(data)
  .resetKeys()
  .addKeys(['importer1', 'importer2']) // SETUP MATRIX KEYS FROM THE DATA
  .update() // BUILD A NEW MATRIX AND UPDATE LAYOUT CACHE

matrix.groups() // GET THE GROUPS WITH IDs
matrix.groupTween(arc) // GET TWEEN FUNCTION THAT USES THE LAYOUT CACHE

matrix.chords() // GET CHORDS WITH IDs
matrix.chordTween(path) // GET TWEEN FUNCTION THAT USES THE LAYOUT CACHE

So in the update function, I set the new incoming data, rebuild the keys (more on that coming up), and then update the matrix.  Then I can simply call matrix.groups() or matrix.chords() to get my updated sets.   The groups and chords have been given an ID internally that lets D3 determine if this item is entering, updating, or exiting.  So when we create our chords we can do this...

var chords = container.selectAll("path.chord")
   .data(matrix.chords(), function (d) { return d._id; }); // ID Function

D3 will take it from there and decide what should be in the enter, update, and exit selections. 

Importantly, the groupTween and chordTween methods handle all of the heavy lifting of getting the right cached version of the chord or group matched up with the transitioning chord or group.  The chord tween function is more complicated, but we can look at the group tween function below to get the idea. Both have similar logic.

matrix.groupTween = function (d3_arc) {
  return function (d, i) {
    var tween; 
    var cached = layoutCache.groups[d._id];

    if (cached) {
      tween = d3.interpolateObject(cached, d);
    } else {
      tween = d3.interpolateObject({
        startAngle:d.startAngle,
        endAngle:d.startAngle
      }, d);
    }

    return function (t) {
      return d3_arc(tween(t));
    };
  };
};

It's kind of a mind bender, but we've got a function that returns a function that returns a function.  I'm going to avoid all the details on this piece, but the code is very concise and clear if you want to take a look.  Bottom line is we need to interpolate between two objects, the current or cached position (or some default position) and the new position for the group or chord.  D3 needs the start and end angles of each object to do this.  If you look at the chord tween function it's more complicated to look-up the right values to work with, but the basic idea is exactly the same.

Building the Matrix Index

Image may be NSFW.
Clik here to view.

The code here assumes you want to create a square matrix.  The number of rows and columns are the same.  The items that make up the the rows are the same set of items that make up the columns.  If you refer back to my other article on chord diagrams, I explain there that the hair color example used by Mike Bostock (which in turn came from Circos) can be thought of as a tabulation that looks like the picture at the left.

In the case of the hair color chord diagram the "keys" would be black, blonde, brown, and red.  In my demo the keys are the unique set of country names in the data.  As a convenience, I made it easy to iterate over the data objects array and specify the fields or properties to use to build the index. In my case I want to use the unique set of items in the importer1 and importer2 columns to make up the items in my index, but this is where each project will differ.  You need to define what the items that make up the matrix are and how to calculate a matrix value where they coincide.  You can basically get the keys updated two ways...

// PULL OUT UNIQUE VALUES FROM THE DATA BY GIVING AN ARRAY OF PROPERTIES
matrix.data(data)
  .resetKeys()
  .addKeys(['importer1', 'importer2']) // GIVE IT AN ARRAY OF PROPERTIES
  .update()

// OR BUILD IT YOURSELF - YOU CAN ALSO STORE SOME DATA ON THE KEY
matrix.addKey(key, data)

So after I have my index built (a unique list of countries that appear in the data) I call update and generate the new matrix.  Nothing fancy here, just a brute force iteration over all the elements in the matrix, but internally it's keeping track of the last position of the chords by examining the IDs (country names).  It also maintains the matrixIndex which is locked to the the current matrix of objects (so we can look up values on the index using the matrixIndex and vice versa).  Chords get an ID that's is made up of the two countries that are the source and target of the chord.

So on each update we need to update the cache and build the new matrix.  From the matrixFactory file...

// FROM THE MATRIX.UPDATE METHOD

// UPDATE THE LAYOUT CACHE
layoutCache = {groups: {}, chords: {}};

this.groups().forEach(function (group) {
  layoutCache.groups[group._id] = {
    startAngle: group.startAngle,
    endAngle: group.endAngle
  };
});

this.chords().forEach(function (chord) {
  layoutCache.chords[chordID(chord)] = {
    source: {
      _id: chord.source._id,
      startAngle: chord.source.startAngle,
      endAngle: chord.source.endAngle
    },
    target: {
      _id: chord.target._id,
      startAngle: chord.target.startAngle,
      endAngle: chord.target.endAngle
    }
  };
});

// UPDATE THE MATRIX AND SET IT ON THE LAYOUT
matrixIndex = Object.keys(indexHash);

for (var i = 0; i < matrixIndex.length; i++) {
  if (!_matrix[i]) {
    _matrix[i] = [];
  }
  for (var j = 0; j < matrixIndex.length; j++) {
    recs = dataStore.filter(function (obj) {
      return filter(obj, indexHash[matrixIndex[i]], indexHash[matrixIndex[j]]);
    });
    entry = reduce(objs, indexHash[matrixIndex[i]], indexHash[matrixIndex[j]]);
    entry.valueOf = function () { return +this.value };
    _matrix[i][j] = entry;
  }
}
chordLayout.matrix(_matrix);

What's cool about this is that we can do some really sophisticated data processing.  We store information on the keys of the index and then use that information in the filter and reduce process.  The actual object stored in the matrix can have any arbitrary data you want and then you can access it for coloring, sizing, etc.  You can see in the code above that the filter and reduce function we provide gets passed the actual object from the indexHash for the row (i) and column (j) in the matrix.  This comes in handy when you want to do more complex things.

The code gives each entry object a valueOf property that returns the "value" property i.e. the numeric values you gave to the matrix entry in the reduce function.  This is so the matrix can be evaluated as a matrix of numbers or an array of objects depending on how its called.  When it's supposed to be numbers (e.g. when D3 calculates the layout) it will be numbers.  

Wrapping Up

And there you have it.  A nice little demo that does some cool transitions.  It goes without saying that trying to be as efficient as possible with your input data and the process of calculating matrix values the better.  For this demo I spent a lot of time optimizing the input data to be as small as possible for fast look-up.  Give it a shot.

Have suggestions on how to improve the code or this article?  Send a pull request or let me know in the comments below.