Explainers Supplementary Material

Figure 0: A Simple Example

This figure aims to give you a simple example of an "explainers diagram" so that you are more likely to be able to make some sense out of the more complicated ones in the paper.

An explainer is a function mapping between objects in a set (each one is a row of the data, if you like to think that way), and a number. It projects the set onto a single axis.

In this example, the set of objects are Shakespeare's 36 plays. The function was designed to measure the "comedicness" of a play - that is, that plays that are comedies should have higher values than plays that are not comedies. For this example, we have chosen a very simple function: a linear function that has only 2 variables (it selects two columns of the data matrix), and has unit coefficients. With such a simple function, we might not be so surprised that the function gets a few things "wrong" (that is, a non-comedy has a higher value than a comedy). The data (and the function) will be explained later.

The diagram consists of a number of parts.

The SVG files work in any browser with good SVG support. However, the html pages with embedded SVG seem to appear wrong in browsers other than Chrome. For some reason, the text gets messed up (probably a CSS issue).

The left most part is a list of the plays in the order of comedicness (so that the play with the highest value is at top, the one with the least value is lowest). Here, the colors are used to denote genre (green is comedy, yellow is history, purple is tragedy, and red are the "late plays"). Yes, there should be a caption labeling these colors. Some example things to notice:

The next column is a stack of spline curves each connecting one of the plays on its left to a position on the right. The right hand side indicates the value of the function (ranging from the minimum value to the maximum value). So each of the little curves connects from an object's rank value to the object's value. The vertical scale (of the right side) is the same scale that is used for the next two columns (histogram and boxplot). Reading this, you should be able to notice:

The next column is a stacked histogram showing the distribution of the data over the scale. For 36 items, this isn't so interesting - but it's here for completeness. There are fixed size buckets ranging from min value to max value (for this, there are only 7 bins to try to make the histogram look a little less empty). The vertical scale is the same as the right side of the connecting curves. What you should be able to see:

The last part of the display consists of 3 modified boxplots. I emphasize modified because its parts do not have the standard meaning. The line represents the range of the data (the whiskers are at the min and max). The boxes represent the inner quartiles, and the strip represents the median. The leftmost boxplot shows the distribution of the entire data set. It is always shown in gray. The other two boxplots show the two classes (comedies and non-comedies). In cases where the classes have colors, the boxplot is colored. So the rightmost one is green, since comedies are green. The middle boxplot (for non-comedies) is uncolored, since there is no "non-comedy" color (it is 3 different colors). Some things you can pick out from these boxplots:

At the bottom there is some text describing the explainer's performance and what variables it depends on.

The top line lists various performance metrics. Here it gives the nth score (the non-threshold metric from the paper) and the mcc score (the Mathews correlation coefficient, a common metric for judging classifiers in machine learning).

The lowest lines describe the linear function. This one is -1 * inclusive + 1 * metadiscourse. Inclusive and Metadiscourse are features (columns) in the data. For this example, these features come from Docuscope (see references in the paper), which basically counts the number of phrases in the document that are of this "type" (where the "types" are rhetorical categories defined by the creators of the Docuscope system).

If you're curious about how I chose this explainer, over all the other "comedicness" explainers, you should look at the explanation of the scagnostics plot in Figure 0s.

For some further practice with this, you might want to move on to Figure 0b or Figure 4.

Scaling Up

These diagrams get more complicated in a few different ways: