Due Dates:
Initial Analysis (critique, task analysis): Wednesday, April 29th — Turn-in on Canvas
Re-Designs: due Wednesday, May 6 Friday, May 8 — Turn-in on Canvas
Description
For our last design challenge, I wanted to give you a real problem. A problem that I care about. A problem I need help with…
The challenge is: re-design the Explainers diagrams!
We’ve seen Explainers briefly in class. Explainers are a way of using machine learning techniques to create projections (reductions from high-dimensions to a single dimension). The exact techniques of Explainers aren’t important for the design challenge (but you are welcome to learn about them!). For the purpose of this assignment, you are only responsible for displaying the output of the process.
An Explainer takes a set of “items” (say, cities or plays), and scores them (gives them a single number). We usually also know what the “property” of the object is. So, the trick is to see how the objects score (and do objects with certain properties score higher or lower). And then things get trickier, because we might want to look at multiple Explainers. And there might be many items. And …
The online “supplement” for explainers: (http://graphics.cs.wisc.edu/Vis/Explainers/) has a link to the paper, as well as many examples using the simple diagram I designed (as well as some scatterplots and parallel coordinates diagrams). If you look at Figure 0, there is a full explanation of the diagram, as well as a discussion of some of the problems.
You do not need to understand what an Explainer is, or how they are generated in order to succeed at this assignment. You can view it as scoring data (each object gets measured), and each object has some discrete properties. So, your data might tell you whether the object is green or not, and a measure of how green it is (or multiple measures of how green it is). And the goal is to help the user understand how the score relates to the concept. (replace “green” with “comedy” or “European” or …)
Start by looking through the examples online – these should give you a sense of some of the things one might want to do with Explainers. By the time you see all of these examples, you’ll probably have a good sense of what these diagrams are used for, and what is good and bad about them.
My initial design has very limited interaction (since interaction is hard to implement!). And there is no interaction for specifying either how the explainers are built, or how the pictures get made (well, there is an interface – but it involves me writing Python programs to make the pictures).
Your challenge? Come up with better explainer diagrams! (if you want to address other aspects of the user experience of explainers, great – but that’s not required)
How can they be better? Well, that’s for you to figure out. Some ideas (coming soon – link the ideas to examples):
- Scale to more items.
- Scale to allow for comparing more explainers.
- Scale to allow for seeing how well an explainer correlates with a more complex relationship (like, predicting the year of a play, or a conjunction/disjuntion of category properties)
- Scale to deal with combinations of properties. If you have something that scores for a property (is the play a comedy? is the city European?), do certain subgroups of the items have the scoring work well or not (do the plays written by Shakespeare work with the same comedy relationship? are cities that hosted the Olympics more or less likely to have their “Europeanness” score be an indicator of their actual location).
You could even address some of the other aspects of Explainers, not dealt with in the diagrams (note: this is optional, since it requires you to understand more about explainers, and some of the statistical/machine learning underpinnings):
- How to specify the relationships that get trained on
- How to pick which explainers to show (out of the whole ensemble of explainers that get generated)
- How to get a sense as to whether or not the explainer you are looking at is doing better than chance (is the difference between classes statistically significant)?
- How to specify different aspects of the diagrams so they are better suited to specific situations (without having to change the program each time)
These are just examples… You should think of more.
Note: you are not expected to implement anything. Just to critique the current design, and to “describe” (which probably means give pictures) your alternatives. Describing interaction can be tricky (but that’s part of the challenge here – how to “sketch prototype”).
If you want to try to prototype something, I will happily give you the data. But beware: prototyping is more work. Functioning prototypes will be appreciated, but its better to do a good “sketched” design (even using computer drawing tools to mock-up what things look like) than to do a mediocre, but functional, design.
There are two parts to the assignment:
Phase 1: Critique.
In this phase, you will analyze the task and the existing design.
- What is the task? What might the user be trying to do?
- How does the design address the task(s)? What is good about the design?
- How have the ideas we’ve learned about in class been applied in the design?
- What does the design not do well? What kinds of tasks should things be improved for?
Phase 2: Re-Design
In this phase, your goal is to propose some alternative designs. You will probably want to pose multiple designs. You might want to show how your designs work in multiple settings. You might also try to come up with designs that address specific examples. (e.g. I use the same design for almost all examples, but you might want to have a specialized design for something like Figure 3).
But wait, this is hard!
Yes, it’s hard. The best I came up with is flawed, but it does work. So you have a pretty high bar to beat. But you have taken a Visualization class and I haven’t so you should be able to do better than me.
What we expect
For the Phase 1 (Critique) deadline, we expect each person to turn in a PDF document with a discussion of the task, and your analysis of my design. Be sure to relate the concepts you learned in class to your analysis. This is due on Wednesday, April 29th.
Note: we may not be able to provide feedback on your assignment Phase 1 before Phase 2 is due. (we will probably try)
However, if you would like feedback from your classmates, and are willing to provide feedback to them, we will arrange for this. We will allow people to “sign up” to participate (it’s totally optional). If you opt-in, we will give your critique to 2 people, and give you 2 others to look at and provide feedback for. This feedback trade is optional and ungraded.
For Phase 2 (Re-Designs), we expect each person to turn in a PDF document describing their proposed re-designs. Each person should propose at least 2 different designs. (don’t go too crazy coming up with lots of alternatives – we’ll only look at your 2-4 best ones). With your designs, be sure to discuss their goals, and give them a self-critique. Even if you come up with things that are better than the original, you should be able to articulate why the design is great, as well as point to its drawbacks.
With all critiques (both your assessment of my design, as well as your self critiques) be sure to consider where the design is (and is not) appropriate. My design tries to work for all explainers problems – and maybe I should have used different designs in different situations?
You can try to take on the challenge of showing multiple explainers at once. For example, in the paper/talk/web page, I show scatterplots (for showing two explainers), parallel coordinates (for 3 explainers), and simply putting different explainer diagrams side-by-side to give a sense of multiple different explainers. At least one of your designs should try to take on the challenges of a single explainer though (with an eye towards juxtaposing multiple ones)
If you want to turn in something more than a PDF for phase 2 (you still need to turn in a PDF) – be it a working prototype, a movie that illustrates your interaction, … Contact Alper and make arrangements. Note that you still need to turn in a PDF – even if there is a program for us to play with, we will want to read about the design and its rationale/critique.
