DC2: Design Challenge 2

The Basic Idea

In this Design Challenge, you will develop designs to help users work with threaded discussion data.

The goal of this design challenge is to provide you with opportunities to work with a “weird” data type – a form of data that doesn’t fit the usual kinds we see often or work with standard designs. It will hopefully also get you to think about comparison.

The data in this case are records from collections of threaded online discussions. You can think of the online discussions we have in this class on Canvas. For any discussion, we have a collection of messages. Some of those messages are in response to other messages. Each discussion can be thought of as a tree (each message is in response to some other message). Any particular discussion has a topic. For a topic, the set of people might be broken into independent groups.

In this assignment, we are considering only the statistical properties of the messages. We do not consider the content of the messages (other than their statistics). All we know are the lengths of the messages, and how many embedded images they have.

Given the data about the messages, there are many different kinds of questions we might want to ask (e.g., tasks):

• Are there some topics that get different kinds of discussion than others?
• Are there some people whose postings get more responses?
• Are there some people who tend to engage in long “conversations” (long chains of messages)?
• Does a particular person tend to join conversations late or early?
• Do different topics have different properties?
• Do different groups have different properties?

If it helps to make the problem more concrete, you could consider a tool for analyzing the messages from this class. You might ask questions about which discussions (assignments) lead to good/bad discussions. You might ask questions about how different people (students) performed. You might compare groups (which is basically comparing different conversations).

Looking at one conversation can be hard – it has an unusual structure (it’s a tree), and can involve different attributes (time, tree depth, message features, user, …). A lot of the interesting properties are about the “shape” of the tree – not just simple statistics. But to make things even harder, most tasks involve comparing multiple conversations.

In the project, you will need to come up with a set of tasks. Then develop a set of visualization designs that address these tasks. And then critique these designs with respect to the tasks.

For this design challenge, implementation is a level of degree. You can choose the kind of implementation for your designs. Your “prototypes” of your design might be pencil and paper sketches. Your prototype might be a system that can read in data files that we provide. Depending on the kind of implementation you choose, we will adjust the expectations accordingly.

The Schedule

This 4 week design challenge has a milestone every week. Be aware that in some cases, the deadlines are tight. Design Challenge deadlines are always on Fridays.

• Friday, October 19 – Task Analysis and Critique
• Friday, October 26 – Sketches and Plans
• Friday, November 2 – Checkin and Grading Plan
• Friday, November 9 – Final Handin

These are detailed below.

The structure of the assignment (or Why is this so complicated?)

There is a broad range of skills and interests in the class. I want to keep my promise of “Programming Optional” – and even for people who want to program, there is a wide variance – I don’t want this just to measure how much experience programming you’ve had before this class.

The main objective of the assignment are for us to use a non-standard data type to explore visualization principles and design.

However, I want to give people who want to learn about implementation a chance to try it out. Or, if you are a good programmer, you might think it’s easier to explore a design idea with code. Or . . .

So, I am giving you a lot of choices. I may regret it. And I make no premise that the hardness / amount of effort will be balanced. Your assignment must excel in at least one area – and you can choose which one it is. However, you must do at least some of each part of the assignment.

• Task / Design Analysis – You must think through what people do with this kind of data, and what they want from their visualization tools. Then you will look at the available visualizations and see how well they work for the different designs (both the already known ones, and new ones you will create).
• Novel Designs – You need to come up with solutions to some specific use cases (or general cases) with “new designs.” This might be some radically new visual encoding, or maybe just some tweaks on an existing design (like coming up with a clever use of interaction).
• Implementation – You will need to show off your designs. This might be building an actual system that lets people try it out. It might be a set of hand-drawn sketches. There are lots of things in-between. Some notable categories (and these aren’t hard distinctions). Also the names are somewhat arbitrary.
  • Tools – are programs that can read in new data sets. They will let a user (or a grader) try out your designs with their own data.
  • Prototypes – are visualizations that are created with real data (like a tool), but only for a fixed set of data. For example, if you write a program that only works with a few data sets, or manually draw a picture from a data set. If the grader can’t try their own data set, then it’s not a tool – but a prototype.
  • Sketches – don’t use real data. They approximate data, or use fake data. Usually you do this if you’re drawing something by hand, but you can imagine a program that draws a picture of what a visualization would look like without actually computing it from data. A textual description of what a visualization might look like is a form of a sketch.

Implementations

You are expected to provide “realizations” of your design(s). As described above, realizations/implementations can be a range of things (tools, prototypes, sketches).

Note that non-programming implementations can still convey complex designs and represent substantial effort. We have seen students prototype interactive designs using post-its on a whiteboard, documenting the designs with series of photographs and/or animations.

Below (under expectations) there are some notes on expectations that deal with issues of balance.

Some Ground Rules

You may work with a partner – but only if your project will be an implementation that will work on example data sets including the “class examples.” If you want to work with a partner, you must tell us as part of your “plan.”

You may use whatever tools you like – We will ask you to turn in images and PDFs. Even if you write a program, we will expect example outputs as part of the documentation. But if you program, you can use whatever programming language, libraries, etc. that you want. We will ask you to turn in your source code, but we may not be able to try it ourselves (as we might not have access to the same tools).

We may need you to “demo” your program for us – if you build a tool, we may ask for a demo in person. This will allow is to see any interaction you create, or give you a data set to test your program on.

Example Designs and Data

I will provide a bunch of example data. Most of it are larger sets created from anonymizing canvas discussions from older classes. I will also try to provide some simpler / smaller examples. See the DC2 Sample Data page.

I will provide (at least) 2 example designs (see DC2 Example Designs). At least one will be presented as a tool with example code. You will be asked to critique these designs in phase 1 of the assignment. You should avoid designs that are “too similar” to the examples. (your designs should be better!)

Unfortunately, we might not be able to provide example data that shows off your design well. Our data might be boring. So, if you (for example) have a great design that is really good at keeping track of long discussions, you might want to generate sample data that shows it off. Remember: don’t add additional variables!

If you make interesting example data, please let us know – we may ask if you are willing to share it with others!

How the Assignment Works / What you will turn in

The Design Challenge is four weeks long, at the end of each week, there is a check point.

Friday, October 19 – Task Analysis and Critique

DC2-1 Task Analysis and Critique

In this part of the assignment, we expect you to turn in 2 things. Note that every person must do this phase independently – even if they will work with a partner for the rest of the assignment.

These should be separate documents that are clearly labeled. PDFs or text files are file.

1. A list of tasks (at least 10, beyond the ones described in the assignment or example solutions). You may want to make some of the ones listed in the intro more concrete.
2. A critique of one or two of the example designs we provide at DC2 Example Designs. Note: if you choose to only provide one critique, that might be OK if you do more in later parts of the assignment.

Friday, October 26 – Sketches and Plans

DC2-2 Sketches and Plans

This part of the assignment also has two parts:

1. You should provide a sense of your plan. If you will work with a partner, let us know. If you will be building software, let us know – and give us an idea of what you intend to build and how you’ll build it (language, etc). If you plan to take the tool incentive (below), let us know.
2. You should provide a sketch of 1 or 2 of the designs you are considering, with a sense of what tasks you think the design(s) address. The goal here is to show us that you are thinking of designs. These do not need to be the designs you ultimately turn in.

For pairs, only one partner should turn in these two parts. The other partner needs to have something that simply says who they are working with.

Friday, November 2 – Checkin and Grading Plan

DC2-3 Checkin and Plan

Again, this assignment has two parts

1. We would like some evidence that you’re making progress. Give us a paragraph or so describing your progress. If you have a screen shot of your program, or a scan of a sketch, that’s helpful. The goal here is to make sure that no one is getting stuck – we won’t have any unpleasant surprises next week.
2. We would like you to describe what you are planning to hand in, so we can make sure we can set our expectations correctly and have mechanisms in place to get your materials and to grade them. This is mainly an issue for people who are programming. Giving the basic idea is great “I will turn in Python source code, and screen shots for sample data sets X,Y, and Z. But I’d like the opportunity to show off how well my program works on other data sets and the cool interactive features I’ve made.” Even if you are doing design by hand, give us a sense of what kinds of examples you’ll give, etc. We would also like you to give us some idea of how you think we should evaluate your project (e.g., what are you focusing on).

For pairs, only one partner should turn in these two parts. The other partner needs to have something that simply says who they are working with.

Friday, November 9 – Final Handin

DC2-4 Final Handin

You should turn in a ZIP file with:

1. Your designs.
2. For any designs that have programming implementations, please turn in:
   • the source code
   • images (even screenshots) of the programming running on different example data sets. For each, let us know what the example data is.
   • instructions on how to run it. be clear what tools/languages/libraries are needed and what we’d need to do if we had them. We may or may not try.
3. If you made your own example data, please provide it.
4. For each design you turn in, provide a self-critique including:
   • a description of the design and its intent
   • a discussion of tasks it is good for (what you can easily see)
   • a discussion of at least some tasks it is not good for
   • a discussion of how well the design scales (see expectations below). how will the design work as there are more messages? for comparing larger numbers of discussions?

Note: you are required to turn in (at least) two designs. Both designs should have a list of tasks and critiques. The two designs don’t need to be the same level of prototype. So, for example, if you implement one design with a program, you still need to turn in a second design – but that design might be a sketch of an alternative design.

For pairs, only one partner should turn in a final thing. The other partner needs to have something that simply says who they are working with.

Some thoughts on expectations

A difficult part of this assignment is that its hard to compare the amount of work for building a tool versus doing some paper designs. Different designs will excel in different ways.

We will evaluate:

1. Your task analysis and critiques (both of the sample designs and the ones you turn in)
2. Your designs
3. Your implementation

We will use these factors to determine your letter grade. We will also keep a separate score for how well you followed the rules (did you turn things in on time, etc). As in other aspects of the class, we consider this at the end (if you are chronically late throughout the semester, we may penalize you – if you are late occassionally, probably not).

You must do at least something for each of the 3 parts (analysis&critique, designs, implemenation). But the amount of each will vary from person to person. For example, you could make an excellent assignment by:

1. Developing a large number of designs that each address a specific task and providing hand drawn sketches that convey each one well, along with critiques that really explain how they address the tasks they design for.
2. Developing a single design into a tool that addresses a few useful tasks, with enough critique to show that you’ve thought through why it addresses at least some tasks.

There are all sorts of ways to make an assignment good. You should pick a combination of things (for example):

• thorough critique
• robust implementations (tools)
• nice looking implementations
• innovative designs
• many different deisgns
• designs that scale well to larger/harder problems (see below)
• really good looking implementations
• good example data generation

I was going to say that “if you program a simple bar chart, its probably not going that great” – but if your simple bar chart is connected to an interesting, task addressing analysis and really is a well chosen visual encoding for the task, then maybe it can be a great design.

A note on scalability

There are many ways the problem gets hard:

• A discussion gets big (lots of people, lots of messages, …)
• There are lots of discussions to compare
• The tasks are complex / the patterns considered are complicated

We will consider how “hard” the problem you are solving is. If you only show things for one or two small discussions, your assignment should excel in other ways.

For implementations that are programmed, there is some expectations that things will work on reasonable sized data.

But what grade will I get

It’s hard to set expectations ahead of time, since each person will do something very different. I encourage you to talk to me (for example in the DC2 discussion session on Friday, October 26th, or office hours) if you want to get a sense of whether you are being appropriately ambitious.

This assignment is designed such that it should be possible to get an A whether you program or not. In past assignments like this, there have been A assignments that were sketches on paper, and B designs that were programs.

See the factors under expectations (above).

Some hints

Remember, that designs are meant to serve tasks. When coming up with a design, you should start with a task.

In coming up with tasks… Think about the kinds of insights that someone might want to get from this kind of data. It might help to think about a particular use case. For example, imagine you are the professor for this class trying to look at last year’s class discussions in order to improve this year’s class, or looking at this year’s data to decide if groups are well-balanced.

Because of order dependencies in class, we won’t get to talk about Graphs and Networks until after this assignment is over. You might want to look ahead at the readings on graphs (a tree is a special type of graph). In particular, you might check out TreeVis.net to see a wide range of ideas on how you might represent a tree visually. Of course, you may not want to develop designs that show the data as a tree.

Another topic we might not get to discuss before this assignment is Comparison. You may wish to read ahead and check out my paper on Comparisons. I think it is helpful in coming up with design ideas.