Thoughts on Reading Papers

by Mike Gleicher on January 13, 2011 · 1 comment

in Other Stuff

In this class, we’ll do a lot (most?) of our reading from “original sources” – the (academic) technical (research) papers where topics were first introduced. Part of the reason for this is a lack of good “secondary” sources (textbooks, survey papers, tutorials, …).

Note that the original sources aren’t always the best places to learn from:

  1. They are sometimes old, and don’t have a sense of what follows.
  2. They are written to convey the novelty of the idea (at the time it was submitted) and convince the reviewers of the importance of the contribution – not necessarily to help someone new to the field understand the basic ideas.
  3. At the time things are written, the authors may not know what the real lasting impact or importance of the paper is. Only time, experience, and follow on work can make you really know about it.
  4. Academic papers are an odd genre. (and you should think of them as a genre – just like romance novels or Shakespeare’s comedies). The specific sub-genre of graphics papers (particularly SIGGRAPH) has its own peculiar style.

Over the course of the semester, one of my goals is to help you appreciate this genre. Kindof like learning to appreciate plays. To help do this, we’ll have a number of assignments besides just reading. We’ll purposely pick some papers that are unusual – historically significant, non-standard styles, etc.

Some Features of Papers

Not all papers have this form. But these are common features in the “Sub-Genre” of graphics/animation papers. Looking for them is a good way to appreciate/learn from a paper.

The Sentence

Most, if not all, papers have a clear message that can be stated simply – in a single sentence. Being able to craft this single sentence is actually a good sign that (as an author) you really have a focused message, and have figured out how to articulate it.

If you look at a paper, you can often find this sentence. In one place (preferably near the front) the author can tell you what the paper is going to “do” for you – what you will learn by reading it. Sometimes this single sentence appears in the abstract, sometimes it is right at the beginning of the paper, sometimes it is at the end of the introduction (after the motivation).

Some interesting thoughts on this (in terms of a thesis) are given by Olin Shivers here. (Or see my take on theses and prelims).

The Problem

In graphics, there is a preference for papers that solve a specific problem. Preferably, this problem can be neatly stated in the introduction, explaining why the problem is hard (and needs the new solution that the paper will propose). This statement of the problem may be closely related to “the sentence.”

Note that sometimes, finding (and understanding) the good problem is a major contribution.

In defining a problem (or in thinking about a problem), it is usually good to know why the problem is hard. Good papers will spell it out: showing why the existing literature doesn’t have a solution to the problem (see Contribution below). As a reader, its often worthwhile to think about that the “naive” solution might be, and why that isn’t good enough. If the authors are kind, they will make this clear.

The Key Idea(s)

While a paper may need pages and pages of details to explain how it solves the problem, there is often a key idea (or a small set of them) that makes it all possible – the real creative breakthrough. Sometimes, the solution to a problem just requires a lot of hard work, and a bunch of good decision making and system building. But often, there’s a few key insights/ideas that made the content of the paper possible.

The Basic Idea / Overview

This is actually different than the Key Idea. Usually, you can state concisely what the solution is (overall). When you describe things at such a brief level, you might leave out the key trick

The Contribution

The central feature of the genre of an academic paper comes from the fact that it needs to make “a contribution” to the knowledge of the community. In most cases, this is new knowledge. In some cases (like survey papers, or retrospective papers) the contribution might be to present an existing idea in a new or better way, or organize some existing knowledge.

For getting a paper into a top tier conference, the usual metric is “what is the contribution of this paper.” It’s so important than in some sub-genre’s (like current SIGGRAPH papers), the authors usually try to be explicit and say what their contributions are. This is a little wierd / heavy handed (it’s like saying why the paper is good), but it certainly makes it easier for a reviewer to appreciate why a paper contributes.

The Compelling Evidence

How are the authors of the paper going to convince you that their solution is right/good?

It could be they authors will show you a few examples. It could be they do some thorough empirical study. How the authors do it might vary, but at some point, the authors need to convince the reader (and the reviewer) that their method really does deliver on its promises.

The degree of evidence varies from paper to paper, depending on the nature of the contributions. In some cases, just presenting the ideas are enough. In other cases, very formal evaluation is required to really be convincing.

{ 1 trackback }

Previous post:

Next post: