Group discussions, critiques, and in-class discussions
In groups of about 3-4 people, meet for 1 hour after reading
the paper but before writing your critique. Spend some time
(perhaps 20 minutes, depending on the paper)
making sure everyone understood the important points, and spend
some time starting to discuss the points related to the critique
(contributions, strengths, weaknesses).
A paper's discussion leader should also use this time to
get feedback from his/her group, especially on his/her intended
After the group meeting, write a critique that has five short
Although each paragraph can be short, avoid paraphrasing the paper
and avoid superficial statements. For example: "The authors propose
an approach to detect atomicity bugs, and they evaluate its performance
and accuracy." This sentence doesn't really say anything unique to one paper.
What's the key aspect of the approach and the key insight that makes it work?
What is the (single most important)
outcome of the performance and accuracy results, and why?
- Problem that the paper is solving (and why is it important and/or hard and/or why doesn't prior work address
Try to identify the key technical problem being solved.
- Key insights / contribution / strengths (in your own opinion; don't regurgitate what the papers says).
What are the key technical insights and contributions of the paper?
- Weaknesses / fallacies / flaws (not the obvious ones or the ones mentioned by the paper, but rather the ones
you think are the most serious / most consequential, and why).
- Unsolved problem(s) / opportunities for future work.
- Your difficulty/ies in understanding.
Some additional advice:
- Try to say non-obvious, interesting things.
That may seem like generic advice, but it's a useful filter to apply
to any given sentence, i.e., "Is this sentence going to be interesting
or non-obvious to someone who has already read the paper (but perhaps
not thought about it in depth)?"
- Try to convince the reader that you've not only read the paper, but that you
understood (at least some parts of it) at a fairly deep level, and you
really thought about the papers ideas and contributions.
- You can assume your audience (me) has read the paper. This assumption
should help you skip unimportant sentences.
- Don't try to cover everything. Depth is more important than breadth.
- Try to figure out what are the paper's real contributions,
strengths, and weaknesses—which might be different than what
the paper says about itself. Take risks; it's okay to be "wrong."
- Take risks. It's better to be wrong than boring/unimaginative. :)
The critique for a paper is due by 6 pm on the day before we discuss the
paper in class. As part of your critique, include your name and the names
of the group members whom you met and discussed the paper with, both to
acknowledge their potential influence on your ideas, and so I can make sure that
group discussions are functioning okay. Do not list the names of anyone with whom
you did not discuss the paper. You'll still get credit for the critique, but I might
bug you about what's going on with your group. Here's a template (keep it simple and plaintext):
I discussed with these people:
Problem (and why important and/or hard and/or why doesn't prior work address it):
Key insights / contribution / strengths:
Weaknesses / fallacies / flaws:
Unsolved problem(s) / opportunities for future work:
Difficulty/ies in understanding:
I'm planning to send your critiques to everyone, but without your names.
Please read others' critiques before class.
I will periodically grade and provide feedback on your critiques.
Craig Zilles has great
advice for writing critiques.
Discussion leaders should prepare to lead the class in a discussion of the paper's most interesting points.
Note that the discussion is not a presentation, i.e., it's not like a conference talk. Everyone has already
read, discussed, and critiqued the paper (and read others' critiques).
That said, these papers are hard and will have material that's new to most of us, so typically half or more
of the time should be spent leading discussion of key points from the paper. Roughly the second half
of the time should be spent leading discussion of questions or topics that go beyond just understanding
the paper's key points, such as discussing strengths, weaknesses, connections to related work,
and opportunities/challenges/ideas for future work.
You do not need to make slides to lead your discussion, but it often helps to put up a few diagrams,
as well as discussion questions and related follow-up questions and ideas.
It's often a good idea to draw on the whiteboard during a discussion.
You may obtain the authors' slides if you like, but don't use these slides (they will cause you
to give a presentation rather than lead a discussion). It's okay to copy-paste diagrams from the slides to aid
discussions, but it's also okay to just copy-paste diagrams from the paper's PDF. If you use any existing
content, please give appropriate credit somewhere (on that slide or your first slide).
Timeline for group discussions and critiques, and for discussion leaders
- Finish reading paper: two days before class (or sooner)
- Meet with group: Day before class (or sooner)
- Write & send critique: by 6 pm on day before class
- Read others' critiques: before class; after 6 pm on day before class
- Start reading: 2 weeks before class
- Finish reading paper; outline discussion and start preparing discussion questions (you might want to contact me at this point): 1 week before class
- Contact me to set up a meeting (include full availability and your current status): 4 weekdays before class
- Send outline (i.e., points & discussion questions) to me: day before our meeting (or earlier)
- Bring any slides (not required) on thumb drive (or send to me early) and also put them on the Web.
Arrive 10 minutes early (5 minutes if no slides).