CS 672: System Reliability at Scale

Spring 2024

Paper Reviews

You will need to submit one paper review per class based on the assigned readings. This helps ensure you do the readings and are primed for in-class discussion.

Note that anything you write in the reviews may be anonymously brought up in class at some point or another. Therefore, treat the review as if others (in a conference setting, the paper authors or even the general public) are going to see it.

Review Guidelines

There are many resources for how to write good reviews (some provided below). I will not enforce any particular format but can recommend including the following components:

  1. Problem and Motivation

    Think carefully about why this work exists. In no more than a few sentences, express:

    • What problem(s) does the paper address? Why does this matter? What is their importance?
    • How is the problem motivated (e.g., by a real-world problem, theoretical predictions)?
    • Who cares about and benefits from the problem(s) being solved? How?
    • What kind of impact can solving this problem have?
  2. Paper summary
    • What research questions do the authors raise?
    • What are the paper's key ideas, claims, insights, and contributions? Be brief but specific.
    • How does it advance the state-of-the-art?
  3. Strengths and weaknesses

    Think critically about how well the paper does its job and provide 3-5 concrete bullet points for each. Try to avoid making vague or subjective criticisms. Some prompts to get you thinking:

    • Does the paper address the problem? How well?
    • How well does the paper substantiate its claims?
    • How relevant, applicable, and extensible is the work?
    • How reproducible are the paper's findings?
    • Could its ideas be communicated better?
    • Does the paper open up new topics to study?
    • Are there any flaws that you observe in the work?
  4. New ideas and impact

    What are your personal thoughts on the paper? Some prompts to get you thinking:

    • How is the paper connected to other research or papers you know?
    • What did you learn from the paper?
    • What are your most important takeaways?
    • What did you like/dislike the most? Why?
    • How might you approach things differently?
    • How might you improve or extend the paper?
    • Do you think it is likely to have impact? Has it already done so?
  5. Other comments

    Any other questions (e.g., things you didn't understand), comments, or discussion points you might like to raise. These are helpful for me to plan future lectures.

Format and Submission

Grading

Reviews will be graded based on how well they demonstrate your critical thinking. Although we are not reviewing for the benefit of the authors, practice in doing so is an important component of this graduate seminar.

I will score papers on a 4-point system with 1 point of possible extra credit:

Note: You may skip 3 reviews throughout the semester without penalty. This is to give you flexibility (e.g., when you have an upcoming deadline and need to suddenly drop everything).

External Resources and Examples