This page elaborates the evaluation criteria I will apply to your projects in this class. You may also use these criteria as a guide for evaluating your peers' work.
As it happens, my evaluation criteria are also very similar to the approach I take to writing peer reviews of conference and journal submissions. For (highly relevant) general guidance on writing effective peer review check out this column from The Chronicle of Higher Ed and this extremely helpful OrgTheory post from Northwestern's own Brayden King.
I've provided a list of some generic questions/topics you may attempt to address when evaluating empirical research. It may be that some questions/topics are more applicable than others. For each one that you address, try to articulate the paper's approach and strengths as well as constructive criticism and suggestions for how the work might achieve the author's goals more effectively. Keep in mind that your evaluation of the work should neither be based on the hypothetical paper you think the author should have written nor the hypothetical paper you would have written (suspiciously, these two hypothetical papers often have a lot in common...), but rather the actually existing paper created by the author.
- Central question & rationale: What question or puzzle does this study aim to resolve? Does the paper state this question or puzzle clearly and justify it effectively?
- Analytical approach & methods: What analytical strategy does the work pursue? If the paper is pursuing a causal claim, what is the identification strategy? Are the methods described clearly, appropriate to the question, and executed well?
- Primary claim or finding: What is the primary claim or finding of the work? Does the claim fit with the author's overarching conceptual model for thinking about the underlying social processes involved? Are the findings presented clearly and effectively?
- Evidence and argument: Is the evidence presented a good match for the argument? In what ways, if any, might the claims be refined to better suit the evidence or might more evidence be provided to better support the claims?
- Threats, limitations, & validity: Given the analytical approach, evidence, and methods, what limitations, threats, or validity concerns has the author addressed? What limitations, threats, or validity concerns remain unaddressed and how might they be handled most effectively?
- Miscellany: Are there issues with structure, formatting, style, etc. that the author should try to address? What other things came to mind that you feel might help the author improve this work?