By Eduard Hovy
USC Information Sciences Institute

1. The idea
• Is there a claim?
• Is the claim clear?
• Is the idea large or small?
• Is the idea as large as it can be? Can you generalize it, or apply it elsewhere?
2. Motivation, Use, or Application
• Why should we care?
• How can the claim be used? Are there other applications?
3. Details of the Idea
• What are the basic items/elements /representation units of the idea?
• What are the rules or types of interrelationships between them?
• How elaborated are these items and rules/relationships?
• How much of the phenomena do they cover?
4. Data
• Is there enough data in the study?
• Is it representative? trustworthy? applicable?
5. Discovery Methodology
• Is the method of investigation clear?
• Is it appropriate? Does it ignore phenomena that look relevant?
• Is it well-reasoned? no biases or mistakes?
6. History
• Is prior work recognized? used?
7. Proof
• Is there an evaluation?
• If so, is it adequate? Complete enough?
• Does it speak to the claim?
• Does it actually prove the claim?
• If not, why not?
• Is there a discussion of how one might try to test or prove the claim?
• Can one make predictions and (easily) test them?