7. Understanding the Review Process
it is important to identify review criteria, understand exactly how the agency defines them, and determine the relative weight (if any) that the agency assigns to each of them.
When evaluating a grant application, reviewers will not only consider the quality of the ideas, but also the extent to which the application addresses the funding agency’s review criteria. Therefore, it is important to identify these review criteria, understand exactly how the agency defines them, and determine the relative weight (if any) that the agency assigns to each of them. This information can then be used to develop an application that clearly addresses these criteria and that is much more competitive.
The table below provides links to the descriptions of the specific review criteria and basic review processes for each of the top seven sponsors of research at Texas A&M University.
Description of Review Criteria and Review Process Relevant Section
DHHS (NIH)
NSF
NSF review process, criteria Sec. 3
DOD
AFOSR review process, criteria Sec. 2.14
ARO review process, criteria Sec. 3
DARPA review process, criteria
ONR review process, criteria Sec. 5
USDA
NRI review process, criteria
NASA
DOE
ED
ED review process, criteria Sec. 5
Identify the Review Criteria
Most agencies publish their standard review criteria on their web pages and/or in their proposal preparation guides. However, it is important to note that some agencies assign additional review criteria to some of their special programs; these criteria will be delineated in the proposal solicitation, so it is also important to read this document.
Understand the Review Process
The review process varies – sometimes significantly – from one agency to the next. The review process may include a peer review, where outside experts from related fields are invited to review the proposal; an internal review, where agency personnel evaluate the proposal; or a combination of both. However, most agency review processes share some common features. At most agencies, for instance, an application will first undergo a merit review and, depending upon the results, an administrative review.
The review process begins when the application is received. The application will be logged into a tracking system, checked for completeness, routed to the appropriate division, and then to the appropriate program, where it will subsequently be forwarded to independent reviewers and/or to a review panel. Review panels vary in size, but will include experts from the various disciplines represented in the program. The independent reviewers and/or review panel will conduct a merit review of the application to evaluate the quality of the proposed research project and the degree to which it addresses the agency’s mission and meets the agency’s review criteria.
While the chair of the review panel will give all members of the panel access to all of the applications earmarked for that panel’s review, he or she will seldom ask all panel members to review every proposal in detail. More typically, the chair will assign a primary and sometimes a secondary and/or tertiary reviewer to each application. The primary reviewer is typically the individual whose field of research is most closely related to the applicant’s own. The reviewers will be asked to provide written reviews within a specified length of time. At NSF, for example, most program managers request a complete review within 90 days of when they send the application out to reviewers. In the case of a review panel, reviewers will evaluate the applications assigned to them, and then mail or e-mail their reviews to the chair of the review panel and/or travel to the agency’s headquarters to meet as a panel. During the panel meeting, the chair of the review panel will typically ask the primary reviewer to present his or her review to the rest of the panel, and will then give other members of the panel a chance to provide additional comments. When all members of the review panel who want to provide comments have had an opportunity to do so, the chair of the panel will typically ask the members of the panel to rank the applications and/or identify which ones are meritorious, i.e., worthy of funding.
After the review panel adjourns, some agencies mail or e-mail the individual review comments – and sometimes a summary of the comments and a rank or priority score – to the applicant. This information can oftentimes provide a very good sense of whether or not the application is likely to be funded.
Applications recommended for funding then typically undergo an administrative review to ensure that all agency requirements have been met and to allocate funds to the project. After this process is finalized, the award notification is sent, the contract is negotiated, and, finally, the contract is signed by all official agency and university representatives. Campus grant and contract administrators will lead the principal investigator through these negotiations.
Know the Reviewers
While it is important to identify the review criteria and to understand the review process, it is equally important to identify – as much as possible – who the reviewers will be. Knowing the reviewers’ scientific and/or scholarly background enables the applicant to ensure that the proposal is written at the appropriate level, provides the appropriate background information, and addresses the reviewers’ priorities and concerns. Some agencies set up standing review panels that include reviewers from a mix of disciplines, with some reviewers (especially at the mission agencies) being researchers who are full-time employees of the agency, others being researchers who have agreed to serve on a review panel for a designated period of time, and still others being researchers who have agreed to serve on a review panel on an ad hoc basis when a program manager determines that the regular review panel lacks the particular expertise required to properly evaluate an application. Agencies may also set up review panels to evaluate proposals to a specific program, or they may depend on mail reviews from ad hoc reviewers.
For the most part, those applying to larger agencies can count on independent reviewers having backgrounds related to the applicant’s research focus. In this case, the reviewers are the applicant’s peers, i.e., people who review their journal article submissions and attend the same scientific and scholarly meetings. However, while the reviewers will have expertise in the applicant’s field, it is quite probable that only one or two of these reviewers will possess expertise in the applicant’s sub-discipline. This is especially true on panels charged with reviewing multidisciplinary and cross-cutting proposals, such as those prepared to establish centers or programs; indeed, these panels will likely include at least some reviewers who are from completely different disciplines. Therefore, it is extremely important that the applicant write the proposal at a level that enables all reviewers to understand the important points, even though they may not understand all the discipline-specific details.
Some
agencies post rosters of review panels on their web pages. These rosters not
only list the names of the reviewers, as would be expected, but also designate
the period of time that each reviewer will serve on a panel. This information can be used to gain a
competitive advantage. With the names of the reviewers in hand, the applicant
can look up and read reviewers’ recent publications and thereby identify their
field of expertise, gain insight into their research interests, and determine
the extent to which their interests align with the applicant’s own. This
knowledge can help the applicant assess the likelihood that the proposed
research project will be well received. If it is determined that the majority
of reviewers will be likely to view the project favorably, the applicant can
proceed with confidence. However, if that is not the case, the applicant can
consider either submitting the application to another program or agency or
submitting it at a later time.
Write for the Reviewers
When preparing a grant application, it sometimes easy to forget that the application will be reviewed not by a monolithic agency, but by real people. Like the applicants themselves, reviewers are smart, accomplished, and dedicated, and may be struggling to balance the time they spend on their research, teaching, service obligations with the time they spend with family, friends, and community. In short, they are extraordinarily busy people.
Because reviewers are typically given multiple proposals to review at a time while still performing their “real” jobs, it is important that the proposal include absolutely everything the reviewers will need to read, understand, and evaluate the proposed research project with ease and efficiency. To this end, it is important to synthesize key concepts and to articulate the links between the overarching goal and the specific objectives, between the specific objectives and the hypotheses, between the hypotheses and the approach, between the approach and the expected outcomes, and, finally, between the expected outcomes and the significance and broader impacts of the project. In short, it is important to present the proposed research project with absolute clarity so that all reviewers will not only be able to understand the ideas presented therein, but also will be persuaded to advocate on behalf of that particular research project. Incorporating reviewer-friendly text, formatting, and graphics greatly enhances the competitiveness of a proposal.
Create Reviewer-Friendly Text
Techniques for developing reviewer-friendly text include the following:
- Divide the proposal into the required sections.
- Place the sections in the required order.
- Use parallel structure at both the sentence and section levels.
- Incorporate logical paragraph breaks.
- Open paragraphs with clear topic sentences.
- Discuss important items first.
- Avoid the use of inflated language.
- Use declarative sentences.
- Define potentially unfamiliar terms.
- Spell out acronyms and abbreviations.
- Employ appropriate style and usage.
- Use correct grammar, punctuation, and spelling.
- Run a spell-check and proofread the application.
Create Reviewer-Friendly Formatting
Techniques for developing reviewer-friendly text include the following:
- Follow page limitations for both the whole proposal and for individual sections.
- Conform to margin requirements.
- Conform to font and point size requirements.
- Incorporate headings and subheadings to provide a “roadmap” for reviewers.
- Incorporate ample white space.
Create Reviewer-Friendly Graphics
Techniques for developing reviewer-friendly text include the following:
- Make graphics large enough to be useful.
- Place graphics as close to the text they are meant to illustrate as possible.
- Refer to graphics in the text.
- Number and title all graphics.
- Prepare a caption for all graphics.
- Label axes and data points, as needed.
- Provide a legend, as needed.
- Provide color copies if color and/or color gradient are important.
