Understand the SBIR review process for applications after submission, from application completion to receiving a final IMPACT score.
Attempting to understand the Small Business Innovative Research (SBIR) review process as a beginner can be a difficult task. It is hard to grasp the intricacies and nuances of each review session, as every reviewer and application is different. Knowing where your score came from and what it means can help you build a better revision or future application. This article will outline the basic review process and teach you how to leverage that process to your advantage.
🔬Learn more: What are SBIR/STTR Grants?
The first thing to try to understand is that you are writing your application for reviewers. This means that your application is read by real people in the industry, three per application. The agency you submit to will attempt to match reviewers to your application that have the most relevant expertise, but that does not mean that always happens.
The most common agencies are the National Institutes of Health (NIH), the National Science Foundation (NSF) and the Department of Defense (DOD). This article highlights the NIH review process.
🔬Related: 8 Tips for Writing a Winning SBIR
You should also understand the basic timeline of a submitted SBIR application. First, it is sent to the Center for Scientific Review (CSR). The CSR is the group that assigns reviewers to specific applications. They will assign your application to three reviewers (with relevant expertise/experience) who will read your application at home or on their own time.
The 3 reviewers will then give your application an initial score. Based on the stratification of the initial scores, applications are sorted into groups that are discussed or not discussed in a group review setting. This process cuts the number of applications by about 30%. Then, a larger group will discuss your application for 1 day, and will score it again. Based on those scores, another 20% (of the total) applications are dismissed from the process. This means that only around half of the applications submitted are discussed on the second (and final) round of review.
As stated before, around 30% of applications will receive an ND when their application is returned. ND stands for Not Discussed. This means the application was not discussed by the entire review board and only read by the 3 initial reviewers. This can be disheartening because it means you may need to reevaluate your innovation and whether or not you want to apply again.
Each SBIR application is discussed for an average amount of 15 minutes. This may seem like a short amount of time, but the 3 initial reviewers are responsible for filling the other reviewers in on their initial opinions and overall summary. They can ask questions and clarify problems as a group. The primary reviewer and the other 2 reviewers set the tone for the discussion; this means that if the 3 of them did not deem it acceptable, it is generally unlikely the rest of the group will. An open discussion follows the first 3 reviewers’ opinions, then the chair summarizes this discussion, and repeats it back to the group. Finally, all of the reviewers will assign their final score and they move on to the next application (for up to 9 hours per day)!
🔬Related: 10 Tips for a Competitive SBIR or STTR Proposal
The IMPACT score is the main criteria used by the NIH to decide on funding a project or not. It takes into account the subscores of the 5 axes listed below, but is not a numerical average of the 5. The IMPACT score is a ‘gestalt’ by the reviewers, or a best guess on how they see the project impacting the community. The score is from 10 (best) to 99 (worst). The score can be quite subjective and mysterious in a way. Even if the 5 individual scores above are satisfactory, it does not mean that the IMPACT score will be the same, which often confuses applicants.
There are 5 main scoring axes that an SBIR reviewer looks for. Each of these are important for the overall score assigned to the application:
The NIH uses a 9 point scale to grade applications overall. The scale is ‘backwards’ to the traditional way we score, 1 being the highest and 9 being the lowest. A score of 9 is described as having very few strengths and numerous major weaknesses. A score of 5 is described as strong but having at least one moderate weakness. A score of 1 is rarely given but describes an application that is exceptionally strong with essentially no weaknesses.
Some refer to the scale as being from 10-99, which is generated simply by multiplying the score by 10. For example, a 3.2 would be a 32.
In most cases, scores from around 1-3 are funded by the NIH. Even 3.5s can be funded depending on the division, department, or necessity level. Scores form around 3-5 are encouraged to resubmit, as they normally have only a few errors that can be fixed. A score of 6-9 may need a reconsideration on the part of the PI (Principal Investigator). High scores like these are given out less often, so there may be a fundamental issue with the application.
An important note to keep in mind is reviewers are not responsible for allocating funding. They are solely responsible for giving the projects scores on a 1-9 scale, and the funding is determined at a later time.
Scores tend to cluster around 3-5. It would be hard to determine the exact cause of this because each reviewer has a different mindset on each application.
🔬Read more about: How Incubators Can Improve Your SBIR/STTR Grant Application
A payline is the score cutoff that designates projects that do and don’t get funded. The issue with paylines is that they can never be completely accurate, as it is ultimately up to the funding department (not the reviewers) to decide who gets funded and who doesn’t. The average payline is around 3 (as stated above) but that does not mean that a project with a score of 3.2 or even a 3.5 can not be funded.
The summary statement is the product of the review board’s discussion of an application. It is described officially as the “Overall Resume and Summary” of a review discussion. The notes on this statement are very helpful because they can reflect opinions of the reviewers beside just the raw score. Here are the main aspects of a Summary Statement:
The Summary Statement is different from the written critiques provided by the initial reviewers, as it reflects the opinions of the entire review board.
There are two primary ways to respond to a reviewer’s comments about your application: rebuttal or revision. In a re-submitted application, applicants are allowed one page to explain any changes they have made or respond to some of the criticisms of the first review. Rebuttal can involve challenging a specific comment from a reviewer, however directly or indirectly you see fit.
Or (what typically works better) you can make a change in your application to fix what the reviewer saw as incorrect or problematic. The same reviewer, or another assigned reviewer, will see the change in the next round of advised application. This might improve your chances of being funded the second time around because it shows you are committed to developing your innovation to the best it can be.
🔬Learn more about: How to Create a Competitive SBIR Application
Here are some resources you may find helpful while writing your NIH SBIR application.
Q: Which reviewer concerns should you respond to first in a revised application?
A: You should respond to those most prominent in the Resume and Discussion section. You should also try to select concerns that are common between reviewers, because that is likely the most important change(s) you need to make.
Q: Is there a difference between the NIH, National Science Foundation (NSF), and Department of Defense (DoD) review processes?
A: This article is mainly about the NIH review process, but the process for the NSF and the DOD are very similar. Check out their SBIR websites here: DoD and NSF.
Q: What is the best way to highlight changes in an SBIR resubmission?
A: In re-submitted applications, you are allowed a full page to elaborate on your resubmission. This page is called the “Introduction to Revised Application”. You can use this page to highlight and respond to comments from the Summary Statement.
Q: How should first time applicants go about writing their SBIR?
A: First time applicants are often at a disadvantage because they do not have credibility with the SBIR program yet. You may require writing help from consultants like those at ScienceDocs, or you may want to create an established review board to solidify your innovation.
Q: Do I need a Letter of Support?
A: Letters of Support are extremely valuable to your application, and they are even required for a Phase II application. They give you credibility and let the reviewers know your application is worthwhile.
This content comes from a 4 week webinar series, hosted by University Lab Partners in conjunction with ScienceDocs consultant Dr. Jim O'Halloran.
Dr. O’Halloran has more than 17 years of experience with the SBIR program and has received funding from the National Institute on Aging (NIA), National Institute of Mental Health (NIMH) and the National Institute on Neurological Diseases and Stroke (NINDS). He has served as principal investigator on more than 20 grants, totaling more than $10M.
Be sure to subscribe to the ULP Youtube Channel to never miss another webinar, and connect with us on LinkedIn to stay in the loop!
Do you have a great company in the bioscience or medtech industry? Do you need wet-lab and/or fabrication space to develop and test your product?
