Grant Writing Articles- May 1, 2009
Grant Writing Articles:
Article 1: Developing Education Components for NSF Grants
(An interview with Dr. Jeff Froyd)
Article 2: Report from NSF conference
Article 1
By Lucy Deckard
Developing Education Components for NSF Proposals
An Interview with Dr. Jeff Froyd, Research Professor and Director of Faculty and Organizational Development in the Office of the Dean of Faculties at Texas A&M University
Dr. Froyd has been PI or co-PI on numerous NSF-funded projects that either focused primarily on engineering and science education or included large education components. He served as Project Director for the NSF-funded Foundation Coalition, which extensively reworked the first-year and sophomore-year engineering curricula. Dr. Froyd has also served as a reviewer for proposals to NSF’s Engineering Education and Centers Division in the Engineering Directorate
Q: What strategic advice would you give a new faculty member who is developing an education component for a CAREER or other NSF proposal?
A: An important thing to do when planning an education component, particularly for CAREER proposals, is to think about what will differentiate your proposals from those of others. What is your distinctive educational contribution? For example, you may propose to develop a new graduate course. That’s fine, but many other people are going to propose the same thing – what will make your proposal stand out from all the others? One thing you can do to make your proposal distinctive is to include a strong assessment plan that will allow you to add to the body of knowledge in STEM education. Another thing you can do is develop a plan that describes the materials (text, videos, exercises, projects, problems, etc) that will allow others to implement what you have done more easily.
Q: How should a PI go about developing an assessment plan?
A: First, think in terms of your goals for your educational component, and describe to yourself evidence that would help you to determine if they were met. For example, if you want to improve student learning in a course, how do you recognize when they have mastered the material? You would say that they need to “understand” the material. That’s a good start, but understanding is an internal mental state and we cannot observe understanding yet. So, ask yourself what you might observe to convince yourself that a student has understood the material. What things would you look at?
There has been a lot of work done on developing instruments to assess things such as critical thinking and cognition, and there are efforts funded by NSF and the Lilly Endowment to collect assessment information and tools and to make them generally more available. For example, the Wabash National Study focuses on assessing Liberal Arts Education, but much of this information can also be applied to STEM. You can find information on their outcomes and assessment instruments for different outcomes on their website [http://www.wabashnationalstudy.org/wns/instruments.html].
In cases in which you want to compare influences of your intervention, one place to start is where you have multiple sections of an undergraduate course. Then, you might collect data with assessment instruments that would allow you to compare how students perform in one or more sections that experienced the educational innovation, with students from other sections that did not experience the educational innovation. This is pretty easy if the sections already have common exams. If they don’t, you might collaborate with a faculty member teaching another section to give common exams. Another option is to compare student performance in a version of a course with student performances in previous years.
Often, you may need to involve others with expertise in assessment who can advise you on the use of a particular assessment instrument. In most universities, you’ll find faculty members with that expertise in the College of Education and Human Development. Exactly who you recruit will depend on what you’re trying to assess. For example, if you want to look at critical thinking, you would reach out to one group of people; if you need a survey instrument, you would reach out to people with expertise in surveys. If the assessment requires more qualitative research, you’ll want to reach out to folks who are experts in qualitative research methods.
If you are writing a CAREER proposal, this may be tricky since only the PI can be funded at the faculty level. However, you can ask a faculty member with this expertise to serve as an advisor to you on your CAREER project. They would then give you a letter of collaboration for your proposal. If a significant amount of effort might be involved, you could offer to provide part-time support for one of their graduate students, who would help you with your assessment.
Q: If a PI seeks to involve faculty who are experts in assessment, will this fit into their own research agendas?
A: Often times it does fit into their agenda, and in that case it can be collaborative project that will advance their research. Other times, they can connect you with experts who may not consider it part of their research but who can participate as practitioners: maybe a graduate student or others in their community who would be willing to participate on a part-time basis. For example, in a recent project I was working on, a faculty member in the Department of Statistics helped us to find a statistician in the community who helped us with our data analysis.
Q: What are examples of things a PI could do to develop a strong dissemination plan?
A: The PI needs to answer NSF’s question: What comes out of this education component? It might be the production of materials – curricula, course materials, lab manuals, software, etc. which are posted on a website – that enable others to implement the same educational innovation. It might be in the form of published papers describing what was done and documenting its effectiveness (which brings us back to assessment).
I think you also have to ask yourself what do other faculty members really want. Too often, faculty members focus on what they want to make available instead of asking themselves what they might want from faculty members who are working on educational projects in which they are interested. For example, a faculty member might make available text resources that describe how they organized the material or how they explained specific concepts. Faculty members may be looking for this type of material, but on the other hand, they may be looking for activities that students can do: projects, experiments, or simulations. A popular site, http://nanohub.org/ , got started as a site for faculty members to share simulations at the nanoscale, but now the site has incorporated social networking elements because faculty members wanted to interact about how they were using the simulations and what other things could be developed.
An Open Education Resource (http://www.oercommons.org/) movement is growing in the education community, and it can provide good avenues to disseminate course materials and tools. For example, Rice University has the “Connexions” project [http://cnx.org/ ], where you can post your materials, and people can try them out and review them. This brings the added benefits of peer review. The PI could commit in his or her proposal to put these resources where other people can access them and use them and then provide feedback. This approach fits well with a movement that’s starting within NIH and NSF to require that data generated by funded projects be posted in a publicly accessible way.
Q: Are there particular educational innovations that interest NSF?
A: In NSF’s Division of Undergraduate Education, the solicitation for their Course Curriculum and Laboratory Improvement program often provides insight into things in which they are interested; check out what has been funded in that program [go to http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5741&org=DUE&from=home and click on “Abstracts of Recent Awards Made” near the bottom of the page.] It is crucial that you know what educational innovations have already been tried in a discipline and which have been more widely adopted. In some subject areas, there have been numerous innovations, while in others, very little has been tried.
In the Division of Graduate Education, their main program is IGERT, but there are also a lot of ideas that are being discussed behind the scenes about where they see graduate education headed. I’d recommend talking to the program directors in that division to get their thoughts.
Q: What about technology-assisted learning?
A: There is a lot of interest in technologies that provide rapid feedback or allow collaboration among students, but, like any tool, there are right ways and wrong ways to use these technologies. Folks will say they want to use a particular technology in their classroom, but the question is, “why?” The educational goal should come first, and the technology should serve that goal rather than vice versa.
Student Response Systems (such as clickers) are an interesting method to provide more rapid feedback to students (see the review article by Fies and Marshall, Journal of Science Education and Technology, 2006). Another method is online homework questions. Getting feedback even just 2 or 3 days later can be discouraging, and everyone – not just students – appreciates more rapid feedback. (Think about how it feels to get reviews back on a proposal 6 months after you’ve submitted it.)
Online homework can generate problems with randomly varying numbers (here at Texas A&M, we use systems such as “Blackboard” (http://www.blackboard.com), but it’s still a challenge to generate questions that require graphs or chemical equations for balancing that use varying numbers. Online homework has been developed for some popular subjects such as first-year physics or calculus, but is not necessarily available for other subjects, so a PI could propose to pull together material and develop online homework for a new area. A project might involve pulling material together and putting it on-line for a new area (a good example of education tools posted online is on North Carolina State’s website “Tools for Schools” at http://ced.ncsu.edu/onlinetools/#Online TechnologyTutorials). You could also assess the effectiveness of using these rapid-response approaches.
Also, just knowing if the answer is right or wrong is not always the most valuable way to help students learn. Formative feedback helps students to discover where they are making mistakes. Researchers have discovered that there are a number of patterns of mistakes, and those can then be used to reveal what students are doing wrong. Software is being developed that can then help students to understand what conceptual errors they are making based on their wrong answers.
Another possible direction for a particular subject is to develop what is called a “concept inventory.” A concept inventory is a collection of carefully developed concept questions. Each concept question poses a scenario and asks students something about what will happen without asking students to generate numbers or formulas. Research on concept inventories has shown that subjects can successfully complete a course, even earn an A, but lack conceptual understanding of the subject. The best known concept inventory is the Force Concept Inventory developed by Hestenes and Halloun (http://modeling.asu.edu/R&E/FCI.PDF ) for the introductory physics mechanics course. Developing and validating an entire concept inventory would be beyond the scope of a CAREER proposal, but some initial work toward that goal might be proposed as part of a CAREER. For example, a researcher might propose to develop a set of concept questions dealing with several key concepts. For emerging fields, even identifying key concepts would be a substantive contribution. However, to do that kind of project, a PI would definitely need to engage people with expertise in developing concept inventories.
What about labs?
One promising innovation is to use a set of approaches that are variously called inquiry-based, problem-based or project-based approaches. There has already been a lot of research on these kinds of approaches, so be sure to read the literature so that you can build on existing knowledge. I wrote a white paper discussing promising approaches in this area for the Board of Science Education of the National Academies [http://www7.nationalacademies.org/bose/PP_Froyd_STEM%20White%20Paper.pdf ].
An inquiry-based example for a lab might be to get the students to design the experiment. The students work together to formulate the exact question and figure out a method to address the question. However, any time you pose more complex learning tasks, you must provide support and feedback. If you include this in your proposal, then be sure to figure out: what is my contribution in this area? Is it the materials, a curriculum, a lab manual?
Q: NSF places a lot of emphasis on involving undergraduates in research. Are there any innovative approaches to that?
A: Involving undergraduates in performing research is an excellent way to get them interested in conducting research, but it doesn’t scale up easily. What do you do if you have 50 undergrads? How do you deal with that? Some people have been working on models that allow a faculty member to expose large groups of undergrads to research. One approach is to have students work in groups, perhaps mentored by graduate students or postdocs. Chris Quick in the College of Veterinary Medicine here at Texas A&M has developed a program in which tens of undergraduate students can participate. More information can be found in a paper he and others have published [http://www.ncbi.nlm.nih.gov/pubmed/18539852?ordinalpos=4&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum].
Also, if you have a large number of undergrads, you may not want to put a lot of freshmen in front an expensive piece of lab equipment, but you could explain what it does and how it works and then have the student work on data generated by the instrument. This approach is an example of engaging students in inquiry-based learning, a type of teaching that engages students in authentic reasoning and thinking using typical tools, models or data generated by research. Bruce Herbert in Geology & Geophysics at Texas A&M has done a great job incorporating inquiry-based learning into one of his courses in a way that is similar to undergraduate research and allows Dr. Herbert to integrate his scientific research with his teaching. Another innovative approach is to expose undergraduates to a topic and then ask them to come up with research ideas. You can get the students working on sorting through the ideas, and some gems may emerge in that process.
Steve Balfour has taught over 200 students in an introductory psychology course at Texas A&M, and he created a website where students wrote and posted papers about topics in introductory psychology. The technology supporting the web site allowed the papers to be peer reviewed, and the original contribution and the peer reviews served as indicators of student learning. One student in the course generated a PhD-quality research question. Other professors have students create Wikis that are a synthesis of work in an area and have peers (other students) evaluate them. The quality of the material generated could be used for assessment. Remember that encyclopedias are high quality both because of who writes the articles but also because of the review and revision process. Working together in this way can be a valuable experience for students.
Q: Are there common mistakes that PIs should try to avoid when developing an education component for an NSF proposal?
A: One common mistake, which I discussed earlier, is proposing an education component that is not distinctive or innovative, so that there is really nothing that makes their idea stand out from what everyone else is proposing.
A second common mistake is to try to do it alone. Invest the time to talk to people rather than repeating mistakes hundreds have already made. A good place to start here at Texas A&M is our Center for Teaching Excellence [http://cte.tamu.edu/]. The CTE gives individual consultations as well as workshops, and you can even call them up and request a workshop on a particular topic if you don’t see it on the schedule, and if there’s enough interest they will develop one. Many other universities have similar faculty development centers.
A third common mistake is proposing an education innovation without knowing what has already been done in the area. This is another place where support from other people can help.
Finally, PIs may make a mistake proposing an educational component without asking themselves if this is something they are really interested in doing. If they are funded and then have to invest time and energy in a project that they really don’t believe in, it can suck the life out of them. Pick something that really matters to you.
Q: What about outreach to K-12 students?
A: I’m really not an expert in K-12 outreach, but I will offer one piece of advice: think systemically. If you want to have a strong impact on K-12 education, I think it’s most effective to influence the teachers, not just the students. By influencing a 5th grade teacher, you will affect all the classes of 5th graders that will come through that teachers’ classroom. You could also work with PTOs and parents. There’s also a movement to start putting engineering in the high schools and to increase the number of science classes most high school students take. If we want to entice more students to pursue science and engineering, these high school educational experiences need to be really good. This could be an opportunity faculty to reach out and help high schools. The College of Education and Human Development has quite a few researchers who work in K-12 science and math education and the College of Science has experts in mathematics and science education. They can be very helpful in advising you on how to plan and execute a K-12 education plan and in helping you to connect with schools.
Article 2
By Lucy Deckard, John Ivy & Robyn Pearson
Report from the NSF Regional Grants Conference, held March 30 – 31, 2009
NSF offers regional grants conferences twice each year at various locations across the U.S. aimed at faculty and research administrators. In these conferences, they talk about NSF’s programs and priorities, discuss new directions, and hold breakout sessions that cover each directorate as well as specific programs such as CAREER and Integrative Activities. This is a great way for new faculty to learn about NSF and to meet and talk to Program Officers.
NSF recently held a regional grants conference at Arizona State University, which several members of OPD attended; highlights are discussed below. In addition, NSF has posted conference materials, including presentations by NSF Program Officers, from their March 2009 Regional Grants Conference - NSF Regional Conferences and Workshops (go to http://www.nsf.gov/bfa/dias/policy/outreach.jsp and scroll down to "Presentations from Recent Events" for links to presentations made at most recent NSF Grants Conference).
The next NSF Regional Grants Conference will be held on October 5 -6, 2009 at Jackson State University in Jackson, Mississippi. Registration usually opens a few months before the conference; a link for registration will be posted on the website mentioned above.
Below are news and highlights from the conference.
American Recovery and Reinvestment Act (ARRA)
New Solicitations - NSF will be issuing three new solicitations based on ARRA funds as follows:
· A special Major Research Instrumentation (MRI) solicitation – the solicitation is currently in “clearance” at NSF and they expect it to be issued some time in April with a likely July due date [Note: as of 4/23/09, the solicitation had not yet been issued]. It will be very similar to the MRI solicitation for the last regular competition that had a January 2009 due date (http://www.nsf.gov/publications/pub_summ.jsp?org=NSF&ods_key=nsf09502 ), although there may be some minor changes in cost sharing requirements, the amounts that can be requested and perhaps some other minor requirements. It will still have limits on the number of proposals per institution.
· A resurrection of the Academic Research Infrastructure (ARI) program – this solicitation is also expected out some time in April, with a due date around July [Note: as of 4/23/09, the solicitation had not yet been issued]. The NSF program officer said that it will be a revision of the existing program that was last competed in 1996, so anyone who wants an idea of what it will be look should look at the old solicitation (http://www.nsf.gov/pubs/stis1995/nsf968/nsf968.txt ). The key thing to remember is that the intent of the program is to modernize existing research facilities, not to build new research facilities. It typically will not support renovation of buildings. An example of what this program might support would be to modernize an old lab that currently has outdated equipment. This will also have limits on the number of proposals per institution.
· The new Science Masters program. There was less information available on this program, although it was briefly mentioned that it will support terminal masters programs and that a new solicitation should come out within the next couple of months.
Eligible Proposals and Funding Rates - The American Recovery and Reinvestment Act (ARRA) made an additional $3B available to NSF. Of this, $2B will be made available to proposals that are already in house and will be reviewed and/or awarded prior to Sept. 30, 2009 and for proposals that have been reviewed but not funded since Oct. 1, 2008. In answer to a question about how ARRA funds might affect funding rates for proposals that have not yet been submitted (e.g., the CAREERs due in July), the program officer replied that although the ARRA funds will be committed relatively quickly on proposals that are already in-house and on the three new solicitations mentioned above, it is anticipated that funding rates for many programs will rise next year since other funds that might have been committed this year may be freed up and since there may be fewer resubmits. This is a guess, though, since they don’t yet know what their budgets will be.
Contacting Program Officers about ARRA - NSF’s mantra is “Ask Early, Ask Often!” They repeatedly stress the important of speaking with your Program Officer to determine if your research is a good fit with NSF program priorities and to identify other funding opportunities that may be appropriate. However, it was emphasized many times that NSF would prefer that PIs not contact Program Officers to ask questions about ARRA (such as whether the PI’s pending or recently-declined proposals might still be funded, or about how the ARRA funds will affect particular programs). POs are overwhelmed and are not replying to such requests. They are in the process of going back through well-reviewed, unfunded proposals to decide which to fund, and they are not soliciting PI input at this point (the message: Don’t call us, we’ll call you). In fact, they said if a PI needs to contact a PO about a subject unrelated to ARRA, they should put in the subject line, “This is not about ARRA” in order to get the PO to read the e-mail.
For more information on ARRA and NSF, go to http://opd.tamu.edu/arra/national-science-foundation.
America COMPETES Act
The America COMPETES Act (ACA) of 2007 is a mandate to increase federal science and engineering R&D over a period of 7 years. Under ACA, congress set the goal to approximately double NSF’s budget over 7 years. NSF is implementing four sections of the Act that affect proposal writing and the research environment:
· SEC 7008 – Mentoring of Postdoctoral Research Fellows. NSF requires a one page mentoring plan for all proposals that support postdoctoral researchers. This one page plan requirement, now in effect, should be attached as a supplementary document. (For more information on this requirement and resources for addressing it, go to http://opd.tamu.edu/proposal-resources/resources-for-postdoc-requirement/resources-for-addressing-nsfs-postdoctoral-mentoring-requirement .)
· SEC 7009 – Responsible Conduct of Research. All NSF-sponsored research or education will be required to have a plan for the training and oversight in the responsible and ethical conduct of research for undergraduates, graduate students, and postdoctoral researchers participating in the sponsored activity. Beginning Oct. 1, 2009, the Authorized Organizational Representative must certify that the institution has a plan in place. A training plan is not required to be included in the proposal. NSF will support development of online resources for RCR training.
· SEC 7010 – Reporting of Research Results. Section 7010 requires that
o Citations of published research documents resulting from NSF-sponsored research be made available to the public in an electronic form through NSF’s website, and
o Final Project Reports will include a new summary specifically for public release.
NSF expects to implement these changes in Fall 2009.
· SEC 7013 – Cost Sharing. ACA required that the National Science Board evaluate NSF’s policy to eliminate cost sharing. A preliminary report recommended cost sharing for
o Experimental Program to Stimulate Competitive Research (EPSCoR),
o Engineering Research Centers (ERC), and
o Industry-University Cooperative Research Centers (I/UCRC).
A secondary report made nine recommendations, still under Inspector General review, with two primary objectives:
o To allow, but narrowly circumscribe, mandatory cost sharing requirements in NSF programs, and
o To prohibit voluntary committed cost sharing and eliminate post-award tracking and reporting requirements.
Other news
· The Chemistry Division in the MPS Directorate is fine-tuning its topic areas for the core programs. The new topic areas will be posted on their website this summer.
· In the Division of Materials Research, they plan to mainstream nano research (meaning it will be absorbed into their core programs), so no new general nano-related competitions are expected out of that directorate, although there may be some targeted nano-related solicitations.
· In the CISE talk, the program officer pointed out that the Computer Research Infrastructure program will fund equipment purchases for CISE-related research, and the competition is typically low for these grants. He also pointed out the FODAVA program (Foundations of Data and Visual Analytics), which just started last year and focuses on developing methods for taking high dimensional data and reducing it so that it can be visualized on a computer screen. This is a partnership with the Department of Homeland Security, and the idea is to apply this research to intelligence analysis.
· In addition to CAREER awards targeted to new investigators and meant to provide a foundation to foster and integrate research and education, some directorates have special programs for young investigators. For example, BIO offers Research Initiation Grants which are intended to increase the diversity of researchers.
At the BIO talk, the program officer indicated that success rates for Doctoral Dissertation Improvement Grants, supported by the Divsion of Environmental Biology and Integrative Organismal Systems, is high: 40% and 30%, respectively.
