Hope you had a nice break! Thanks for the invitation — I’ll be there. I think that’s the same day as the Compound Interest session and the associated poster session that evening. Portland should be fun. I look forward to meeting you, too!

Britt

Janice

I will invite my colleagues who I know have stories about panelists being invited to serve based on their expertise in education and public outreach. The CAREER proposal solicitation jumps to mind as one where this has happened. JIm Yoder also has contacted some NSF program managers to weigh in on the discussion and offer their thoughts on the points you raise.

More soon

I get the idea — and think it’s a good one — of partnering the Zephyr Education Foundation with scientists who are seeking NSF funding as a way to help those scientists enrich the BIC aspects of their proposals. A couple of thoughts occur immediately:

(1) Do you have a plan to make sure that the educational expertise you provide is incorporated into the research being proposed/conducted by the scientists, as opposed to being a mere add-on element?

(2) Do you mention BIC on your site at all? I glanced around quickly and didn’t see any explicit discussion of broader impacts.

(3) Do you have, for lack of a better term, a “marketing plan” to let scientists know about your organization?

Britt

That’s excellent news, and a great project! I will be at the Ocean Sciences Meeting in Portland in February at a session organized by some folks you no doubt know (including Liesl Hotaling): http://www.agu.org/meetings/os10/program/scientific_session_search.php?show=detail&sessid=153.

If you and others interested in talking in person, and since I see from the program that you and Jim also will convene a session there: http://www.agu.org/meetings/os10/program/scientific_session_search.php?show=detail&sessid=169, maybe we should try to organize a meeting to talk about how to pool our resources.

Britt

Great! I think we’ll have lots to discuss.

Yes — lack of consistency in how BIC is reviewed is a recurring issue, at least according to anecdotal evidence. We’ll have more hard data on this once we complete our current research. Interestingly, although we have often suggested the possibility of using BIC “experts” — the scare quotes you use here are essential, since the very idea of expertise in broader impacts is contested, even if it is clearly possible to identify experts in one or another area related to broader impacts — as reviewers, at least in an ad hoc fashion, we have consistently encountered resistance to the idea.

So, I’d love to hear from you and anyone else who knows of anyone who has been asked to serve as a reviewer for NSF specifically to address broader impacts.

In my experience, however, people (including program officers, proposers, reviewers, and just about anyone else you can think of) and agencies tend to think about peers along disciplinary, or even subdisciplinary, lines. (There are some interesting exceptions to this that we’ve found, however. More on that if anyone’s interested.)

My own opinion is that BIC considerations ought to be incorporated starting with the proposal writing stage, included during the review process, be integrated with the actual research, and continue well past the dissemination of the research findings (e.g., it would benefit everyone if we had long-term data on outcomes, as well as a set of direct research outputs). There are, of course, institutional barriers to such an approach; but there are also some institutional enablers, such as BIC itself. There are issues of what one might call “compliance” with BIC — including inconsistencies on the part of some POs and reviewers, as well as proposers — but there are also issues how the merit review process is designed (for instance, the issue of whether to weight broader impacts more, less than, or equally with intellectual merits, or not to specify a weighting) that affect “compliance.” So, it’s not just a matter of how the process is implemented; and it’s not only a matter of how the process is designed; it’s some combination of both of these, plus lots of other factors (such as the individuals involved, etc.). These are just the sorts of issues we are trying to draw out in our research.

Thanks, Janice and Jim for getting this started. I hope others out there are interested in carrying on what I think is both an interesting and a vital conversation!

Britt

We are in fact planning to update the survey next month. We will be launching it to two different ocean professional organizations. We definitely want to compare results.

In addition, we are in the process of setting up case studies at each of our centers to supplement the survey data with more rich examples from different types of scientists (career stage, level of engagement in EPO, etc). This work will be starting in the next few weeks and you will be hearing about it on this blog site.

I completely agree that graduate students are perhaps the most important target audience for discussion and education about BIC. We also are involved in a project lead by Annette de Charon from University of Maine where we will be developing tools for getting graduate students engaged concept mapping and thinking about how to use concept mapping to improve the transfer of knowledge to public audiences. You can view concept mapping tools at http://cosee.umaine.edu/.

One of our biggest challenges in offering support to scientists around BIC writing is the lack of consistency in how BIC are reviewed and considered by NSF. It is almost completely up to the program manager whether they include BIC in the panel review process or not. We have heard stories that range from not considered, discussed at all in panel at all to bringing in someone specifically on the panel to serve as th BIC “expert”. Do you find the same lack of consistency in your work? We look forward to hearing more about your work. Please keep us informed!

Janice