things are often not what they seem to be...

Things are often not what they appear to be.

This is a good, basic rule of life, not just one of those things you say when you don't have anything to add except a shrug.

Nature had a very interesting article about the Broader Impacts criterion, one of two criteria that the National Science Foundation uses to determine whether grant proposals should be funded.  I'm not just saying that because I'm quoted in it, either. Corie Lok does a great job establishing the issue in the first paragraph.

"Research-funding agencies are forever trying to balance two opposing forces:  scientists desire to be left along to do their research, and society's demand to see a return on its investment."

NSF requires that every proposal - research, education or outreach - include a plan to promote the 'broader impacts' of the project, with broader impacts basically meaning "How does this work benefit society?"  Beyond your publishing papers and the basics of preparing graduate students for careers, what are you doing that helps us explain to the taxpayers and the Congress why it is so important to fund science?  There is a lot of misunderstanding of the Broader Impacts criterion in the community and, I think, within NSF itself.  Still. (Note added:  I run the broader impacts toolbox site, which was funded by a grant from NSF, but isn't an official site.  There is a lot of information on the criterion itself, along with some reports.)

The reason I raise the issue is that a recurring theme of this blog is the image of scientists as portrayed in the media.  Just after I arrived at the University of Nebraska, my department chair and his wife went for dinner and a movie with myself and my husband.  Sue Kirby, who became one of my favorite collaborators, was an elementary school teacher with a deep love of science.  The other three of us were university physicists.  We went to see the movie "The Saint".  If any movie could have benefited from the Science and Entertainment Exchange, this would be it.  A basic premise of the movie is that Elisabeth Shue has developed a valid way to do cold fusion that will permanently solve the energy crisis.  She is carrying the secret around on six little post-it notes that she hides in her bra. 

OK, maybe that's just a plot device so that the Saint (Val Kilmer) can seduce her in order to steal the notes.  Or maybe some misguided writer thinks that science is actually so simple that one person discovers something and can write it down on six tiny pieces of paper with no lab notebooks or computer files or anything.  Anyway, poor Sue was mortified during the movie because the three of us were laughing so hard that people were starting to look at us and wonder if we were all on something.  We couldn't help it.  The portrayal of science was so outrageous that it was fall-down funny.  Come to think of it, the four of us never went to another movie together after that...

And then last week, I saw the same movie on television.  The hosts were offering helpful commentary at the commercial breaks) and had an exchange something like...

SHE:  "Are you cold?"

HE:  Yes - that's all due to the beautiful and extremely intelligent Elisabeth Shue.

SHE:  But she's not cold, she's hot.

HE:  Yes, but we all know there's no such thing as 'hot fusion'.

Uh...you mean like what happens in the Sun?  I was watching movies because I was home with a respiratory infection, so the laughter was a little limited because my ribs hurt every time I had to breathe in.

This relates to Broader Impacts because of a story I told for the Nature article.  My collaborator Gayle Buck (a science education researcher now at Indiana University) and I were doing a pilot project in which we had three or four graduate students working with Sue Kirby's fourth-grade class on circuits.  We were planning a grant in which we wanted to study the impact of contact with real scientists on student images of science and scientists.  The program to which we were writing had a goal of teaming graduate science and engineering students with K-12 teachers, so we had recruited a few graduate students -- all of whom happened to be female -- to come and work with the kids.  We didn't set out to get women students, those were just the students who were interested in participating.  Our goal was to see what the students learned about the process of science in their quest to make a bulb light with just a battery, a bulb and a single piece of wire.   

About halfway through the process, as I'm standing there watching with a smile as bulbs are lighting and students are saying "cool" and smiling about how they understand science, Gayle approaches me. 

"Guess what?" she asks.  "The students don't believe you're scientists."

She had been interviewing students out in the hallway and asking them questions I never would have thought to ask.  Like "Who are these people helping you?"

And darned if the fourth grade students weren't overwhelmingly positive that the women graduate students in the class could not possibly be scientists.  Even with prompted with "could they be scientists?", the kids had all sorts of reasons why they weren't.

"They're too pretty.  Pretty women wouldn't be scientists."

"They smile too much."

"They talk in ways we can understand."

"They act like they want us to understand them."

In short, the students were sure that the women in the classroom helping them -- all doctoral students actively working in labs -- were student teachers.  Gayle, Sue and I ended up writing a paper about the study - we tried a bunch of things to reinforce the idea that the women were, in fact, scientists.  We videotaped them in their labs explaining their experiments, we mandated that they be called "scientists" or "engineers" and not "graduate students" and we even bought a button machine and made them nametags with "Scientist" in big letters.  I was outvoted in my idea to tattoo the word "Scientist" on their foreheads.  Some issue about IRB considerations or something.

That was a pivotal moment for me, because it made me realize that I had spent a lot of my career doing things for which I had absolutely no evidence of any impact, let alone a positive impact.  I've spent my life being told that I should be out there and visible as a role model for women in physics, when the fact is that I may have been talking to an audience that thought I was there to explain to them what scientists do and what science is like without it registering that there was actually a scientist in front of them.  My skepticism about the Broader Impacts criterion is in part because it fails to consider that having a limited number of projects that are well thought out and assessed might be a much more impactful way of addressing the issues than having a lot of well-intentioned scientists doing things that could have minimal, if not negative impact. 

Changing the perceptions people have about science and scientists is nowhere near as easy as putting good role models in front of them.  Sure, you will hear scientists in interviews talking about how Neil Degrasse Tyson, or Eugenie Scott (or my personal hero, Laurie McNeil) inspired them when they were young and doubtful about their ability to succeed, but that's a minority of people.  We learned, for example, that the fourth graders got much of their ideas about who does science from the cartoon 'Dexter's Laboratory', which was big at that time.  (The study was done at a school where about 78% of the students qualified for free or reduced lunches.  Few of the students in our study were likely to have relatives who were scientists or other first-hand experience with scientists or people they recognized as scientists). 

Jennifer and I had a short e-mail volley about a blog that was posted in response to a question about what it is like to be a scientist.  One quote from the blog in question:

The only scientists you’ve probably met are the ones you see on TV or in the movies. Who’d want to be any of those? Who wants to be Dr. Doofenshmirtz from Phineas and Ferb or Flint Lockwood from Cloudy With a Chance of Meatballs or anyone on The Big Bang Theory? Flint and those painfully nerdy guys from The Big Bang Theory are nice enough but few people would want to be them. And Doofenshmirtz is lame and evil.

OK, I've had animated discussions myself with David Saltzberg, the scientific advisor for The Big Bang Theory who periodically blogs about the science in the show, about why they don't have a gorgeous non-geeky woman scientist on the show.  (And if the writers aren't sure how one would behave, I can give you a list of people they can visit.)  David assured me that people like Sheldon and Leonard.  My experience with the fourth graders makes me skeptical without a little more study.  I certainly don't endorse the blogger's comments about 'lazy writers' using scientists as easy targets.  Anyone who's worked with writers for television or the movies knows that a 'lazy writer' remains unemployed.  Busy writers on deadline without even any idea of who to ask might write something that 'sounds' scientific without checking it, but most writers are pretty dedicated to getting things right.

Jennifer reeled off a plethora of scientists, mathematicians and engineers on fictional programs (i.e. entertainment, not documentary or educational) that counter the points made in the blog.  I doubt it took her more than two minutes to come up with those names.  But my experiences with the kids make me wonder whether my perception of those characters is in any way similar to the perception of the average television watcher.  Do these characters have any impact (much less positive or negative) on the way people perceive scientists?

I propose a study of what the average person -- not me, Jennifer or probably most of the people who read this blog -- thinks about scientists on television, becoming a scientist or even knowing scientists.  I'm somewhat hampered by the fact that time has prevented me from watching a lot of popular television in the last year. But on the plane home last night, I was thinking about some of the questions I would ask. 

I would pick a couple different cohorts:  one of people who are scientists, one of people who work with scientists, one of people who aren't scientists, but say they like science, and one of people who aren't scientists and say they don't like science.  I'd ask them all the same questions.  And if there is one thing I've learned about studying people, I've learned that qualitative research (i.e. on a scale of one to seven where one is strongly disagree and seven is strongly agree, how do you feel about the following statements) is very limited.  So I'd give them a web-based survey with a lot of open-answer questions and then I'd follow up with an interview in which you could probe some of the common themes that emerged from the survey. 

OK. In reality, I'd enlist the talents of my friend Vicki, who is one of the best mixed methods researchers in the country.  She'd take my ideas, add her own, and put them into a useful form from which something valid might be deduced.  A good collaborator is a blessing, especially when they are also a dear friend.  So these are just the ideas off the top of my head.

Maybe you'd like to try this yourself:  The ground rules are that we're dealing only with fictional television programs.  No NOVA, no news, no documentaries.  Dramas, comedies, cartoons are all fair game.  Let's consider 'recent' as being the last two years or so.  I won't impose a hard cut-off, but don't pull out shows that were canceled five years ago.

1.  List all the recent television programs you can think of that have scientists or mathematicians on a recurring basis.  (Not the Law and Order episode where there was a scientist killed, please.)  Only characters who have names are eligible.  (Not the one listed in the credits as "Scientist #1".)

2.  List all the characters on those shows who were scientists and as much as you can remember about the particulars of the character.  Name, type of scientist, relationships with other characters.

3.  For each of the characters you listed, rate on a scale of 1 to 7 with 1 meaning strongly disagree and 7 meaning strongly agree (I didn't say that was useless or invalid, just not complete).  I'd ask some specific questions like:

    a.  I would aspire to be like that character

    b.  I would invite that character to a party that included my friends and family

    c.  I would date/marry/sleep with that character

    d.  That character is a good role model for people in terms of encouraging them to become a scientist.

    e.  That character makes scientists look evil

    f.  That character make scientists look stupid

    g.  That character would be a good mother or father

    h.  That character causes all of the trouble he or she gets into by things that he or she does.

I probably should plan to disguise the purpose of the study by asking similar questions about doctors or lawyer or firefighters. Maybe veterinarians. 

That would be my start -- and I wager that the analysis of those answers would raise another series of questions to answer in the interviews. 

And, of course, that's the way science works.  Sometimes, the mark of good research is that it raises more questions than it answers. 

Now, I just have to find some funding for this study!

Comments

[Danna Staaf]

What a neat idea for a study! Comparing responses about scientists to those about doctors or lawyers would do more than just disguise the purpose, I think, it would provide a great context for the public perception of scientists as compared to other professions.

I'm curious about the fourth-grade circuit class. The kids may not have believed the grad students were scientists, but did they learn some actual science? I wonder if it's better to create separate projects to address those two goals, or if it's desirable to have the same project teach both who scientists are and what science is . . .

Curious tangent: I've done science outreach throughout grad school, and I think it was only somewhere in my fourth year that I started telling kids "I'm a marine biologist" instead of "I'm a grad student." I've had several conversations with fellow grads about how we identify ourselves to people we meet--as scientists or students? There's a lot of uncertainty.

[Diandra Leslie-Pelecky]

Chad Orzel has a nice post about Broader Impacts at: http://scienceblogs.com/principles/2010/06/the_problem_of_broader_impacts.php where he touches on a lot of good points I didn't get to in this article.

[Diandra Leslie-Pelecky]

Hi Danna: Education research is so hard! The kids learn more by doing rather than listening to and it's unclear to us whether the kids learned more because they had scientists working with them or because there were more people in the room and thus each set of kids had more help. I mean, one person trying to do anything hands on with thirty fourth graders is insane.
The place we've seen an impact from the scientists is in middle and high school classes, where the scientists are real-life models of the scientific method when something doesn't work the way it is supposed to. Scientists are inherently comfortable with things not working because (let's face it)that's the way our real lives work most of the time, right?
The scientists start troubleshooting and that is where the kids really learn something about discovery and attacking a problem that you don't already know the answer to.
Our graduate students had similar qualms about calling themselves scientists. They felt (especially the women) that they were somehow being presumptuous using that title before they had their degrees. I think it's really important that we make sure grad students understand that an important part of their education is becoming part of the community. That's a process, not a step function that happens when you defend your thesis. You become part of the community when you start your study and you strengthen those bonds every time you read a journal article,give a paper at a conference or have a scientific talk with someone in a hallway. We are all, even twenty years after getting the Ph.D., in the process of becoming scientists. You ARE a scientist and I encourage you to make that point when you are doing outreach.
Sorry - bit of a rant here, but after nine years of running a project that places graduate students in the schools (www.physics.unl.edu/~fulcrum), I have so many results I haven't written up yet that really need to get out there!

[Lynette]

I don't watch a lot of TV, but the one show that does come to mind? Eureka. Yes, they take some liberties, but the show is about scientists. A non-insignificant number of them are female scientists, which makes me happy. The entire lab? Run by a woman who is smart, struggles to balance work and home (like the main male character), is a parent - all the kinds of things real people have to do.

[Pat Dixon]

In a presentation for teachers, I recently used the images on television to discuss nature of science and the idea that students come to classrooms with preconceived notions of what a scientist is and what he or she does. These notions affect the way children think about and learn science. In addition precious few of the images depict women. Google images of scientists from cartoons, movies and television and see what happens!

I would add to your proposal, Diandra, that we survey elementary students -- it would certainly add to the data that's been collected by many through the "draw and scientist" protocol. Not one of my favorites because it never parses out the real issues -- it's almost a foregone conclusion because of the medium.

As one who has struggled with these issues over many years of formal and informal science education, I'd love to work with Vicki and whoever else is interested -- Vicki has certainly added her expertise to our own research at the Magnet Lab!

[liz]

Johnny Test's twin sisters are genius scientists who use Johnny as a lab rat. It's one of my 8-yo son's favorite shows. And the sisters are gorgeous. Cartoons, but gorgeous.

So...maybe the next generation of 4th graders will associate gorgeous red-heads as scientists.

[Radu Prisacaru]

Finally this is my first time I pop in here. I discovered a lot of remarkable substance on your webpage specially its conversation. Keep up the good work! I invite you to see my post, I hope you will find interesting too.

[Danna Staaf]

Thanks for the responses, Diandra! This is great: "The scientists start troubleshooting and that is where the kids really learn something about discovery and attacking a problem that you don't already know the answer to." Totally! I also love it when kids ask a really basic natural history question (usually about some obscure marine invertebrate in the tidepools) and I can tell them: NOBODY knows the answer to that. You could be the one to find out!

Project Fulcrum sounds very cool. I look forward to hearing more about it in future posts/articles/etc.

[Terry Bleizeffer]

Who wouldn't want to be Dr. Doofenshmirtz?! He's a guy who clearly loves his work and doesn't give up in the face of adversity. I doubt he'd score well on the love interest scale, though.

Regarding your study, it seems to me that you're going to run into an issue of distinguishing between how people feel about a person and how they feel about the person's profession, which won't be the same. To take an extreme example, it's easy for someone to think that Bob Knight is a fantastic basketball coach while simultaneously thinking he's a pluperfect jerk. Likewise, people will like George Clooney even if he's playing a criminal... or worse, a scientist!

So just like your grade school experience, the question is whether they attribute any positive or negative impressions of the character to the fact that they're a scientist. To make it worse, many people probably have a very constrained/distorted idea of what a scientist is. For example, my gut tells me that most people don't think of the CSI characters as scientists... they are law enforcement. But they probably think the hackers in Transformers were scientists... because they were smart and worked with computers or something.

Like you said at the bottom, you'd want to disguise the study by including other professions as well. I would take it a step further and not mention professions at all and ask respondents to self-report what profession the character has. But you'd still have to figure out how to tie the responses to the profession, and I don't know how to do that.

[Oliver]

I would like to mention one of the more 'famous' scientists on TV:

Prof. Frink on The Simpsons

Just my $0.02, lol!

[Joana Johnson]

This was a very interesting post. I'm not a scientist (I'm an Historian), but my daughter wants to be one. She totally looks up to scientists. Her love of science began with her taking a science experiments class "Wonders of Science" offered through the community services of our city. She's 10 yrs. old and I'm excited to think she may pursue science. Thanks for the thought-provoking post.

[Emmie]

Great idea for a study!
The first scientist that comes to mind is Abby on NCIS. She literally runs the entire lab, single-handedly, giving the impression that science is *that* easy. Perhaps that's the drawback!

[Kevin]

You're kidding, right? Like the ENTIRE CSI oeuvre is filled with HOT scientists of both genders. And they solve crimes and carry guns and everything!!!!

OK. I can stop channeling a 13-year-old now.

My guess is that these kids aren't allowed to watch the "hot" scientists -- yet.
As for me, Marg Helgenberger still has "it".

[Cocktail Party]

I love watching CSI just for Marg Helgenberger..
http://www.vivamagonline.com/CoverStories_Cynthia.php