NHM+Assets+x+Audience+Engagement+Strategies+(Group+A)

Intersections between Natural History Museum assets and 21st Century Audience Engagement Opportunities


 * Facilitator: Ellen McCallie **
 * Recorder: Rebecca Bray **

Email Rebecca if you want to edit something on the wiki:


 * Rebecca's email address: **** brayr@si.edu **

The 21st Century has already brought with it a broad, deep array of audience engagement opportunities that enhance and/or expand previous strategies and/or offer new possibilities that we are just starting to grasp. There is also the potential for exponential growth in opportunities to connect with our audience that we have not yet imagined.
 * Context **

This conference theme explores the intersection of historical and current natural history museum assets and these new engagement strategies.

As a starting place for discussion, we consider the following unique assets of natural history museums:
 * Natural History Museum Assets **
 * **// Collections, Data, and Evidence. //** Natural history museums contain vast collections of objects and specimens, as well as terabytes of data derived from those collections and their histories, that tell the story of the Earth and its peoples since their origins. Objects, the data surrounding their collection, and the research conducted with them - including new areas of research enabled by technologies such as genomic sequencing - form a unique foundation for public engagement, learning, and participation among informal science education providers.
 * **// "Built-in" Research Communities. //** Many natural history museums have one or more research scientists, collections managers, curators, and other scientific and technical staff that energize education, public, and outreach opportunities with active scientific research.
 * **// Historical Perspectives on Nature and Culture. //** The research conducted at natural history museums and with natural history collections, data, and evidence is a specialized type of science that usually incorporates long term perspectives on nature and culture. This historical view - and the educational opportunities it affords - provide critical insights for understanding current science and cultural challenges, such as climate change and language loss.
 * **// Large-scale Exhibits. //** Many natural history museums provide large-scale exhibits that have provided a basic understanding and inspiration to explore nature and culture for generations. Exhibits and the tradition of exhibiting are important assets to be considered in connecting with audiences in new ways.
 * **// Partnerships. //** Natural history museums are a part of a larger, broader landscape of learning about nature and culture. Perhaps more so than any other type of museum, the content of natural history museums is supported through informal interactions at home and outdoors, as well as by a wealth of additional informal learning institutions and organizations, including nature centers, zoos, aquariums, and camps. Existing and potential partnerships are important assets in advancing informal natural history education.

The [|preconference discussions] on ASTC Connect identified six overlapping audience engagement opportunities that are new or have the potential to transform relationships between natural history museums and their audiences in the 21st Century.
 * 21st Century Audience Engagement Opportunities **
 * **// Programs and Audiences. //** The 21st Century is bringing with the potential for a broad expansion of audiences beyond those traditionally served by natural history museums. New audiences suggest exciting opportunities for new, collaborative programming opportunities. We need to understand our audiences and their needs to develop new approaches to engaging and involving them in the work of natural history museums.
 * **// Rapid Response and Current Science. //** In response to increasing awareness of scientific challenges and increasing expectations for current information, it is important that we identify, develop, and test ways to engage our publics in using natural history collections, content, research, and processes to understand current science events.
 * **// Participatory Experiences. //** 21st Century audiences are so far accustomed to active participants in their own entertainment and education. Exploring the intersections of participatory culture, media use and creation, and natural history collections, content, research, and processes to involve our publics more fully in the life and activity of natural history museums.
 * **// Learning Across Natural History Settings. //** The historical and new informal education providers provide a context within which to identify the strongest niche for natural history museums in lifelong learning. Identifying and synthesizing the strengths of natural history museums as they are unique from and overlap with those of organizations with similar goals and missions is a vital task. We need to consider how existing models apply to building experiences that enhance lifelong learning across natural history settings.
 * **// Technology Integration. //** Scientific and communication technology advances are opening up new avenues for engagement and participation. It is critical that we examine how scientific and communications technologies can enhance learning in natural history museums; the optimal audiences, occasions, and uses for technology; and the limits of technology for learning in and about nature.
 * **// Priority Content. //** Understanding how the assets of natural history museums can contribute in unique ways to understanding highly relevant content for 21st Century audiences, including evolution, climate change, and biological and cultural diversity loss is essential to identifying how best to link our audiences and our resources to address 21st Century scientific challenges.

One way to conceptualize the opportunities to advance our practice is by a grid that creates an area of overlap between and among these assets and opportunities. The working group will consider how the different assets and opportunities overlap in multiple ways to identify potential areas of innovation and the research agenda necessary for us to learn together about how best to bring them about.
 * Linking Assets and Opportunities **


 * || ** Audiences ** || ** Rapid Response/ **

**Current Science** || ** Learning Across Settings ** || ** Participatory Experiences ** || ** Technology Integration ** || ** Priority Content ** ||
 * ** Collections, Data, Evidence ** |||||||||||| Possible area of overlap between and among assets and opportunities ||
 * ** Scientific Community ** ||||||||||||^  ||
 * ** Historical Perspective ** ||||||||||||^  ||
 * ** Exhibits ** ||||||||||||^  ||
 * ** Partnerships ** ||||||||||||^  ||

The space below is an active, real-time record of the deliberations, ideas, recommendations, and concerns of the working group. It is intended to start as a note-taking space and take shape as the ideas take shape. Format is flexible: Paragraphs, questions, bulleted lists, and other strategies are welcome.
 * Working Group Recommendations **

// The working group should provide their own input or ideas on the core opportunities of linking natural history assets with these emerging engagement opportunities and realities in the 21st century. Why is it important to make these links? What is the potential for growth and innovation? What would audience experiences look like if we were to be successful at making these links? What are the consequences of not pursuing new ways to link resources to new engagement opportunities? //
 * Summary of the Opportunity **

Session 1

**Intro from Ellen and questions from group** (Review Session 2, 3 and 4 goals)
 * Question about the 3 approaches to creating a research agenda.
 * Seems like those 3 require really different first steps to start. How important is it to know which approach we’ll take when we start. How important is this framing to the work we’re doing.
 * There are three groups doing the audience strategies – will each take a different approach.
 * Ellen: We have been asked to focus on problem identification. We’re not expected to come up with a research agenda. We’re meant to have the discussion to come up with a research agenda. To identify major issues. This first meeting is to identify major issues. (Review of the grid on the wiki). Our job is to use this as a start and we’re being asked to use our expertise to deal with that. This morning, we are asked to address summary.
 * ** How do we move from nice to necessary? **
 * If you think about everything yesterday, that question is at core. How do leverage collections so you HAVE to go to a museum to learn about. What does the world lose if we don’t exist? What are the innovation spaces that we should attend to as we move forward? What do research, best practices, and evaluation say to us?
 * Proposal to divide into 3 groups of 5 to address different aspects.
 * Could we stay as a large group until we figure out what we’re doing? We need the facilitator to help everyone know what we’re doing. And maybe natural groups we’ll evolve.
 * As soon as it’s boring, stop.
 * The matrix is a reference and tool. Not needed.
 * Session 1 goals.
 * Nice to necessary
 * What would be lost if we didn’t exist
 * Innovation space. Not details but questions they
 * Best practices. Research, evaluation and best practices

**Start**

Let’s start with collections data and evidence. But also think about the other parts of grid. (Everyone read the definitions of the assets and opportunities)

Collections are most important and most unique to NH museums.

Collections, data, and assets are most unique.

Historical perspective next important.

Also the curators. They don’t have them in science centers.

Collections and research are most important.

What difference do collections make to the public?
 * Public trust and scientific process. Not just each ideas but give them an idea of how science works. Collections can be used to help people do science and build trust – don’t take my word for it – do it yourself. When you think about all the kinds of scientific evidence, collections are least abstract, most understandable to public.
 * Scale. So many of the interesting parts of science right now are huge or tiny. What’s cool at Natural history museums is the human-scale stuff. Nanoscale science education network really struggle with educating without facilitation
 * Place-based learning. NatureBridge – also, natural places as collections.
 * K-16 education doesn’t have collections.
 * Oh wow factor.
 * Objects are more universally accessible for various ages and abilities.
 * The ‘real objects’. Coelacanth and giant squid in Ocean Hall. Important engagement factor.
 * The same object can be learned for very different learning experiences.
 * Uniqueness in time and space – this creature will never exist again.
 * Also, audio and other types of collections – bird songs, video
 * Not just place based. Don’t limit by place. Digital important for more access.

Nice to necessary – necessary for what and for whom? Why do we need to know science?

Because we need an informed, voting public to make educated decisions.

Now, the public has problems understanding what a good reference is.

Necessary because relevant to important conversations and decisions. For example, fracking. Rapid Response very important.

When doing evolution education, we had consensus. With hydrofracking, we didn’t. Climate change as well.

Communication – you can have the best collections in the world. But the opportunity – doing communication at a huge scale is important. **To go from nice to necessary, we need to TELL the public we’re necessary.** Public should see scientists on television, at the table everywhere.

**Not just collection, not just research, but the fact we interpret it.**

If we’re going to use research and collections to create participatory experiences.

Do people even know our collections exist and that they’re important? What will it take to leverage collections for public engagement?

Make scientists celebrities to engage in the public space. So people know who scientists are so it’s part of a larger discussion of what’s going on in the world.

Another critical aspect – as we’re thinking about voting public – we need to think about who that voting public is. **If we’re going to be necessary communicators - What about the people who aren’t even coming in our doors?**

**Can we insert proactively ourselves in the critical path of learning?**

Multisensory learning.

How do we advocate science?

Steve Jobs didn’t ask customers what they wanted – just told them. Can we proactively go out and do it.

Place-based. Even if you’re communicating digitally, there is still a place.

What about being more creative technically?

Multiple channels – people consume on multiple channels – in the museum, web, AR, mobile, books, etc. Need to be in all spaces. **More necessary if we infiltrate across mediums.**

Across the public discourse, why aren’t people talking about science more? They would if they hear and see it more it will be necessary. So communication is key. We’ll always have a physical relevance of our things. That’s not going to leave, but to be relevant in a world that is increasingly digital and virtual, we need to communicate.

**Natural history museums need to get over their fear of being wrong.** Fear is a huge obstacle. The process of doing science is full of debate but often show only final.

**We have these unique collections but why do they matter and why is it relevant.** If we don’t ask those questions fully, there’s a disconnect with the public.

Do we leverage our collections well? Would people notice if we disappeared?

What are some examples of collections being used well?

Kirk said – the idea of using the dioramas as time capsules. Look at what we have to make connections with public now.

10 years ago there wasn’t an issue of incredible misinformation about science – Wikipedia – it’s crucial to get the knowledge out there. This is how we find out – through collections.

Our colleagues should go on to Wikipedia to make sure the information is right because not connected to my institution. But this is crucial – this is where the public gets the information. **Forcefully insert our selves into the places where people are getting their knowledge.** To be necessary, go to where people are getting information. YouTube, twitter, TV, etc. Brooklyn Museum does this well, and integrate to onsite.

There is a syndrome of ‘if we could just get people in the door.’ **We don’t look outward enough.**

**It’s about conversations.** Race exhibit did that well.

Gaining an appreciation about a topic tends to lend to more learning.

What’s the problem?

We are looking at trying to use our collections to be more innovative. (David Attenborough doing a programme called Flying Monsters. Could use at the museum but many issues – access to collections, security, policies, time and money, etc. Opportunity to really reach people. But gets mired in policy.

What question could be asked to start to answer?

Why are we doing it? What is the base benefit to the museum and how does it benefit the whole?

Silos – so the various depts. not working together towards the end.

Also, a tension of preserving the objects can run up against the need to be fast and responsive.

Cornell – citizen science observations of birds across country. One database. This is a kind of collection. Then the idea came up to visualize where people can people can see it and use tools to actually map occurrence and part of report to help government - The State of the Birds. A different way to think about collections – leveraged participation, made relevant, place-based, active, and fun. Synthesized a bunch of things.

We have not failed.

ILI does a good job.

Kirk – people are coming in the door more.

We have been defining success based on people coming in the door. So that’s where we are succeeding. Some of those people are learning.

Have research and collections been leveraged to get people in the door?

Blockbusters bring people in the door. Sue.

Why?

And are they learning more?

Why does attendance drop off after the blockbuster?

It can be asset to have small staff. Alaska museum staff can decide and do something. The attendance goes up 20% a year. A lot of collections can be picked up by visitors.

**Any replica should be available for touching.** The realness of the collections is really compelling.

Why don’t we have all **glass walls** and open storage throughout museums?

The most innovative use of collections or best research.

- Travelling exhibits

- The evolution of Maize

- Cornell The State of the Bird. Real data and visualization and tools available to everyone

- A project on 3d visualization – earthquake and plates

- Karaoke – see sound frequencies. Drosophila in Explore Evolution

- Access to CT scanner, DNA and microscopy – see the process on collections

- More broadly connecting to the research that’s happening from collections

- See scientist doing the work

- Denver – participate in the scientist

- Open up boxes from the basement

- Trusting and respecting the audience

-

// This section will grow during and after the conference. The tagging exercise at www.informalscience.org has identified r //[|//esearch and evaluation reports relevant to learning in natural history settings//]// that could be brought to bear as foundational research and best practices. The working group is encouraged to add their own knowledge of applicable research and evaluation, as well as best practices (projects, programs, museums, etc.) at the intersections of natural history assets and opportunities. //
 * Foundational Research and Best Practices **


 * Session 2 (Monday afternoon) **

- Great idea to pair natural history museums and nature centers

- We are in the business of purveyors of wonder

- We underestimate the fact that most people live in urban areas. And our museums often are. There is urban nature. Nature vs. humans. It's a changed, domesticated nature, particularly in Europe. Manage those conversations. Not just about wild places. Living in the middle of an urban place, many visitors don't see stars. Can take something small and talk about nature, not something huge or completely wild.

- What even is the wilderness? Everywhere you go there are trails and signs. It's hard to find true wilderness. So start small. We have backyards, parks. Try to find the smaller natural places.

- There are contemporary places, small state parks near or in most urban areas that are accessible. We need to help people find those local places. Tell people - to see more of this - go to this park...

- It is important that we think about the larger community.

- Where we are as a group: We are getting where we need to be. The voices of people outside of natural history museums are important.

- Bill tasked the group with dig into 'What if I wonder'. Think about why we'll involved in natural history. Why are we here? The core passion for each of us. The core dream of 'what if'. Build on conversation from this morning. How do we leverage the core assets? What if we really did let all our visitors touch all of our stuff.

(Break into smaller groups)

**Small group notes from Rebecca:**

Core passions:

- people learning about science so they understand the world you live in

- what does it really mean to understand

- what if natural history museums were able to really extend into people's backyards. destination is your own backyard. if we're trying to saving the world (which is coming up a lot!). Awareness needs to be injected into people's daily lives. To inform people about interacting with their own environments.

- Core passion - connecting people to the natural world using science.

- Notion of transference. We can have them for an hour in the museum but I'm most concerned with what happens when they leave. What can people do in their own lives to connect people, have them transfer to their own world.

- Not everyone can visit our institutions. We have to overcome being one place. Get beyond being a place. How do we transfer the place of learning, the lessons of our collections to transcend a physical place. But do help learners understand a sense of place.

- To change people's perspective on what their environment is. Shake up people's perspective. And to extend the relationship beyond the museum. Have an agenda at London to get people involved in nature.

- Natural history museums as too much about science. You have to tell the story from the head and heart. But it is about science. Data viz - there are different ways to get people to shift the way they see things.

- Personally, like to look at things, see relationships, behavior, .... fascinating.

- A very popular thing in London is DNA through looking at malaria, mosquito.... kids love to see the process, seeing people work out how to get from A to B is really great.

- What does it look like to expand beyond the museum

- partnerships. What are networks and partnerships and why different?

- people have less money, less proximity to natural spaces, less interest in natural history. We need to be strategic about a pipeline, linking content across organizations.

- need to get people who aren't already converted. BioBlitzes in parks are great but those people are already doing it.

- need to get people in urban areas involved. How do we introduce people.

- Maybe the recession offers opportunities. Cleveland has the largest urban farm in the country. Urban homesteading. Farm squatting. Renature some cities. We could play a role. What used to be here?

- BioCube - applicable to urban spaces as well as wild spaces. Soil studies. Kids contribute. Small scale - small enough to make a contribution but the many parts of it means that as a whole it gives really broad perspective. That's where partnerships are crucial. If you do something similar across many institutions.

- Collecting data locally and then sharing it on much bigger scale.

- Bring in cultural value. A cultural history. How to get in populations from all over the world in an urban space. Bringing cultural relevance is important. Bring in native understanding. Learn a lot from our audiences. Bring culture into museum. The human perspective. Refugees from local area. Making it real, involve the emotions. In the way that television tells good human-sized stories.

- How do we make a conversation, not just one way and didactic.

- Denver does a good job to talk about how people have used land and place.

- Bringing in locals to teach. Live people! Native, and other locals.

- Different perspectives on nature from various communities.

- We can learn a lot from these communities. Real dialogue. Scientists really interacting with the communities on their fieldwork.

- We haven't talked about exhibitions as a possible way to really engage.

**Small group notes from Anna:**

Multiple contexts of research on objects in the collection - science around the collections is dynamic, current, evolving. Transparency/sharing of research currently being done on the collection.

· Balanced approach to process of science, and the consensus knowledge that comes from science. Just because we're still asking questions doesn't mean we don't (generally) know what's going on.

· What if every collection were put behind glass walls - public perception of what the museum is about

· Rather that saying we are for diversity, we actually did it. Becoming more diverse in our thinking and staffing to reflect the communities we serve. Saying we want to diversify but not changing what we're doing. Staff has to reflect the diversity you want to engage. Image of the staff reflecting the groups we want to engage.

· Really engaging people beyond our walls. Go to audiences (real or virtual), also bringing scientists, and collections to the public (no collections in educational area).

· Technological innovation - museum experience is part of the continuum - keeps the long tail going. Put the museum into the community discussion.

· Being the best at what we do - museum is fundamentally part of the learning experiences, but doesn't have to be in every mode of informal learning. Each institution doesn’t have to be everything to everyone and do it on their own.

· Alliance of Natural History Museums of Canada - network of NHM. Connecting so we're not competing but we're collaborating. Taking leadership in areas but not competing with each other. Different institutions in different places - working in tandem but "finding your mastodon"

· NH is an object-based science - how do you deal with the fact that not everyone can share everything with their audience? How do we create the opportunities for collaboration without all of these problems getting in the way?

· How do you get over the siloing? Leadership training. Working your way up vs. getting right into an administrative track.

· Not just opening our doors to audiences, but going to knock on theirs.

**Small group notes from Carlin:**

What if…

Questions mulling – “what research has changed the way you think about things?”

What if…

1. - all natural history museums acted as facilitators?

Learning researchers; facilitator is not in the role of the expert; doesn’t have knowledge in head to unpack for the listener; facilitator brings people together

One of the things that is really important is to draw out – one of the things that is missing 0 enough content knowledge to be a good facilitator. Facilitators need to know about the discussion.

Draw someone attention Strength of NHM as educators, scientists, etc. can work as pairs with people-guiding to get there – to provoke meaning-making opportunities for meaning making.

Avoid “meandering discussion

Pedagogical content knowledge

2. What if…

- We frame the experience to a mutual learning experience between researcher and public, with notion that they might think differently about their work based on public response.

- McCallie et al 2009 paper: //Many Experts, Many Audiences// at caise.insci.org/resources

- Reframing – we have something to learn from our audiences; they have lived experiences that can inform the work we do – at least the pedagogical knowledge we impart, and how we impart that knowledge.

Aren’t we already facilitators?

Yes – that’s what exhibits are, designing programs etc. to get people to think

0 Difference is – public engagement facilitator is engage in a conversation, actively listen to publics;

I wonder what would happen if:

We put sticky notes by every exhibits - visitors response

3. What would happen if…

Visiting David in Florence – admire that; Stanford project scanning that; in that display you don’t see how you merge the virtual and real

What if there were a digital way to explore every specimen? How do we integrate the meaning of the real and digital?

Carbon Nanotubes; phantom pen – “feel” a virus; take objects haptic technology force feedback. Interact with these objects in ways we couldn’t otherwise. Whether paintings, objects, etc.

4. What if….

Our goal was to empower folks to have new eyes with which to see the world. May be a way to Message our Mission: to empower people to see the world in a different way?

5. What happens if…

- we have galleries that have doors that are heavy and close. confined space; kids can’t get out, multiple age kids, parents sit down, feel they and their kids are safe.

- small, private spaces.

- Why museums aren’t a network? Appalachian Mountain Club – long tail, all connected experience; different places. Would instill a sense of pride, link ourselves; create network…

6. What would happen if – we shared some of our exhibitions? Your interpretation of an event that is happening at different venue?

Problem with _not created here”; what would happen if we did leverage each other’s resources on current events?

Nano project – exercise in game theory; changed the behavior between institutions - grant that was large enough to include a collaboration to

CoI’s? things are being more collaborated? fighting against a history of subawards; etc.

- What if all natural history museums acted as facilitators, not just 'expert'. Facilitator brings people together in non-threatening environment.
 * Big "I wonder" items from smaller groups: **

- What if rather than just saying we're for diversity. What if our staff reflected the audience we want to engage. What if we knocked on people's doors.

- What if we could get beyond the idea that you are a place. Get outside our doors to everyone's backyard. Transference to people in their own lives.

- What if we could collaborate and not feel like we have to compete. Have something special we do but also work together.

- What if we leverage your resources. Pipeline. Scaffolding experience. Make sure to extend experience.

- Mi mastodon es si mastodon.

- What happens if we have galleries that have heavy and closed. Confined spaces that kids can just wander where parents feel safe. In practice, people really are coming to spaces like this. Makes it a safe social space.

- We had talked about all these big ideas, but none of it had involved exhibits.

- What would happen if we shared some of our exhibitions.

- Are we competing with each other? For funding. But some say - not a feeling of being competitive. More that people do things in isolation. More like people are reinventing the wheel not learning from what others have done. There's a hubris of people not wanting to collaborate - want to do something new. Does Exploratorium feel competitive with CAS? No - feels like they build off each other. Good for others if other recent museums are doing well. Exploratorium numbers higher since CAS re-opened. Other examples of this.

- Carnegie - trending down - average 300K. Actively looking to change to trend up.

- Funding can result in competition

- What if we could have a digital way to explore every specimen?

- 3D printing. Could print out dino bones on $1000 makerbot in classrooms all over the country? Then people can touch them.

- Accessibility

- iDigBio project - putting out in public domain digital images and 3d models. As many images/3d objects of natural history specimens as possible. Director of Ed and Outreach on NSF project - iDigBio. Can compare. Can scan your own object and compare to objects on the site. They will have object/visual recognition and match to objects they have.

- LeafSnap

- Merlin app will do it with birds

- There is a convergence of tools

- Museum collections will be accessible in the cloud

**What are critical questions for research and evaluation?**

- Josh: Some research that changed his perspective. Carol Dweck - studied kids models of intelligence. What do you think it means to be smart? Most had static view - you're smart or you're not. The researcher promoting a dynamic model of intelligence. A muscle you build. The research was amazing. Also devised interventions to get the kids to shift from static to dynamic models. Kids with static models would give up faster. Kids with dynamic model both more persistence and self-efficacy. Why am I here? Empowering people to think for themselves. Give people new eyes to see the world. Often we talk about transfer - we have knowledge to give to people. Not empowering, smacks of static model. Instead - dynamic model we're building all the time, help people build skills.

Critical question. We value process of knowledge in our science staff. What would it take to make process a critical part of our audience experience?

Appreciation for process, no absolute answer. How to balance lack of one answer, discussion, arguments while still having authority.

How do get our audience to think in terms of dynamic learning? We have to treat them that way, and model that ourselves. Show our own staff building knowledge, changing.

Articulate questions about technology pushing the boundaries. The way that Ailse is pushing people in London... use technology to embed ourselves in what people are doing in their everyday lives. For example, the fact that kids are using media so many hours a day. There’s a suspension of belief when you get into a museum, especially under 10,11. After that, they think the stuff is dead. Trying to bring things to life again. Also, extend beyond the museum, so the way that use tech at home is something we're embedded in. Instead of division between physical and digital.

Digital strategies - blogs. Naturalist talking about what's happening on campus all seasons. Also from living collections. The story of the museum itself. Not just the scientific expertise. People are curious about what you're doing behind the scenes. Building a new exhibit, etc. Staff can give voice, and not just the communication, education staff. These blogs are really popular. Never edit what those staff say. Audience needs to know its authentic. Needed to convince leadership that this trust was crucial. Scares people but it works.

What happens if you really trust your staff?

What happens if you actually showed the people, not just scientists.

Really changes how this staff thinks about themselves.

Other institutions have done this - voices from all over the museum. Building a hoist for a whale - engineering, etc. - really popular.

What other ways based on trust of staff?

What if the collection people welcomed people into collections?

You can see people doing science.

Darwin Centre has fishbowl. Some problems. Some scientists really don't want to go in.

But blogs, identification forums are working. The virtual environment where scientists have conversation with community is really working. Video diaries from scientists are very successful - much more than physical fishbowl.

What if - to be a museum employee - you had to work in a public space doing your work? What if part of your work was sharing your work?

- London - tried this - tried to change contracts. The scientists said NO.

- But some museums have done it.

- John Gerche - made bronze casts - made casts in Ithaca in public. People loved it. He hated it at first but liked it at the end.

- Exploratorium - exhibit developers are on the floor. They wear headphones to block out noise.

- Steve: Engage, inspire, and empower (Monterey Bay Aquarium). Also can talk about knowledge, understanding. Cognition, Affect, and Sensory-Motor handbooks. Bloom's Taxonomy. We live in false dichotomy between cognition and affect - cannot separate. What is the real relationship between engagement and understanding? How does it link to the golden ring of behavior change? And, we're not doing this TO people - people are choosing. Need a better understanding of who our audiences are.

- Behavior change, concept change - long-term. How do we approach long-term?

- Actually holy grail now is not behavior change but identity change. Draft of recent study 'Framework for ....". Identities - you see yourself as a technician, science literate, or XXX. In what ways do we engage with people such that they are comfortable identifying. Anything that you cannot touch you need to have a virtual version of. What does making everything accessible do?

- But there's something quite different between real and virtual object. A virtual object is mediated. Leaves people cold in a way a real object doesn't.

- But not only virtual objects.

- What if you had a real object next to perfect replica. Ask people what it means to them? How much value do people place on the replica they can touch? No value in replica?

- Much science is actually practiced through computers - science is already mediating experience of itself. We can't get at current science without the virtual.

- Replica vs. real, physical vs. virtual. Depends on the learning context, outcome varies. There's value in touching and manipulating while also value in the real thing while also value in the virtual object.

- Are museums putting away from putting out real things vs. casts? There's much more of a wow factor of seeing the real thing.

- Can have both. Have the real object next to cast.

(Why Evolution? Because no one else can do it. We have the greatest potential to educate about evolution.)
 * For our next meeting tomorrow: **
 * - We'll divide into 3 groups and do the 'What if" and "I wonder" about: **
 * - If the goal is to make a difference in evolution literacy, how will we leverage our assets? What is the 'what if' and 'I wonder'? **

(Why? NSF is interested. Public is interested).
 * - Rapid response and current science. What happens if we leverage our assets to respond to current issues? If the goal is to really make a difference in people understanding science, what do you want to know, what do you dream about? **

**What are you most excited about that came out of today?**

- What Scott said - We are purveyors of wonder. Proust quote. Those sound bites can help shift the paradigm.

- Communicating breadth of what happens at museum - not just science

- Reaching broader and more diverse audiences. Shouldn't be talking to the same audiences in 10 years. The diversity of learning of school groups is changing. Need to look at who we are trying to serve.

- We are stronger together. we can be a pipeline, can scaffold learning, and work together.

- Museums as one stream of information. Try to build parallels to other information industries like newspapers. They struggle to retain relevancy. We're similar but we have things others can't replicate, and can't replicate online. So that all is hopeful

- Excited about getting people involved behind the scenes.

- Museum is really a transformative place. That's why people go to museums.

- There are so many artificial barriers that we can get rid of.

- So much energy and passion.

Are there other venues where natural history museums collaborate? Let's talk about this next time.

// The working group will expand on its identification of the potential for growth and innovation and what audience experiences "look like" as explored in the Summary of the Opportunity. Here it will be useful to explore two different avenues of opportunity: 1) Ideas for innovation that are or are not "shovel ready.” The less formed and more grandiose ideas are those that are likely to help identify critical questions in the next section; 2) Hypotheses about what the group thinks might be an effective innovation, for whom, under what circumstances, and so on. //
 * Opportunities for Innovation **

(Review of what will happen in the afternoon and choice of sessions)
 * Session 3 (Tuesday morning) **

This morning, we will do the following:
 * Introductions again
 * Start with 'What if' and 'I wonder about' in regards to Evolution. We've been talking about real objects and collections and how do we leverage them(for evolution ed). What do I want to change? And, Technology as Tools towards evolution literacy. We will have 2 tables of evolution and 1 of climate change.
 * After that we'll talk about research and the strands of research. How do we apply these strands to what we've been discussing.

**Why Evolution? Because that's what we can do better than anybody else.**
 * What if all collections went aw
 * ay? There would be no evidence of life in deep time in the fossil record. Our records help understand time and evolution.
 * What does evolution mean to people on an everyday basis? How is the evidence important? Why is hard communicating that evidence.
 * Mohammed Nor, biology: for those who doubt evolution, I offer them last year's flu vaccine.
 * You can't talk about the future without talking about the past
 * How can communicate the importance of evolution
 * DeepTree and FloTree **
 * NSF funded conference on understanding evolution and the Tree of Life.
 * 40 museums got together over years to imagine this, in particular in exhibits
 * One outcome was a funded 3 year project
 * Year 1 is done. This is prototype
 * sdr.seas.harvard.edu/life-on-earth
 * The data from the Tree of Life visualization from toll.org. 90,000 species. EOL for common names and 200 species that are evolutionarily important and that visitors are familiar with.
 * One challenge is that scientists use the tree as a tool to depict evolutionary relationships. It's a very active area of research. Trying to reach consensus. One outcome is the tree of life web (tolw.org?).
 * Now journalists are using trees more.
 * But there's a gap between the ways that scientists are using trees and the way the public understand trees and the relationships that underly
 * (demo)
 * Can explore relationships and can explore related traits visually. Can compare any two species.
 * Can explore relationships and can explore related traits visually. Can compare any two species.

**Group discussing Evolution**

One thing we struggle with in citizen science projects is keeping taxonomy recent

Question: when we're talking about getting evolution across to the visitor. Are people self-selecting? People who already believe.

Research on Partners in Evolution. One outcome - not such a clear division between believers and non. Most people have seemingly contradictory ideas. Even the most fundamentalist visitors are rarely purely creationist. Vary tremendously in terms of how informed they are. Much more a chimera of ideas.

Move people along a continuum towards a better understanding.

Example from London Natural History. Limited disply in museum - no permanent offering. Attenborough theatre - wanted film for 11-15yo. 'Who do you think you really are?' How much DNA do you share with a banana? Twin screen experience. Start with family history and look back. Look at morphology and what other species you have... People can do things on their handhelds. Visitors make choices. Results are shown on the screen. Very effective for learning. AT that age, kids want personalized experience. Asking questions about identity. And the twin screen so you have both narrative and a personalization aspect. This is layered information and its amazing how much information people can take in.

What if we have a virtual display like this in every exhibit?

How do we show impacts on visitors for these sorts of things?

NIH is demanding a version of randomized control trials.

If we were going to compare these exhibits to something else... what is a legitimate comparison group? Do you need to compare to like programs. Debate in the field -- comparing.

If you don't have a comparison group, you are able to understand what is unique. Steve - can just compare to other onsite experiences.

What aspect of the experience are you trying to evaluate? How self-guided? Is it the interactivity? What's the question? Complicated variables. Do we need an interactive table. Text vs. visual. You could present the material in a more traditional approach - non-interactive video, for example.

The breakthrough in London was the personalization. You can go online and find your story. The technology and gamification aspects are great wow factors. All of those tie together.

In Alaska museum, soften evolution message. They generally don't say 'evolution' - they say 'change over time'. There are often chaperones who take kids out during evolution part of the workshop. How to push those boundaries.

Similar in New Mexico. Sometimes parents will reinterpret for kids and tell them it's not true. Maybe it's about smaller scale and understanding those examples. Population change over seasons based on food supply. From there people can conceptualize short to long term change. Real-world examples. Show how to scale up from years to deep time.

I wonder if we can make evolution relevant not by focusing on evolution as a big concept but by focusing on how it is affecting you daily. If people understand or relate to micro-evolutionary process does it make them easier to understand or relate to large scale change.

Bruce: if you look at Gallup surveys in the last decades there's been no shift in the US in public perception. John Miller article - 47th in terms of public acceptance of evolution. A little cynical. These things seem incremental.

Focus the audience. Kids - their minds are still open.

3 types of people. 1/3 like science, 1/3 could be interested, 1/3 will not happen.

Need to focus on a target audience. Not just museums - need to reach people who don't come here. Need TV, internet.

Need to strategically think big about how we'll do it.

Think about it on continuum. In Alaska, little girl said her family believes in the Bible but when she sees the dino bones she doesn't know where it fits. How to talk to this person. Maybe the goal is to think about just exposing people to the idea as a starting point. Dinosaurs are a great start. Dinosaur Train.

What if every gallery had something about evolution, especially in the Dino halls. Maybe see how long ago, and connect to current, relevant info. What if every gallery speaks to the fact that things change over time.

Judy - when they first started Explore Evolution galleries, and deeply fundamentalist population, big debate. Decided its not about belief, it's about understanding. Decide to make the term Evolution very visible. Worked with various people to figure out how to talk about. Now fundamentalists visit more. They bring high school kids to the exhibit every year, telling them to understand evolution in order to make up your own mind. As museums, by being proactive and direct and not afraid to teach good science, it turns out that it opens communication in ways not predicted. Expected to be hated. But by being honest, it actually brought people in.

Honesty - let others politicize it.

Steve - Life Changes, Evolution, and Health both at NY Hall of Science. Evaluation - understanding, attitude, belief. Changing belief takes more than 10 minutes. Provide opportunity to get people engaged. Not a fan of the continuum - very linear. Prefer more about cognitive dissonance - how does that work? Kids looking at dinos and birds. The kids don't have to get it exactly right to have a good outcome. Especially with kids, understanding is more a cloud than a set of facts. If the kid adjusts attitude - sees that there is a different viewpoint that might make sense, that's good.

Chia - How do we evaluate. Media Lab - measuring how people play video games. People have to be interested. Web cam to take your picture to put in the interactive (but no, because kids would just take their own picture). We're terrible at measuring affect.

What if we measured success by

Kids starting experiences by doing experiments by trying to reach outcome. Doing it outside of a science class might make a difference.

A lot of museums and schools are starting to have their own local gardens. I imagine there might be a way to use gardens to look at how a bean might change.

A museum in Korea has genetics labs where kids can see up close. Kids really engaged in understanding. Schools and museums doing molecular biology. Dogs. I'm making this myself, I own it. Not only from the big, abstract picture. Birds - the fact that birds are threatened touches people. What if the museums were focal points but need to inject the message into every day life, not just at the museum.

It's hard to jump from micro to macro level.

What does the bigger concept get us?

Evolution and Health exhibit - is good at relating to us. Pelvis example - childbirth. Lactose intolerance.

Should focus on the things that produce tangible outcomes. Calibrate those to the kinds of experiences that people are having. Optimize learning.

Imagine you're defending natural history museums to Congress. They'll want more than just attitude change - of people like this, appreciate nature more, etc. - they will want evidence of real learning.

Debate on evaluation - cognitive

Big question - Is it more effective to start with the affective interest motivation to end up with cognitive evolution literacy. Or is more effective to understand...

It would cool to show that our exhibits increase interest and appreciation AND made a big difference later on - that would be great.

Where is the aha moment? Do you show them or get them to find it themselves? Can't separate affect and cognition.

Sid the Science Kid. Young kids learn hypothesis, experiments. There are ways to get in with kids. Start small. Encouragement along the way.

(back to big group)

The main goal for this project is to have an outcome for a research agenda.

Based on Ellen's notes (now included below). Reflect back on - how do you think about developing research agenda? What do the learning researchers and evaluators think about?

Josh: Where do you start? Maybe - am I interested essentially in audience? Or a design perspective? A good research question - How would the audience experience change if every collection object were accompanied by a virtual interactive object. Or, what do people think about change and how does that relate to evolution. How do I get people to engage in the process of inquiry? What does the reasoning look like?

Books - good accessible entry: with Sue Allen "Group Inquiry in Science Museums: Asking Juicy Questions" inquiry game that visitors can 'play' and engage in deeper inquiry. "Fostering Active Prolonged Engagement" about creating exhibits.

When we think about a learning research agenda, want to be pragmatic about where to start. Audience based? Design based? Conceptual?

Anna: Some of our research agenda is opportunistic. People doing new innovative things and so as we're doing it we layer on research question. Beyond whether this was successful to a research question that is relevant to broader field. What's the good work we want to do and then how do we learn about that?

Josh: Research and evaluation are different from each other. Goal of evaluation is - how do we make this the best it can or did we? Research - generalizing - not just one thing you're studying. This is a class of things I'm studying that can be applied across.

Steve - Research is messier and more complicated. may end up in a different place than where planned. Often start with audience. Based on this experience, what do you hope visitors will think, feel, and do? Then it's about chipping away. Don't try to do everything. Need to edit.

Judy: Funding agencies not accepting delineation. Want evaluation to speak to larger things. May get rid of summative evaluation. Make evaluation more like research.

One last thing from each person:

- Big barrier is our discomfort with not having the right answer right now. Need to be okay with ambiguity. Okay to put stuff in front of visitors without knowing for sure in order to keep up with rate of change.

- This has been very helpful to take back to my institution

- I would feel more comfortable if there was accountability at the end. Did we actually form the partnerships we talked about? What happens in 6 months?

- Will there be a contact list distributed? Everyone should a picture next to their name.

- We're all getting similar issues coming up. Small or big organizations have similar questions. Network is important.

- Having a group of people focusing on issues specific to Natural History is great. Feeling like there's a community refreshing

- What's the point of literacy in evolution? What goal does that get us to? Not clear. Just because we're well suited to teach it doesn't mean we should.

- If something is going to change, this is the group to start it. This is a good place to start. There are other people not here who should contribute.

- How do we build this?

- Put a scientist's picture next to each thing. And the scientist says, 'here's what I wonder about'. Science isn't done.

- Let's all try something uncomfortable that pushes us

- Let's try something that someone else is doing.

// This section will identify critical questions about audience experiences and learning. This section is the crux of the work of the group because it is intended to identify a broad range of critical practical questions that need to be addressed to move forward with identified potential innovations, as well as critical theoretical questions related to what, how, when, and why people learn in and around natural history museums and assets. The ultimate product should be a set of prioritized, nested questions organized in a way that makes sense to the group. The questions should address the critical challenges identified through preconference and first day activities. //
 * Critical Questions and Directions for Research and Evaluation **

Group A: Public Engagement Opportunities Summarized by Ellen McCallie (314-489-6948 for questions)
 * Explicit and Implicit Research Questions **

Framework (meta-level)
 * Connecting people to nature/the natural world, using science.
 * Learning questions about the role of collections
 * Learning questions that (also) substantiate the value of museum and collection

Collections & Participatory Experiences >
 * 1) What would it take to have visitors value NHM primarily for our collections? Nature of Science/Nature of Museums type question? To what extent do experiences provide opportunities for visitors (and learning outcomes) that support the mission of the museum—that collections are our most important asset. Do publics get our public value?
 * 2) What is the role of the “real:” real scientist, real collection, etc. What is the advantage/value of having the “real?”
 * 3) What are the differences in using new media to either approximate or take it deeper or further than real audiences can? What is the role of new media?
 * 4) How does “embodiment” personalize and impact the learning experience with the collection?
 * 5) How do visitors best learn about and/or connect with NHM collections (affective)?
 * 6) Under what conditions are casts, models, etc. not acceptable/undermine the NHM learning experience?
 * 7) Under what conditions and in what ways, do casts, models, etc, enhance/support learning?
 * 8) What happens when visualizations, casts, models etc. are used in conjunction with real objects?
 * 9) How would the visitor experience change (all LSIE strands) if every real object was accompanied by a touchable, manipulatable, or visualizable replica/representation?
 * 10) How would the visitor experience change if EVERY non-real object could be touched/handled in a museum/gallery? How would visitors’ experiences change? How would visitor perception of, experience of, and connection to NHM change?
 * 11) What are the range of ways visitors are invited to examine collections as primary data related to scientific questions, current issues, controversial issues? Which ways are most effective in prompting change in the ways visitors view the world? View evolution? View science? View the value of NHM?
 * 12) In what ways do digital files, such as bird vocalizations, count as collection? Serve as collections? What are their affordances in conjunction with other collections? What are their limitations?
 * 13) What should we consider as we leverage collections for audiences of different ages? What are the developmental milestones we should take into account when designing interactive experiences?
 * 14) How would the perception of NHM change/Would NHM be seen as more vital places or ones with active research…if “mi mastodon, su mastodon”—if we shared in each other’s discoveries? How would that change people’s feelings or connections to NHM?
 * 15) How do audiences change with increased participatory experiences? Which types of participatory experiences (or how different/participatory) does a NHM need to be before other audiences sees NHM as “for me?”
 * 16) What are effective means for communicating “NHM are for you” for visitors? If we’re changing our experience, what are effective ways to communicate who we are, who is welcome, and how we are changing (what to expect)?
 * 17) What are best practices with respect to enlivening, reawakening, or leveraging old dioramas for understanding the natural present and natural future? What are ways to effectively use dioramas as data sources for examining change over time, place-based issues?
 * 18) There is a lot of controversy about the effectiveness of dioramas for learning. In what ways do dioramas support the LSIE six strands of learning? What kind of interpretation, facilitation, etc. increases their impact?
 * 19) What do we really need to learn from the fact that our attendance goes way up with blockbusters and then returns to “normal?” What would it take to change the curve?
 * 20) Glass walls, open storage, and the ability to view staff working are often cited as potential game changers for NHM. What do we know about small and large scale experiments with making NHM collections and work visible? What makes these efforts successful or end up failing? What impact do they have on visitors? What should factors or best practices should NHM consider when embarking on “making science and collections visible?”
 * 21) What are innovative and best practices with respect to making collections visible and maintaining collections in perpetuity?
 * 22) How do exhibits and the perceptions of NHM //writ larg// change when how we use tools such as CT scans, DNA sequencing, SEM, etc. are integral parts of our communication, display of collections, and practice? What changes when we talk explicitly about our partnerships with community professionals who use these tools to help us examine specimens and propose new understandings?
 * 23) In what ways can we tie passion into participatory experiences, particularly with collections? How can exhibits, programs, and virtual experiences be designed to integrally draw on, elicit, and develop people’s passions? What about beauty/aesthetics?
 * 24) Does providing various context around an object effectively provide for multiple entry points into discussion about the object for different groups or different ages?
 * 25) What is the effect on learning when objects are put in their current research context of what we know about them and what we want to know?
 * 26) What would happen if NHM shared their best interpretation/exhibits/classes with each other—at no/low cost? How would this effect the perceived value of NHM? What would happen within the museum?
 * 27) How would our experience with audiences change if every exhibit had sticky notes by it?

Data
 * 1) How do audiences access NHM data?
 * 2) What data visualization tools are most effective for NHM audiences to understand data and patterns?
 * 3) What data visualization tools allow NHM audiences to explore data and ask questions that they are interested in?
 * 4) What data is available to NHM audiences that relate directly to their local environments, homes, and landscapes?
 * 5) What programs/platforms allow audiences to learn about nature outside the museum with just-in-time technology? What conditions prompt people use the technology?
 * 6) Data access and visualization seem to be the most promising means for achieving connections to content, beauty, and meaning of science/nature data. How can we leverage the LSIE strands to understand the breadth of impact of data visualization? What happens in terms of learning when experiences shift the way people see things?

Scientific process > How do get our audience to think in terms of dynamic learning? We have to treat them that way, and model that ourselves. Show our own staff building knowledge, changing.
 * 1) What ways are most effective in promoting visitor’s view of NHM as places where knowledge is generated and knowledge is in process?
 * 2) How do NHM effectively communicate the process of science?
 * 3) How do NHM effectively communicate the tentative nature of science without bringing science as an endeavor into question?
 * 4) How do public views of NHM change over the short and long term when NHM shifts its identity from having the answers to being a place of discovery, iteration, etc. (“process” as opposed to a place for “final answers”)?
 * 5) If we value process of knowledge in our science staff, what would it take to make process a critical part of our audience experience? Specifically, appreciation for process, no absolute answer. How do we balance lack of one answer, discussion, arguments while still having authority?
 * 1) People seem intrigued by stories and processes. Can scientific stories and processes create exceptional learning contexts? (Mosquito DNA, malaria…)

Technology Integration & ** Learning Across Settings (Ubiquity) ** > What happens if you really trust your staff? What happens if you actually showed the people, not just scientists?
 * 1) ** How can we insert proactively ourselves in the critical path of learning? **
 * 2) What are effective strategies and practices for inserting NHM content and data into daily lives, activities, and technology use of people?
 * 3) How can we “go to” where people already are digitally? How can we integrate ourselves into the platforms visitors already use? How does/can this make NHM an extension of the visitor or visitor identity?
 * 4) How can we support people’s use of multiple channels simultaneously: phone, video, internet, etc.? (=normal for digital natives). In what instances do we NOT want to support multiple, simultaneous channels?
 * 5) How does the credibility or perception of NHM change if we actively contribute to and edit things like Wikipedia? How does our identity change as we coordinate across NHM to edit/contribute to public knowledge places, like Wikipedia?
 * 6) How are NHM are ** forcefully inserting our selves into the places where people are getting their knowledge? **(To be necessary, go to where people are getting information. Youtube, twitter, tv, etc. Brooklyn Museum does this well, and integrate to onsite.) What changes when this happens? How long does it take to measure a change in attitude about or understanding of what NHM do or our role in society?
 * 7) How can we measure transference? How do people leverage knowledge and experiences from one nature experience to others? How can we work together to support transference within a area/region?
 * 8) What factors support seamless integration and visitor experience across NHM platforms (variety of virtual and physical)? What supports help visitors leverage these factors to integrate NHM platforms into their “normal” lives?
 * 9) Technology as tools: How does telling the story of NHM research through the lens of technology as tools support the LSIE practice/skills strand as well as the reflection/nature of science strand? Do NHM have a unique or strong role here? (It also should help with our relevance.)
 * 10) How do we transcend “place-based?” Not everyone can visit our institutions. How do we transfer the place of learning and the lesions of our collections beyond the physical space?
 * 11) Digital strategies - blogs. Naturalist talking about what's happening on campus all seasons. The story of the museum itself. Not just the scientific expertise. People are curious about what you're doing behind the scenes. Building a new exhibit, etc. Staff can give voice, and not just the communication, education staff. These blogs are really popular. Never edit what those staff say. Audience needs to know its authentic. Needed to convince leadership that this trust was crucial. Scares people but it works.

Rapid Response/Current Science
 * 1) Organizational question: what will it take for NHM to actually respond quickly?
 * 2) What supports would effectively allow NHM to leverage, distribute, or display real-time or new interpretations on current events that were developed by other NHM?
 * 3) How do visitors respond to the idea that current event exhibits in the museum they’re visiting were produced by other NHM?
 * 4) In what ways is place-based interpretation reinforced/undermined by including rapid response about current events in other areas? What difference does branding/connection to other place-based museum make? In what situations does branding/connecting rapid response exhibits to other place-based museums reinforce the place-based notion/identity of a museum?
 * 5) Missed opportunities—we often miss opportunities to relate our knowledge to current/relevant issues in society because we are not structured to respond quickly, we want to have “the right answer,” etc. What would a NHM that could respond look like? How would it function? What would it’s role in society be? How would the identity of NHM employees and volunteers change with these changes? What other changes (communicate we’re a process organization, decrease our fear of being wrong) would need to happen for a NHM to embrace this role?
 * 6) How would our relevance change if we shared our interpretation on current issues—or distributed leadership with respect to our responses? Or if we just put the web up as a response/interpreter of current events in our galleries?

Scientists
 * 1) What are effective strategies for making scientists local celebrities? What are the factors/contexts that make this status more likely? That maintain this?

Audiences >
 * 1) How can we get new audiences through our doors?
 * 2) What counts as “wonder” in NHM, if we are in the business of being “purveyors of wonder?”
 * 3) What does “trusting our audiences” look like? What could it look like? What would be different if we did?
 * 4) What are the various strategies for developing and maintaining repeat/frequent visitation?
 * 5) What are the characteristics of experiences or “what happens” to visitors (emotions, experiences, etc.) in places that they are drawn to for repeat visitation? How do NHM experiences map? What aren’t we doing, including approaches, experiences, or attitude shifts, that are low hanging fruit, crucial for our future with our communities, or something we should at least ponder if we want to move from nice to necessary?
 * 6) What role does conversation play in each strand of learning (LSIE)? In what ways does our “attractiveness” for diverse audiences change when conversations are promoted or scaffolded throughout the NHM experience (on all platforms)?
 * 7) How much change and on what platforms does change need to happen (across all, only on web, etc) for repeat visitation to take root? For people to “talk” about NHM as part of their daily lives?
 * 8) How can co-curating or community curating be sustained in NHM? What organizational committee is needed? What factors affect the success (and initial willingness to engage) in community co/led curating? What does community input or led content/experience mean for the identity of NHM and our “position” within communities and with respect to science?
 * 9) How important is relevance, particularly cultural relevance and explicitly addressing cultural connections to facilitating learning?
 * 10) What would happen if we “enacted diversity” instead of just talking about it, so we actually reflected our communities?
 * 11) What would happen to our relationships with publics (and the science we do), if we reframed the relationship with audiences such that NHM were to learn something, too? If we thought our science should change based on interactions with audiences (mutual learning)? McCallie et al 2009 paper: //Many Experts, Many Audiences// at caise.insci.org/resources
 * 12) What would happen if museums saw their role as facilitators—of interactions with collections, of interactions among people (promoting social interaction and science-related interactions)?
 * 13) How would NHM have to change—and what would be the effect of the change—if visitors left NHM thinking they had seen the world through “new eyes?”
 * 14) How does the museum experience change when some of the galleries provide “safe spaces” for parents with kids to relax and explore? (Heavy doors, confined, visually open spaces, where parents can see their kids without having to keep them together, etc.)
 * 15) What if museums treated audiences like “smartness” was a muscle you build? Some research that changed his perspective. Carol Dweck - studied kids models of intelligence. What do you think it means to be smart? Most had static view - you're smart or you're not. The researcher promoting a dynamic model of intelligence. A muscle you build. The research was amazing. Also devised interventions to get the kids to shift from static to dynamic models. Kids with static models would give up faster. Kids with dynamic model both more persistence and self-efficacy. Why am I here? Empowering people to think for themselves. Give people new eyes to see the world. Often we talk about transfer - we have knowledge to give to people. Not empowering, smacks of static model. Instead - dynamic model we're building all the time, help people build skills. How do get our audience to think in terms of dynamic learning? We have to treat them that way, and model that ourselves. Show our own staff building knowledge, changing, etc.
 * 16) The role of school groups in museums is changing; the role of museums in schools is changing. What can be leveraged for better outcomes and experiences for kids? Who are we serving? How can we better serve them? Through teachers?
 * 17) How are museums effectively contributing to the STEM pipeline?

>
 * Priority Content: Evolution **
 * 1) ** Why is evolution priority content? **
 * 2) ** Because we have collections and can therefore provide the evidence for evolution better than anyone else. **
 * 3) ** Evolution is the basis of biology. **
 * 4) ** Evolution is the key to non-rejection of science. **
 * 5) ** Evolution is the theme that ties the natural past to the natural present and natural future. **
 * 6) ** Technology is providing insight and access into evolution and phylogenetic trees. How do people’s understanding of evolution or feeling about evolution change with experiences with trees? Sdr.seas.harvard.edu/life-on-earth **
 * 7) ** People with a diversity of perspectives attend museums. Of these groups, who could we be most effective with in terms of showing progress towards deeper knowledge of, acceptance of, or less antipathy for evolution? Who should we be targeting? **
 * 8) ** In what ways is connecting family, medical experiences such as flu shots, and domestic animals to evolution effective as to evolution literacy? What are the characteristics of effective experiences? **
 * 9) What about the term evolution? Is it more effective in the long run to use it or not?
 * 10) If people understand or relate to micro-evolutionary processes taking place in their daily lives, does it make it easier to understand or relate to large scale change?
 * 11) What are the most strategic ways to spend resources in terms of evolutionary literacy? What should our strategic goals be?
 * 12) Is cognitive dissonance effective in the short or long term with respect to evolutionary literacy—or at all?
 * 13) What is the most important individual or set of learning goals related to evolution: change in knowledge, attitude, engagement, etc.?
 * 14) Do direct experiences with selection make a difference in accepting evolution? FastPlants, etc.
 * 15) In what ways is health an effective entry point for learning evolution?
 * 16) Where is the aha moment? Do you show them or get them to find it themselves? Can't separate affect and cognition?

Place-based
 * 1) When does placed-based make a meaningful difference in the visitor experience, in the connection to place, and in the visitor’s connection/perception of the museum?
 * 2) In what ways is place-based interpretation reinforced/undermined by including rapid response about current events in other areas? What difference does branding/connection to other place-based museum make? In what situations does branding/connecting rapid response exhibits to other place-based museums reinforce the place-based notion/identity of a museum?
 * 3) What are the potentials and affordances of pairing NHM and nature centers together? What are our respective strengths? What is the value-added?
 * 4) How does the NHM experience change—in what ways does it become stronger or more relevant—when nature is thought to include urban nature and when humans are considered as animals and part of nature?
 * 5) How can local parks, yards, etc. be leveraged as part of NHM or the NHM experience? How can we leverage them as part of Public Participation in Scientific Research (caise.insci.org/resources look for Bonney et al.) as well as NHM research?
 * 6) What is the impact of NHM playing a role in recession rethinking: urban farming, re-naturing, squatting?
 * 7) What is urban indigenous knowledge? How does it develop? How does it strengthen a community? How can NHM positively contribute to, understand, or document urban indigenous knowledge?
 * 8) What if NHM really did extend into or serve as an extension of people’s backyards?

Organizational
 * 1) How would NHM experiences change if all NHM (had to work) worked in public spaces or behind glass walls? What would happen if NHM staff all acted as public scientists or as part of the visible asset/exhibition of the NHM? What kind of person would work at NHM? How would the experience change? What is the root of the tension between NHM staff and public viewing? Do certain personality types gravitate to certain kinds of NHM work? What does it take so people become comfortable working in public spaces?
 * 2) How would the NHM experience change if we told our stories from the lens of passion and the heart as opposed to starting/focusing solely on the lens of science? Is science better communicated/embraced by telling its story through other lens (passion/affect, beauty, patience, etc.)? How would these approaches transform NHM practices //writ large//, from parking to galleries to science?
 * 3) How would the perception and role of NHM change if we consistently framed ourselves and enacted the role of facilitator/convener? (The implication is that NHM would consistently promote and imbue opportunities for dialogue throughout the NHM experience: virtual and physical.
 * 4) Multiple contexts of research on objects in the collection - science around the collections is dynamic, current, evolving. Transparency/sharing of research currently being done on the collection.
 * 5) How would NHM be different if staff felt trusted?
 * 6) How would the NHM culture change if staff—not just scientific staff—were encouraged to blog about their museum work?
 * 7) What if - to be a museum employee - you had (mandatory) to work in a public space doing your work? What if part of your work was sharing your work? Would the type of person working at the museum change? Would the museum change? How? Better? Worse? For what/whom? (Fishbowl.)
 * 8) Do catch phrases ground staff through change processes? “Engage, inspire, and empower” (Monterey Bay Aquarium)
 * 9) Why aren’t NHM a network? What would be different if we were?
 * 10) What lessons to NHM have to learn from other “news” and information organizations that have “lost” relevancy in 21st century society, i.e. newspapers/print journalism, network TV?
 * 11) How does our effectiveness change when we link content across organizations?

Evidence of Learning
 * 1) What are the holy grails for evidence of learning?
 * 2) Positive affect
 * 3) Identity as science/nature professional/innovator, contributor/technician, or science literate person
 * 4) What counts as evidence of learning for each LSIE strand?

Emotions/Passion
 * 1) Are NHM more effective in providing lasting learning experiences if our communications is led by the heart/emotions/passion that by science? Research suggests that learning—and specifically engagement, starts with the heart, not with “content.” Is this really true/effective?
 * 2) Cognition and affect are a false dichotomy. What is the question?

Questions/Tensions
 * 1) What are the parameters under which the following statement is true and is false/misguided? “Don’t ask visitors what they want.”
 * 2) NHM often say, “If we could only get new people through the door,” but we also know we aren’t great, in general, at achieving repeat visitation. What needs to change for us to (1) be honest with ourselves, (2) understand what would promote repeat/frequent visitation, or (3) what happens when people visit?
 * 3) What would it take—bright spots/successes, outright mandate (and by whom)—for NHM to break down silos and barriers in order to work together to implement all these ideas and dreams that got us to work in museums in the first place? What would it take? What would it take to keep that ball rolling?
 * 4) We are mostly an urban world; we need to embrace this. It’s not human vs. nature. Or nature=wild. It is what is the nature around us?


 * Discussion of Group A: Learning Research Agenda (Reconfiguration/Session 4) **

Collection is a bucket we keep coming back to: how do we leverage collections (or representations of collections) to achieving learning goals around particular topics and/or with specific audience.

We are now ready to do a mash-up, leveraging our assets around particular topics or audiences.

Under each bucket we want sets of questions in each bucket that individual projects can draw from, so we’re asking similar questions—or some of us start asking research questions related to our work. This is in part so institutions can draw them to get started.

We want to have specific questions so that folks can draw from in their own work, but then we would want to synthesize findings.

What is the effect of having collections on the learning experience?

The audience, collection, and the approaches you are using?

Collections have accrued scientific assets and stories that are associated with them.

We need to do literature reviews about our major topics/buckets, so we can draw on them.

Our overarching question: how do object based collections facilitate learning in terms of the six strands? And to break that out, we were thinking about those six strands in one column and in a matrix, what are the aspects of objects-based collections affect learning? Authenticity: how real are the objects? Completeness. How complete is the collection, including context of the collection? Uniqueness. How unique are the objects in relation to the visitor? Scientific importance. Did science radically shift because of this object or collection? Accessibility to sensory perception? What are the different ways that the museum has allowed you to engage with the collection, including visualizations, representations, etc.?

Cross-cutting issue: Audience. Breaking out different segments (age, education, social groupings)

Where is the affect of having and using object-based collections?

So examples of topics that would fit in these cells are questions/issues like:


 * 1) What would it take to have visitors value NHM primarily for our collections? Nature of Science/Nature of Museums type question? To what extent do experiences provide opportunities for visitors (and learning outcomes) that support the mission of the museum—that collections are our most important asset. Do publics get our public value?
 * 2) What is the role of the “real:” real scientist, real collection, etc. What is the advantage/value of having the “real?”
 * 3) What are the differences in using new media to either approximate or take it deeper or further than real objects can? What is the role of new media?
 * 4) How does “embodiment” personalize and impact the learning experience with the collection?

Research Question: How do different facets of object-based collections engender different kinds of learning (LSIE strands) in Natural History Museums?


 * **LSIE Strands** |||||||||| ** FACETS OF OBJECT-BASED COLLECTIONS **  ||
 * ^  || Authenticity || Completeness || Uniqueness || Scientific Importance || Accessibility to sensory perception & manipulation ||
 * 1. excitement, interest, and motivation ||  ||   ||   ||   ||   ||
 * 2. generate, understand, remember, and use concepts ||  ||   ||   ||   ||   ||
 * 3. Manipulate, test, explore, predict, question, observe, and make sense ||  ||   ||   ||   ||   ||
 * 4. Reflect on science as a way of knowing ||  ||   ||   ||   ||   ||
 * 5. Participate in scientific activities and learning practices with others ||  ||   ||   ||   ||   ||
 * 6. Think about themselves as science learners and develop an identity ||  ||   ||   ||   ||   ||

Strand 1: Experience to learn about phenomena in the natural and physical world. Strand 2: Come to, explanations, arguments, models and facts related to science. Strand 3: of the natural and physical world. Strand 4:; on processes, concepts, and institutions of science; and on their own process of learning about phenomena. Strand 5:, using scientific language and tools. Strand 6: as someone who knows about, uses, and sometimes contributes to science.

In what ways and to what extent does the perceived authenticity of objects in a collection (a) enhance visitors' interest and excitement about the topic and (b) contribute to their understanding of the scientific process or practices involved in studying the topic?

// This section is intended to capture the challenges and opportunities for collaboration within and between natural history museums relevant to the intersection of these assets and opportunities. This section should identify key challenges, key opportunities, and important research questions about how we work together within and across museums to break down traditional "silos" that could hinder our progress. //
 * Challenges and Opportunities for Collaboration **