Marc J. Stern, Professor, Virginia Tech Robert B. Powell, Professor, Clemson University B. Troy Frensley, Assistant Professor, UNC Wilmington
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Marc J. Stern, Professor, Virginia Tech
Robert B. Powell, Professor, Clemson University
B. Troy Frensley, Assistant Professor, UNC‐Wilmington
Best Practices
Systematic literature review Stern et al. 2014. Environmental education program evaluation in the new millennium: Skibins, J.C., Powell, R.B., and M.J. Stern what do we measure and what have we 2012. Exploring empirical support for learned? Environmental Education Research interpretation’s best practices. Journal of 20(5): 581‐611 Interpretation Research 17(1): 25‐44.
Empirical study of live interpretive
programs in the Parks
24 park units
376 live interpretive programs
3,603 retrospective visitor surveys
56 program characteristics
4
Lesson learned
Reliable observation is possible
15 factors emerged that were
most linked to positive outcomes
Confidence of the Interpreter
Authenticity of the interpreter
Organization of the story
Relevance, responsiveness
Not focusing on just the facts!
Stern and Powell 2013. Journal of Interpretation Research. 18(2)
5
What practices lead to better
outcomes in EE programs?
But what is “success”?
Is there a set of crosscutting
outcomes for all EE?
6
Environmental literacy
Knowledge
Dispositions
Attitudes
Skills
Behaviors
Tbilisi Declaration, Hungerford,
Volk, McBeth, NELA, and many
others
7
Educational standards
8
21st Century Skills
A broad list of societal literacies
Critical thinking
Communication
Collaboration
Creativity
9
Positive youth development
Emotional/social Intelligence
Five Cs
Competence, confidence,
character, connection, caring
Twelve student strengths‐
Mind (Curiosity, Self‐control,
Zest)
Heart (Purpose, Social/emotional
Intelligence, Gratitude)
Will (Grit, Growth, Optimism)
10Crosscutting Outcomes for EE: The potential to be transformational
Big Challenges 1. Defining Aspirational “Success” 2. Crosscutting (Relevant) 3. Measuring Outcomes: Variability and sensitivity Addressing skew and social desirability bias
Defining “Success” NPS Advisory Board Education Committee NAAEE Advisory Group Clark, Heimlich, Ardoin, & Braus Delphi Study National Park Foundation Learning Alliance ANCA: Association of Nature Center Administrators
Behavior
Enjoyment
change
EE21 outcomes
Self‐
Learning
Participatory process efficacy
Env. Interest/
attitudes motivation
21st Century Connection
skills to place
Env.
behaviors Meaning
School
Cooperation
behaviorsPilot Studies
Great Smoky Mtns Everglades NP NC Natural Science USFS Snorkeling
Glen Helen, OH NorthBay, MD Cuyahoga NP Oregon outdoor
schoolsPsychometric Testing: Statistical fit!
Confirmatory factor analysis
(CFA)
Consistent 10 factor model for
each sample
Measurement invariance:
configural, metric, structural
Moderate skew (M=6‐7) on 11
point scales!
Revised survey
8‐10 minutes
2‐5 items per construct
Logistical
Retrospective only for
national studyBehavior
Enjoyment
change
EE21 outcomes
Self‐
Learning
12 Outcomes efficacy
2 Single Items and 10 Scales
Env. Interest/
11 point scales attitudes motivation
Administrative Script
21st Century Connection
skills to place
Env.
behaviors Meaning
School
Cooperation
behaviorsEE21: Available for your use
Retrospective for shorter programs
Pre‐Post for longer programs
Powell, R.B., Stern, M. J., Frensley, B.T., & Moore, D. (2019) Identifying and developing crosscutting environmental
education outcomes for adolescents in the 21st Century (EE21). Environmental Education Research. DOI:
10.1080/13504622.2019.1607259What leads to better outcomes? Systematic observation of 334 EE programs in 24 states (plus DC) by 90 unique program providers Day field trip programs Middle school kids (grades 5‐8) What program characteristics are most powerfully linked to outcomes across a range of contexts?
Behavior
Enjoyment
change
EE21 outcomes
Self‐
Learning
Participatory process efficacy
Extensive pilot testing
Env. Interest/
Statistical validation attitudes motivation
Blue = EE21 scale
21st Century Connection
skills to place
Env.
behaviors Meaning
School
Cooperation
behaviorsProgram variables
Educator characteristics
Passion, comfort, apparent knowledge . . .
Pedagogical approaches
Investigation‐focused, play‐based, personal
relevance, issue‐based, reflection, storytelling,
central message, social‐ecological connections . . .
Contextual factors
Group size, environmental context, weather, role of
visiting teachers and chaperones . . .22
Variable Definition Scores
Passion The educator’s apparent 1 2 3 4
Educator seems Educator shows low Educator shows Educator seems extremely
level of enthusiasm for the
completely detached/ levels of passion overall moderate levels or passionate about the lesson
lesson content and the
disinterested from the sporadic instances of
overall authentic emotional lesson high passion
connection with which the
material is delivered
Free Degree to which students 1 2 3 4
No efforts made to Minimal efforts to Moderate efforts to Major efforts to incorporate
Exploration were encouraged to explore
incorporate free incorporate free incorporate free free exploration for most of
the environment in their exploration exploration exploration the program
own way
Relevance 1 2 3 4
Degree to which the lesson
No efforts made to Minimal efforts to create Moderate efforts to Major efforts to link the
content references or makes create relevance linkage to students’ lives, create direct experience directly to
explicit connections to the perhaps just through relevance to students’ students’ personal lives at
students’ experience outside simple analogy. lives home
the realm of the instruction.
Central There is a clearly No effort or not Some effort made but Easy to detect central Clearly communicated and
present. unclear, ambiguous, or message but not fully well-developed central
message(s) communicated central
difficult to discern actual developed or not message.
message (or messages) that central message. clearly meaningful.
demonstrates why what
they learned should matter
to them.
Play‐based Degree to which the lesson 1 2 3 4
No efforts made to Minimal efforts to Moderate efforts to Most of the program was
actively engages students in
incorporate play-based incorporate play-based incorporate play- play-based.
games or competition as an learning learning – possibly a based learning. A
intentional teaching short game. game(s) made up a
technique. meaningful portion of
the program.Methods overview Systematic observations of programs Immediate post‐experience surveys with participants Surveys with teachers Interviews with educators
What leads to better outcomes?
25Preliminary findings
The findings that follow are based on
ongoing analyses
Just because some elements are not
shared here, does not mean they are not
also important to program success (they
may not have varied in our observations,
or we may not have measured them!)
26Demographic considerations
Strong effects of grade level, race, and socioeconomic
context
Lower grades more positive outcomes
Latinx audiences more positive outcomes
Lower economic classes more positive outcomes
27Controlling for grade level and race,
what characteristics were most
consistently linked with student
outcomes?The group Effects on EE21 Staging (the state in which groups showed up) Preparation Smaller group sizes 29
Use of the setting Being outside in natural settings Novelty Place-based education
Instructor behaviors
Emotional support
Passion, sincerity, affinity-seeking,
positive communications
Responsiveness
Confidence & clarity
Apparent comfort and knowledge,
instructional clarity, content clarity
Opportunism
31Quality communication
Verbal engagement
Question quality
Effective class management
Effective transitions
Quality conclusions
32What about lecture‐focused programs? No, my friend . . .
Aiming for behavior change?
Systems focus
Identification as “scientists”
Advocacy
Fact focus (neg.)
34Strongest relationships
(Relative Importance Analysis)
EE21 Behavioral intentions
Transitions Advocacy
Group size (negative)
Novelty
Conclusion quality
Natural settings
35Contributions EE21 surveys New methods to examine a far wider range of programs across wider contexts Identification of what practices tend to lead to better outcomes
What’s next
Expanding the sample
Fall 2020 and Spring 2021
Effective sets of practices in different contexts and
diverse audiences
Establishing evidence‐based EE21 learning
networks
Using EE21 to learn from each other, enhance
programs, and evaluate the results
37Thank you!
mjstern@vt.edu
rbp@clemson.edu
The full research team:
Troy Frensley, Tori Kleinbort, Ryan Dale, Anna O’Hare, Hannah Lee,
Daniel Pratson, Kaitlyn Hogarth, Eric Neff, Neil Savage 38You can also read
