FOCUS GROUPS: GATHERING REAL LIFE FEEDBACK FOR PHARMACY PROGRAM ASSESSMENT - SANDRA S. BOWLES, EDD, RN, CNE ASSISTANT DEAN FOR ASSESSMENT & ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
FOCUS GROUPS: GATHERING REAL LIFE FEEDBACK
FOR PHARMACY PROGRAM ASSESSMENT
Sandra S. Bowles, EdD, RN, CNE
Assistant Dean for Assessment & Special Projects
University of Charleston School of PharmacyOBJECTIVES: This presentation is designed to assist participants to: • Briefly examine the history & utilization of focus groups. • Employ the fundamental mechanics of conducting focus groups. • Evaluate potential use of focus groups to assess pharmacy program effectiveness. • Share experiences with effective use of focus groups.
HISTORY AND UTILIZATION OF FOCUS GROUPS
DEFINING THE FOCUS GROUP • Small group (8 to 12 people) guided though a discussion of specific questions or topics • Purpose of group/objectives clearly defined • Discussion is focused and care taken to avoid getting “off task” • Group has a moderator or facilitator and a recorder
HISTORY • Created in the 1940s by the Bureau of Applied Social Research as a qualitative research technique • Popularity has waxed and waned over time • Frequently used in marketing • In higher education most often used in education, sociology, education, and health sciences
FUNCTIONS • Relatively efficient means for soliciting information • Allows for group sharing in a conversation mode • Can reveal wealth of information and insight • Indirect measure of perceptions, experiences, values
Surveys allow us to collect information but if you need to measure understanding at a deeper level or seek to get at the “feeling” aspect of assessment, then a focus group(s) may be effective.
SURVEYS vs. FOCUS GROUPS • Surveys typically have written, closed-ended, relatively narrow questions which are quantitatively scored • Responses in focus group are spoken, open- ended, relatively broad and qualitative. (Allows for observation of nonverbal communications and group interaction)
AS A TOOL IN ASSESSMENT
FOCUS GROUPS USEFUL TO:
• Gather data
• Test assumptions
• Validate attainment of outcomes
• Generate marketing data
• Identify program strengths or need for
revision
• Triangulation to support or augment other
assessment methodsFOCUS GROUPS ARE NOT: • A debate • A gripe session • An opportunity to problem-solve • A platform for conflict resolution • An educational session
LIMITATIONS • Require time commitment • Small samples may not be suitable for generalizations • Require thoughtful development of questions • Group leader may influence group interaction • Data somewhat labor intensive to analyze
NEGATIVES • As with other forms of qualitative research, there is potential for observer dependency or bias • Facilitator may impede or impact participation • Lack of anonymity
MECHANICS OF CONDUCTING FOCUS GROUPS
PURPOSE • Must clearly identify the purpose for the session(s) • Know how you will use the findings
PREPARE QUESTIONS • Limit number to less than 10 (6-8 ideal) • Open-ended • Avoid biased, leading or sensitive topics • Ask about both positive and negative aspects • Plan on 60 to 90 minute session
• Avoid dichotomous questions • Use questions that can include all participants • Avoid “Why?” • Use questions that take group back to an experience rather than to the future
TYPES OF QUESTIONS • Opening question that all can respond to “break the ice” • Introductory question • Key questions • Concluding question
Moderator & Recorder • Must be impartial • Sets tone for the session • Identifies any ground rules • Must not show verbal or nonverbal reaction to comments • Arrange for audio or video taping in addition to note taking
Group • Should be a representative sample • Eight to twelve is ideal • Use of multiple groups to assure greater consensus of the larger group • Replication of groups contributes to reliability • Group needs to know how information generated will be used
SITE • Should be neutral • Comfortable & conducive to group discussion • Comfortable seating • Participants face-to-face
INDUCEMENTS • Participants need to why session is important • A written invitation helpful • Provide refreshments or meal • Coupons, gift certificates, drawing • Indicate how information will be shared • Thank group for their participation
CONDUCT IN RESPECTFUL MANNER • Let group know session is recorded • Make certain all participants are heard • Draw out quieter members • Focus on questions and keep discussion on track • Do NOT allow session to become a “gripe” session • Avoid domination by one or two individuals
RECORDING
• Notes should be clear
• Listen for notable quotes
• Record key points and themes
• Individual or group insights
• Passionate comments
• Describe body language or non-verbal
activityANALYSIS • Transcribe recording(s) • Combine with written notes • Eliminate non-essentials • If multiple groups, read all responses to each question • Separate responses with questions • Look for themes for each question
• When themes or categories identified sort comments • When possible note number of same or similar comments • Synthesize themes/categories and summarize findings for each • Include notable quotations • Prepare report for sharing as appropriate
EXAMPLES OF USE IN PHARMACY PROGRAM
FOCUS GROUPS TO ASSESS PROGRAM EFFECTIVENESS
STUDENT ASSESSMENT OF
LEARNING EXPERIENCES
• How do students perceive their experience?
• In what activities are they engaged?
• Opportunities for critical thinking, problem-
solving?
• Opportunities to do/observe patient teaching?
• Interactions with health care team?
• How could experience be improved?STUDENT ASSESSMENT OF LEARNING EXPERIENCE (IPPES)
TYPICAL QUESTIONS • Begin with “round robin” – identify nature of site • Tell me about a typical assignment at site? • How effective were your interactions with other members of the health care team? • What was one important thing you learned from this experience? • How could this rotation be more effective?
CHALLENGES IDENTIFIED • Some students uneasy interacting with physicians & nurses • Sometimes felt “out of place” • Didn’t always understand what they were seeing • Sometimes no one available to student
FEEDBACK USED • Created better student orientation to IPPEs • Improved site selection • Preceptor training • Feedback to preceptors to increase awareness of student needs • Additional staff to visit sites
MARKETING
MARKETING FOCUS GROUP • What are the best qualities of the School of Pharmacy? • What is our niche, what is special here? • How would you “sell” the program to perspective students? • How can we more effectively market the school?
PROGRAM ASSESSMENT WITH GRADUATING STUDENTS
GRADUATING STUDENTS (P4) • Reflecting over your four years in the SOP, what has been a positive experience for you? • What are the strengths of the program? • What are the challenges? • Do you feel prepared to enter pharmacy practice? • What additional information would you like to share?
TRIANGULATION • Multiple groups with same questions • Students complete AACP Graduating Student Survey • Findings compared
FINDINGS • Report constructed using focus group data and AACP Graduating Student Survey • Identified program strengths and challenges • Identified what done/plans for revision • Shared with graduates in fall after graduation
EMPLOYER
EMPLOYERS • Are our graduates prepared for entry into practice? • What evidence can you share that our graduates understand the health care system? • What would you identify as the major strengths of our graduates?
• What do our graduates lack in the areas of: – Pharmacy content/didactic education? – Clinical training (skill set)? • As you think about where your agency is going in the future, what do we need to be adding to the content or skill set of our graduates?
DOUBLE SET
DOUBLE SET • Independent groups of students and faculty with same questions • “Focus” on common problems, i.e. advising and student portfolio • Able to compare and contrast findings
WHAT EXPERIENCES WITH FOCUS GROUPS CAN YOU SHARE?
QUESTIONS?
REFERENCES Community Tool Box: Section 6. Conducting Focus Groups. Work Group for Community Health and Development at the University of Kansas. Krueger, R. and Casey, M. A. (2000). Focus Groups: A Practical Guide for Applied Research, 3rd ed. Thousand Oaks: Sage. Palomba, C. and Banta, T. (1999). Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. Jossey-Bass Publishers: San Francisco. Revak, M. and Scheffel, D.(2001). A Top Ten List of Assessment Tools for Academic Courses and Programs. Higher Learning Commission: A collection of Papers on Self-Study and Institutional Improvement, Chapter 10: Assessment of Student Academic Achievement: Tools for Assessment. Rogers, F. Assessment Methods, Direct and Indirect Measures of Student Learning prepared for the AAHE Assessment Conference, June 2002. Steward, D., Shamdasani, P. and Rook, D. (2007). Focus Groups: Theory and Practice, 2 nd ed. Thousand Oaks: Sage Publications. Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Anker Publishing Company: Boston, MA.
CONTACT INFORMATION Sandy Bowles, EdD, RN, CNE Dean Emeritus Division of Health Sciences Assistant Dean for Assessment & Special Projects School of Pharmacy University of Charleston Charleston, WV 25304 sbowles@ucwsv.edu 304.357.4376
You can also read
