IMPROVING ENGAGEMENT IN REMOTE LEARNING ENVIRONMENTS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
IMPROVING ENGAGEMENT IN REMOTE
LEARNING ENVIRONMENTS
FACILITATING TRANSITION INTO HIGHER EDUCATION
JOHN WYATT, UNIVERSITY COLLEGE DUBLIN
DR. MAURICE KINSELLA, UNIVERSITY COLLEGE DUBLIN
© 2021 NACADA: The Global Community for Academic Advising
The contents of all material in this presentation are copyrighted by NACADA: The Global Community for Academic Advising, unless otherwise indicated.
Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All
rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except
with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent
with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated,
published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright
infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and NACADA: The Global Community for Academic Advising are
service marks of the NACADA: The Global Community for Academic Advising
OUTLINE • UCD & UCD LEAP • DESIGN & IMPLEMENTATION • COVID-19 • VLE DESIGN CHANGES • KEY FINDINGS • LESSONS LEARNED & FUTURE
UCD VET MEDICINE: THE SCHOOL • UCD Vet Teaching Hospital open 24/7/365 • Top 25 QS World Subject Ranking • AMVA, EAEVE, VCI accredited • Requirements from UCD & accreditors
UCD VET MEDICINE: THE STUDENTS • Approx. 300 1st year students • 33% International Students (23% UCD Avg.) • Classroom & practical learning components • Student Adviser for support
UCD LEAP: SUPPORT DELIVERY ISSUES • Disengagement only apparent post-exams • Difficult re-engaging students • Existing supports under-used • Negative impact on wellbeing • Retention issues • Social integration issues
UCD LEAP: CHANGES NEEDED • Real-time engagement info sources • Support interventions linked to data • More immediate support for better outcomes • Signposting both generic and tailored supports • UCD Live Engagement & Attendance Project
INITIAL DESIGN
Progression
• Student Feedback
2019
• Student & SA Feedback
2020
• Student & Research Team Feedback
LEAP Attendance
2021
DESIGN Data
• Bluetooth attendance data smartphone app
Intervention • At-risk students contacted
Reporting • Underpinned by Self-Determination Theory
• Self-populated
IMPLEMENTATION: INITIAL ROLLOUT • More attendance data visibility • Real-time interventions commenced • Preliminary findings confirmed relationship • Setup issues (accuracy & timetabling) • Embedding issues (student & staff buy-in) • High-attendance support gap
IMPLEMENTATION: FEEDBACK & CHANGES • “Trusted Persons” format • Light touch first intervention • Stage 0 creation • VLE identified as key engagement source
COVID-19
“My appreciation for “My learning is
“It’s a lot harder to the teaching staff has nothing like it was
engage in such a grown significantly for and I have never felt
clinical program the supports and work worse about my
remotely” they put in for us” performance”
Classes 80,000+
Students
(1162) data points
(70x avg)
lostVLE DESIGN: CRITERIA 1. Login Frequency 2. Quality of interaction
VLE DESIGN: PROGRAMME VIEW
VLE DESIGN: STUDENT LOG EXAMPLE
KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)KEY FINDINGS: VLE DATA
N: MVB1 (94), MVB2 (89), GE1 (52), VNUR1 (44)VLE DESIGN: STUDENT LOG EXAMPLE
Flag Info Autumn Spring
Total Flags 95 161
Unique Students flagged 37 43
Avg Flags per student 2.57 3.74
Avg Flags by week 7.92 13.42
Of students who failed modules, 54.5% were flagged, 45.5% were unflaggedKEY FINDINGS: VLE DATA
SEAtS Usage & GPA VET10060 Access % VLE Topic Access
GPA
4.1 90.00%
4 80.00%
70.00%
3.9
60.00%
3.8
50.00%
3.7
40.00%
3.6
30.00%
3.5 20.00%
3.4 10.00%
3.3
0.00%
-6 -4 -2 0 2 4 6 Topics 0 100 200 300 400
SEAtS UsageKEY FINDINGS: ASSESSMENT DATA 2020/21 Assessment Component Type 2019/20 Assessment Component Type
RESEARCH AND FEEDBACK
Site:
School of Veterinary Medicine, University College Dublin
Participants:
Students: 2018:n=13 2019: n=18; 2020: Interviews n=14; 2021: n=21 SAs: 2021: n=10
Methodology:
Mixed-method approach
Instruments:
i.Questionnaire – Written
ii.Qualitative Interview – Phone and Written
Analysis:
Reflexive Thematic Analysis
(Braun & Clarke, 2014; Clarke & Braun, 2018)KEY FINDINGS: 2020 STUDENT FEEDBACK
• F2F instruction is missed “Physical attendance
“Professors are very
is important so they
available for help and
can explain fully
questions”
what they mean”
• Student Advisers seen as vital “I would not be here today without them”
“Really helped with my personal growth”
“Helps you try to solve the problem”
• Support for early flagging “If its not helping “There’s that 1% that
every person but it is you maybe need to
helping one person, keep an eye so
you like that” reaching out is nice”KEY FINDINGS: 2021 STUDENT FEEDBACK
• F2F instruction is still missed “Online learning “I don’t feel like a
makes my studies student in university
seem more like without any practical
chores” work”
“
“(Advisor Name) is a great help”
• Student Advisers still seen as vital
“She is amazing and so helpful”
“Great to know that there is a readily
available advisor always there for you”
“Lack of organization “The balance of
• Challenges of online learning
of lecture content” college work and
“Bombarded personal time has
with work” been lost”KEY FINDINGS: 2021 ADVISER FEEDBACK
• F2F support is still needed “Difficult to support
“My student cohort
students remotely, in
are finding remote
particular when
learning difficult”
students are upset”
• Student Advisers foster engagement “Supporting students who may feel
disconnected”
“Key element of role is supporting student
integration to third level”
“Online space has a “Tasks can be
• Case for blended approach completed at a
place going into the
distance but some
next iteration of face to face contact
student services” is desired”LESSONS LEARNED
• VLEs
capacity to foster multi- • Address VLE Module ‘Siloing’.
dimensional engagement. • Existswithin UCD’s digital
• Ongoingrole of on-site infrastructure.
student engagement. • Ready integration into
stakeholder practice.
Conceptual Operational
Economic Technical
• Off-site
architecture • Scalability
needed.
• Actionable intervention
• Low construction and
data.
maintenance costs.
• Accurate, but limitations
(ie: asynchronous downloading).RECOMMENDATIONS: KEY INSIGHTS • VLE data can enable Advisers to facilitate interventions • Digital and in-person supports are interlinked • Try to capture relative, not absolute engagement • Remote learning has changed support delivery
RECOMMENDATIONS: FUTURE ACTIVITY • Continue assessing VLE engagement model • Implement ‘blended’ engagement monitoring tools • Disseminate academic & internal lessons learned • Identify value-add activity areas for continuation
CONTACT US • john.wyatt@ucd.ie • maurice.kinsella@ucd.ie • niamh.nestor@ucd.ie
REFERENCES
1. Akçapınar, G., Mohammad, N. H., Majumdar, R., Flanagan, B., & Ogata, H. (2019). Developing an early-warning system for spotting at-risk students by using eBook interaction logs. Smart
Learning Environments, 6(1), 1-15. doi:http://dx.doi.org.ucd.idm.oclc.org/10.1186/s40561-019-0083-4
2. Alves, P., L, M., & Morais, C. (2017). The Influence of Virtual Learning Environments in Students' Performance. Universal Journal of Educational Research, 5, 517-527.
doi:10.13189/ujer.2017.050325
3. Alves, P., Miranda, L. & Morais, C. (2016). Learning styles and access to virtual learning environments in the academic performance, Academic Conferences International Limited, Kidmore
End, pp. 25.
4. Ayala, J. C., & Manzano, G. (2018). Academic performance of first-year university students: the influence of resilience and engagement. Higher Education Research & Development, 37(7),
1321-1335. doi:10.1080/07294360.2018.1502258
5. Black, A. E., & Deci, E. L. (2000). The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: A self-determination theory perspective.
Science Education, 84(6), 740–756.
6. Brooks, C., Thompson, C., Teasley, S. (2015) A time series interaction analysis method for building predictive models of learners using log data. In: Proceedings of the Fifth International
Conference on Learning Analytics and Knowledge, pp. 126–135. ACM, New York (2015a)
7. Brophy, J. E. (1988). On motivating students. In D. Berliner & B. Rosenshine (Eds.), Talks to teachers (pp. 201–45). New York: Random House.
8. Chipchase, L., et al. (2017). ‘Conceptualising and Measuring Student Disengagement in Higher Education: A Synthesis of the Literature’. International Journal of Higher Education. 6(2), 31-
42.
9. Cook, David & Artino, Anthony. (2016). Motivation to learn: an overview of contemporary theories. Medical Education. 50. 997-1014. 10.1111/medu.13074.
10. Cooper, M., Ferguson, R., & Wolff, A. (2016). What can analytics contribute to accessibility in e-learning systems and to disabled students' learning? Paper presented at the 99-103.
doi:10.1145/2883851.2883946
11. Credé, M., et al. (2010). Class Attendance in College: A Meta-Analytic Review of the Relationship of Class Attendance with Grades and Student Characteristics. Review of Educational
Research. 80(2), 272-295.
12. Fuller, M. B., Wilson, M.A., & Tobin, R. M. (2011) The national survey of student engagement as a predictor of undergraduate GPA: a cross-sectional and longitudinal
examination, Assessment & Evaluation in Higher Education, 36:6, 735-748, DOI: 10.1080/02602938.2010.488791
13. Gardner, J., & Brooks, C. (2018;2017;). Student success prediction in MOOCs. User Modeling and User-Adapted Interaction, 28(2), 127-203. doi:10.1007/s11257-018-9203-z
14. Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success.
Internet and Higher Education, 28,68–84.
15. Hlosta, M., Zdrahal, Z., & Zendulka, J. (2017). Ouroboros: Early identification of at-risk students without models based on legacy data. Paper presented at the 6-15.
doi:10.1145/3027385.3027449
16. Jayaprakash, S. M., Moody, E. W., Laurıa, E. J., Regan, J. R., and Baron. J. D. Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics,
1(1), 6-47.
17. Kinsella, M., Nestor, N., Rackard, S., Last, J. & Wyatt, J. Using Attendance Analytics as a Motivational Tool for First-year University Students: The Live Engagement and Attendance Project
(LEAP). The European Conference on Education 2020 (ECE2020), 16th-19th July 2020 University of London. IAFOR.REFERENCES
18. Kinsella, M., Wyatt, J. & Nestor, N. 2021c. Responding to Acute Changes in Higher-education Engagement: Insights from UCD ‘Live Engagement and Attendance Project’. Ireland
International Conference on Education (IICE 2021). Dun Laoghaire, Ireland.
19. Kelly, Anna M., and Padden, Lisa (2018). Toolkit for Inclusive Higher Education Institutions: From Vision to Practice. Dublin: UCD Access & Lifelong Learning
https://www.ucd.ie/all/t4media/0274_UCD_TOOLKIT_1118_ONLINE_LR.pdf
20. Morra, T., & Reynolds, J. (2010). Universal Design for Learning: Application for Technology-Enhanced Learning. Inquiry: The Journal of the Virginia Community Colleges, 15 (1). Retrieved
from https://commons.vccs.edu/inquiry/vol15/iss1/5
21. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599.
http://dx.doi.org/10.1016/j.compedu.2009.09.008.
22. Mubarak, A. A., Cao, H., & Zhang, W. (2020). Prediction of students’ early dropout based on their interaction logs in online learning environment. Interactive Learning Environments, , 1-
20. doi:10.1080/10494820.2020.1727529
23. Nistor, N. & Neubauer, K., From participation to dropout: quantitative participation patterns in online university courses. Computers & Education, 55, 2, (2010), 663-672.
24. National Centre for Learning Disabilities (2020). Promise and Peril: Examining the Role of Ed Tech for Students With Disabilities https://www.ncld.org/wp-
content/uploads/2020/07/Promise-and-Peril-Examining-the-Role-of-Ed-Tech-for-Students-with-Disabilities.07212020.pdf
25. Nik Nurul Hafzan, Mat Yaacob, Safaai, D., Asiah, M., Mohd Saberi, M., & Siti Syuhaida, S. (2019). Review on predictive modelling techniques for identifying students at risk in university
environment. MATEC Web of Conferences, 255, 3002. doi:10.1051/matecconf/201925503002
26. Ormond Simpson (2013) Student retention in distance education: are we failing our students?, Open Learning: The Journal of Open, Distance and e-Learning, 28:2, 105-
119, DOI: 10.1080/02680513.2013.847363
27. QS. 2020a. The Impact of the Coronavirus on Global Higher Education. QS. Accessed at: http://info.qs.com/rs/335-VIN-535/images/The-Impact-of-the-Coronavirus-on-Global-Higher-
Education.pdf
28. Sclater, N. 2013. Learning Analytics: The Current State of Play in UK Higher and Further Education. JISC
29. Simpson, S. (2013). ‘Student retention in distance education: are we failing our students?’ Open Learning: The Journal of Open, Distance and e-Learning. 28(2), 105-119.
30. UCD Agile: Mapping our Student’s Experience of UCD – Executive Summary: https://www.ucd.ie/agile/whatagiledoes/mappingourstudentsexperience/thestudentperspective/
31. UCD Index Survey 2019: https://www.ucd.ie/itservices/t4media/UCD%20INDEx%202019%20Student%20Survey%20Quantitative%20Analysis%20.pdf
32. Valle, A., Núñez, J. C., Cabanach, R. G., González-Pienda, J. A., Rodríguez, S., Rosário, P., et al. (2009). Academic goals and learning quality in higher education students. The Spanish
Journal of Psychology, 12(1), 96-105.
33. Webber, Karen L., et al. Does Involvement Really Matter? Indicators of College Student Success and Satisfaction. Journal of College Student Development, vol. 54 no. 6, 2013, p. 591-
611. Project MUSE, doi:10.1353/csd.2013.0090.
34. Wolff, A., Zdrahal, Z., Nikolov, A. & Pantucek, M. (2013) Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment, LAK13, Leuven,
Belgium.THANK YOU!
You can also read
