Eva WONG & Theresa KWONG (King CHONG, Dimple THADANI & Wing - - PowerPoint PPT Presentation
Development and Evaluation of Communities of Practice (CoPs) in Hong Kong Eva WONG & Theresa KWONG (King CHONG, Dimple THADANI & Wing Leung WONG) Centre for Holistic Teaching and Learning Hong Kong Baptist University Lilly
Eva WONG & Theresa KWONG (King CHONG, Dimple THADANI & Wing Leung WONG) Centre for Holistic Teaching and Learning Hong Kong Baptist University
Lilly International Spring Conference
COLLEGE AND UNIVERSITY TEACHING AND LEARNING
May 28-31, 2015
4
We are here!
‒ Academy of Visual Arts ‒ Faculty of Arts ‒ Faculty of Science ‒ Faculty of Social Sciences ‒ School of Business ‒ School of Chinese Medicine ‒ School of Communication ‒ School of Continuing Education
5
report published in December 2010.
– enhance teaching and learning – a network of outstanding higher-education educators in Hong Kong to lead Communities of Practice (CoPs) both within and across institutions
– improve faculty teaching – benefit student learning (Handzic and Lagumdzija, 2010) – relatively new to Hong Kong’s higher education
– a cross-disciplinary group of faculty and staff members engaging in collaborative activities for enhancing teaching and learning (Cox, 2004)
Committee (UGC), matched with its own funds to establish Communities of Practice (CoPs).
Help HKBU students to think creatively, critically and positively
1) Development of the new 4-year degree - Academic Advising 2) Assessing learning outcomes - programme ILOs and University’s Graduate Attributes 3) Enhancing Teaching & Learning - online resources and eTools 4) Enabling students to achieve excellence – showcasing/ publishing quality student work 5) Establishing a community of scholars for interdisciplinarity
Establishment
UGC Supported Retreat on CoPs: learn about CoPs Successfully secured the start-up funding from the UGC University-wide Briefing on establishing CoPs Sharing of CoPs experience Workshop Series conducted by CoP consultant
establish CoPs Conducting evaluation to provide feedback about CoPs’ effectiveness
More details at http://chtl.hkbu.edu.hk/main/cops-funding/
CoP Topic
Community of Practice on Whole-person Education in Medical Services Development of Teaching Portfolio Framework though a Multidisciplinary Community of Practice Enhancing students’ Graduate Attribute (GAs) though Problem-based Learning and Service Learning in Formal Academic Courses Establishing a Community of Multidisciplinary Scholars to Support Education in Environmental Protection and Conservation in Hong Kong A community of Practice to Create a Model for Student e-portfolio as a Tool for Life-long Learning and Assessment Enhancing Student Learning through Holistic Mentoring Programme and Comprehensive Proficiency Test in Analytical Science
Hubball et al (2004) envisage that evaluation represents a key to the long-term success and sustainability of CoPs Development of a robust and assessment plan is crucial, to gather sound evidence for evaluation and comparison.
– Determinate whether the CoPs are successful, and how – Detect changes of faculty participants in their teaching – Ascertain influences on student learning
– Provide evidence – Ascertain students’ achievement of leaning outcomes - the Graduate Attributes (GAs) of HKBU – Compare with CoPs established in the US – Identify the strengths and weaknesses, and experience sharing
Evidence Collection
Faculties
(Changes in teaching & learning practices)
Students
(Representative assignments)
Faculties
(Questionnaire + Focus Group Interviews)
Students
(Survey + interviews)
Questionnaire
established;
participation in CoPs has enhanced their teaching, and help improve student learning;
items;
lowest);
understanding. Focus Group Interview
members who had completed the questionnaire;
questions developed;
group;
transcribed.
Categories
Number of Items
Measurement Scale (5= highest; 1 = lowest)
1
Your changes as a result of the CoP participation
13
Degree of Impact:
5 = a very substantial impact 1 = no impact
2
Changes in personal attitude that affected student learning
8
Degree of Agreement:
5 = strongly agree 1 = strongly disagree
3
Teaching and learning approaches implemented that resulted in changed student learning
19
Degree of Implementation:
5 = a very substantial amount 1 = not at all
4
Assessments used to judge student learning changes
21
Degree of Adoption:
5 = a very substantial amount 1 = not at all
5
Changes in student learning
CoP Project’s activities
31
Degree of Changes:
5 = a very substantial amount 1 = not at all
6
Overall
2
Degree of Changes:
5= a very substantial amount 1 = not at all
94
Questions 1. What overall changes have you experienced after your participation in the CoP? 2. What are the change(s) in your personal attitude that affected student learning, after your participation in the CoP? 3. What teaching and learning approach (es) have you implemented as the result
4. What assessment method(s) have you used to judge the changes in student learning? 5. What change(s) have you observed in student learning outcomes as the result
6. What is your overall experience from your participation in the CoP? Do you have any suggestions for better implementing your CoP project (e.g. operation and arrangements etc)?
Members
Responses Response Rate (%)^
CoP 1
10 10 100.0
CoP 2
11 1 9.1
CoP 3
9 9 100.0
CoP 4
8 6 75.0
CoP 5
12 5 41.7
CoP 6
10 6 60.0
^ As advised by our CoPs Consultant:
the analysis; therefore, only CoPs1, 3, 4 & 6 are included.
Response Rate
19% 55% 26%
Teaching Staff Academic Staff Administrator/Professional
Profile of Respondents for 4 CoPs
CoP 1
10 9
CoP 3
9 7
CoP 4
8 0*
CoP 6
10 7 * Focus group interview for CoP 4 was still on going at the time of preparing for this presentation.
(Can provide more than 1 answer)
8.0%
Designed/em ployed technology in a course AND Incorporated connections with professional
Total number of count = 125
10.4%
Designed guidelines for engaging in a process (e.g. discussion)
11.2%
Made class/ curriculum more inclusive
13.6%
Surveyed students to
information to incorporate into your teaching
Learned about and incorporated approaches to reach different learning styles
15.2%
Your Changes as a Result of the CoP Participation (Questionnaire)
*P-value in BLUE signifies that there is a significant difference at α=0.05 HKBU (CoP 1, 3, 4 and 6) Beach & Cox (2009) 6 universities in US (Table5)
Mean S.D. N Mean S.D. N P-value
aspects of higher education beyond discipline 3.58 0.76 26 3.93 1.11 369 0.0273
3.73 0.78 26 3.86 1.08 361 0.4276
teaching 3.67 1.09 24 3.80 1.14 368 0.5630
3.81 0.75 26 3.74 1.16 364 0.6706
community 3.75 0.84 28 3.55 1.18 374 0.2424
3.59 0.75 27 3.55 1.14 354 0.7851
experience 3.73 1.00 26 3.41 1.27 365 0.1230
3.33 0.92 24 3.38 1.46 341 0.8185
0.98 29 3.33 1.19 354 0.0165
class 3.72 0.84 29 3.29 1.20 352 0.0105
3.55 0.87 29 3.26 1.23 358 0.0946
teaching and learning 4.11 0.85 27 3.25 1.30 351 0.0000
discipline. 3.40 0.82 25 3.18 1.27 366 0.2128
Your Changes as a Result of the CoP Participation
(Focus group interview)
Quotations
“Via sharing with my students, I found necessary to help students foster their academic, career, and life development……” “I designed service learning in one of my courses as the result of participating in CoP, I am happy to know that my students like the course contents, they think what they have learned inside/outside classroom is useful ……” “I found that my students come from different regions with different learning, working, and life experience…… so I am implementing several teaching approaches to enhance their interactions……”
HKBU (CoP 1, 3, 4 & 6) Mean S.D. N
serve student learning needs 3.79 0.98 29
addressing student needs in/out of class 3.72 0.84 29
diversity influences/ enhances teaching and learning 4.11 0.85 27
Changes in Personal Attitude that Affected Student Learning
(Questionnaire)
HKBU (4 CoPs) Beach & Cox (2009) 6 universities in US (Table 10) Mean S.D. n Mean S.D. n P-value
4.19 0.68 27 4.11 0.81 263 0.5925
intellectual pursuit, i.e., scholarly teaching and the scholarship of teaching and learning 4.00 0.75 26 3.98 0.84 252 0.8981
4.32 0.69 25 3.96 0.89 263 0.0160
4.21 0.57 28 3.94 0.98 258 0.0271
4.00 0.78 27 3.85 0.98 259 0.3576
4.15 0.73 26 3.77 0.94 263 0.0137
4.08 0.78 24 3.76 0.95 263 0.0564
4.04 0.69 24 3.59 1.06 257 0.0040
*P-value in BLUE signifies that there is a significant difference at α=0.05
Changes in Personal Attitude that Affected Student Learning
(Focus group interview)
Quotations
“ I became more reflective on what I am teaching……. and how to teach systematically……” “I have become active and positive, and tried to use more teaching approaches and assessment methods.” “Because of joining this CoP, I have tired different methods and found ways to enhance student learning.” “I found most students respect me and they prefer me treat them strictly, they are willing to improve themselves.” HKBU (4 CoPs) Mean S.D. n
4.19 0.68 27
teaching and learning 4.32 0.69 25
4.21 0.57 28
and staff 4.15 0.73 26
Teaching and Learning Approaches Implemented that Resulted in Changed Student Learning (Questionnaire)
HKBU (4 CoPs) Beach & Cox (2009) 6 universities in US (Table 9) Mean S.D. n Mean S.D. n P-value
3.64 1.05 22 4.07 1.06 244 0.0646
3.77 0.87 22 3.99 1.05 236 0.2723
3.82 0.85 22 3.84 1.13 236 0.9115
3.77 0.87 22 3.84 1.12 238 0.7356
3.39 0.85 18 3.54 1.24 222 0.4867
4.00 1.14 21 3.49 1.27 203 0.0549
3.55 0.83 20 3.43 1.40 197 0.5680
3.45 0.76 20 3.34 1.37 209 0.5721
3.55 0.83 20 3.17 1.46 192 0.0753
3.89 0.83 18 3.15 1.35 179 0.0010
3.72 1.02 18 2.99 1.53 188 0.0062
3.47 1.19 15 2.98 1.40 189 0.1335
3.21 1.48 14 2.95 1.43 173 0.5194
3.43 1.40 14 2.82 1.54 158 0.1237
3.00 1.26 11 2.77 1.55 155 0.5672
2.82 1.33 11 2.70 1.51 161 0.7776
3.00 1.33 10 2.50 1.54 144 0.2584
2.75 1.36 12 2.13 1.42 136 0.1328
*P-value in BLUE signifies that there is a significant difference at α=0.05
Teaching and Learning Approaches Implemented that Resulted in Changed Student Learning (Focus group interview)
Quotations
“Students can teach and learn from each other in their own group.” “In our mentoring activity, students can discuss with teacher and classmates on different topics, including academic, and career development……” “…… I have implemented Problem-based Learning in one of my course as a result of joining the CoP….. which were more real life oriented……” “I have used case study method in my teaching, student can share their own cases and experiences……”
HKBU (4 CoPs) Mean S.D. n
collaborative learning 3.82 0.85 22
3.77 0.87 22
learning 4.00 1.14 21
3.89 0.83 18
Assessments Used to Judge Student Learning Changes (Questionnaire)
HKBU (4 CoPs) Beach & Cox (2009) 6 universities in US
(Table 8) Mean S.D. n Mean S.D. n P-value
3.37 1.12 19 3.58 1.25 252 0.4303
3.47 1.07 19 3.50 1.25 252 0.9190
3.06 0.75 17 3.46 1.31 252 0.0450
3.67 0.86 21 3.46 1.18 254 0.3048
3.70 0.86 20 3.38 1.08 253 0.1195
3.85 1.04 20 3.36 1.24 247 0.0470
3.71 1.01 21 3.30 1.20 253 0.0757
3.45 1.05 20 3.18 1.35 248 0.2810
3.65 0.99 20 3.15 1.22 253 0.0334
references 3.44 1.15 18 2.93 1.31 250 0.0704
classroom 3.90 1.00 21 2.87 1.40 249 0.0000
3.29 0.99 14 2.83 1.40 246 0.1053
3.24 0.90 17 2.76 1.42 247 0.0460
3.60 0.74 15 2.74 1.36 248 0.0001
3.40 0.74 15 2.71 1.40 247 0.0012
3.60 0.99 15 2.70 1.42 244 0.0010
3.40 0.99 15 2.67 1.40 248 0.0072
3.38 1.04 13 2.67 1.38 247 0.0189
3.60 1.18 15 2.52 1.39 244 0.0008
3.31 1.03 13 2.49 1.43 249 0.0069
3.25 1.91 8 2.24 1.44 237 0.1395
*P-value in BLUE signifies that there is a significant difference at α=0.05
Assessments Used to Judge Student Learning Changes
(Focus group interview)
Quotations
“Statistics show that lending of CoP books is much higher than other books.” “Student have developed abilities to think deeply and independently on clinical practices…… such as better writing of patient records.” “Students have applied what they had learnt not
their family members by what they have learnt, this is an outcome that I did not expect, it is a surprise!” “Students, who were resistant to us in the past, now become more positive……”
HKBU (4 CoPs)
Mean S.D. n
3.67 0.86 21
3.71 1.01 21
3.85 1.04 20
new/more learning objectives 3.65 0.99 20
connections with students outside the classroom 3.90 1.00 21
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Questionnaire)
HKBU (4 CoPs) Beach & Cox (2009) 6 universities in US (Table 7) Mean S.D. n Mean S.D. n P-value
3.57 1.16 23 3.50 1.17 233 0.7975
3.65 1.07 23 3.46 1.20 235 0.4174
3.65 1.23 23 3.44 1.21 237 0.4293
the parts 3.70 1.15 23 3.39 1.23 241 0.2256
discipline 3.64 1.09 22 3.39 1.21 228 0.3184
3.68 1.09 22 3.38 1.22 242 0.2181
3.65 1.11 23 3.37 1.19 244 0.2488
3.74 1.05 19 3.36 1.17 241 0.1352
learned to new problems and situations 3.63 1.12 19 3.35 1.15 236 0.2922
3.68 1.09 22 3.35 1.17 240 0.1743
3.70 1.02 23 3.32 1.13 234 0.0963
knowledge 3.65 0.99 20 3.31 1.13 231 0.1460
discipline 3.63 1.01 19 3.30 1.23 234 0.1782
3.48 1.12 21 3.28 1.18 242 0.4452
3.52 1.24 23 3.26 1.26 220 0.3366
discipline 3.53 1.02 19 3.24 1.20 221 0.2487
*P-value in BLUE signifies that there is a significant difference at α=0.05
HKBU (4 CoPs) Beach & Cox (2009) 6 universities in US (Table 7) Mean S.D. n Mean S.D. n P-value
3.68 0.95 19 3.22 1.18 233 0.0450
3.70 1.08 20 3.15 1.28 211 0.0336
3.45 1.06 22 3.14 1.23 219 0.1915
3.23 0.97 22 3.14 1.32 214 0.6999
3.45 0.96 22 3.12 1.26 198 0.1366
contemporary social issues 3.56 1.31 16 3.10 1.31 210 0.1763
3.05 1.13 19 3.00 1.27 220 0.8472
discipline 3.42 1.17 19 2.98 1.33 202 0.1221
cultures 3.38 1.20 16 2.98 1.35 210 0.2113
choices 3.39 1.24 18 2.97 1.40 205 0.1765
habits 3.45 1.15 20 2.96 1.21 211 0.0703
3.63 1.12 19 2.91 1.24 217 0.0079
3.28 1.18 18 2.82 1.32 197 0.1200
responsibilities of citizenship 3.53 1.31 19 2.72 1.36 194 0.0112
3.36 1.29 11 2.41 1.39 118 0.0211
*P-value in BLUE signifies that there is a significant difference at α=0.05
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Questionnaire) cont’d
Changes in Student Learning Outcomes as a Result of your CoP Project’s Activities (Focus group interview)
Quotations “…… students have improved their thinking pattern and become more mature when solving problems.” “Some students have read extra references and asked me questions…… I discussed with them about these questions, which I believe also beneficial for me as a teacher.” “…… the internship project can help students enhance knowledge and improve communication skills…… I think that it is an approach to whole person education.” “Student have applied the health-related knowledge which they had learnt to serve elders……” “To implement service learning, we have invited some elders (i.e. patients) to offer students internship
appreciated by these elders……”
HKBU (4 CoPs) Mean S.D. n
3.68 1.09 22
questions 3.48 1.12 21
terms and facts 3.63 1.12 19
commitment to exercise the rights and responsibilities of citizenship 3.53 1.31 19
experience 3.36 1.29 11
HKBU (4 CoPs)
Beach & Cox (2009) 6 universities in US Mean S.D. n Mean
changed as a result of your participation in CoP
3.47 1.01 30 3.29
changed as a result of changes in your personal attitude about teaching
3.43 1.00 28 3.10 *P-value cannot be compiled since there are no corresponding S.D. and number of responses provided from the paper.
Quotations
“Adoption of service learning made our teaching vivid, it is good for training students’ critical thinking…… the learning outcome(s) can be achieved.” “Through participating in CoP, I further transformed my requirements into more detailed steps and explicitly communicate them to students……. my students’ responses were positive because they knew how to achieve.” “…… students were offered with activities to cooperate, such as organisation and allocation of tasks ……”
Quotations
“Previously I thought that students should be self-motivated; after participating in CoP, I found that students preferred teachers to stimulate their learning, they are more willing to improve themselves……” “After joining CoP, I have become more sensitive…… I have unpacked my assessment components…… students’ responses were positive and they improved.” “Before CoP implemented, our mentoring approach was just to have meals with students; after joining this CoP, we spent more time in designing activities to help students enhance teamwork skills.”
the CoPs:
– General changes – Teaching attitudes – Teaching and learning approaches – Assessment methods
student learning
with the direct evidence from students’ assignments and faculties’ pedagogies and teaching practices.
continuing support to selected established CoPs and inviting further applications.
Selected References
1. Barr, R. B. & Tagg, J. (1995). From teaching to learning – a new paradigm for undergraduate education. Change (November/December), 13-25. 2. Beach, A. L., & Cox, M. D. (2009). The impact of faculty learning communities on teaching and learning. Learning Communities Journal, 1 (1), 7-27. 3. Cox, M. D. (1999) Peer consultation and faculty learning communities. In C. Knapper & Piccinin (Eds.), Using Consultation to Improve Teaching: New Directions for teaching and Learning (Vol. 79, pp. 39-49). San Francisco: Jossey-Bass. 4. Cox, M. D. (2001) Faculty learning communities: Change agent institutions into learning organizations. In D. Lieberman and C. Wehlberg (Eds.), To Improve the Academy (Vo;. 19, pp. 69-93). Bolton, MA: Anker Publishing. 5. Cox, M. D. (2004). Introduction to faculty learning communities. In M.D. Cox & L. Richlin (Eds.), Building Faculty Learning Communities: New Directions for Teaching and Learning (Vol. 97, pp. 5-23). San Francisco: Jossey- Bass. 6. Dees, D. M., Zavota, G., Emens, S., Harper, M., Kan, K. H., Niesz, T., Tu, T-H., Devine, M. A., & Hovhannisyan, G. (2009) Shift professional identities: reflections on a faculty learning community experience. Leaning Communities Journal, 1 (2), 49-73. 7. Handzic, M & lagumdzija, A. (2010). Global faculty learning community: as case study. Knowledge Management Research & Practice, 8, 89-98. 8.
9. Hubbal, H., Clarke, A., & Beach, A. L. (2004). Assessing faculty learning communities, in M. D. Cox & L. Richlin (Eds.), Building faculty learning communities: New Directions for Teaching and Learning (Vol 97, pp. 87-100). San Francisco: Jossey-Bass.
teaching, learning and curriculum practice, Journal on Excellence in College Teaching, 18 (2), 119-142.
through faculty learning communities. In M.D. Cox & L. Richlin (Eds.), Building Faculty Learning Communities: New direction for teaching and learning (Vol 97, pp. 127-135). San Francisco: Jossey-Bass.
NH: Heinemann.