theory to application Judy Cohen** & Catherine Robinson* **UELT - - PowerPoint PPT Presentation

The UK’s European university

Team-Based Learning in Practice: Moving from theory to application

Judy Cohen** & Catherine Robinson* **UELT *Kent Business School Chatham, Kent

Thursday 4 May 2017, 10am -1.30pm, Medway Campus, Rochester R2.09

Introduction

• Firstly, an apology….
• I’m here to provide a practical take on implementation

in Kent

• What led us to TBL as an instrument of change?
• What were the challenges faced?
• What were/are the results (so far)?
• Next steps?

The Scenario: (Macro) Economics for Business

• Stage 2 students
• Compulsory for A&M, optional for B&M
• Increased cohort
• 35 in 2014
• 76 in 2015
• 109 in 2016
• ‘Under-performance’
• Low lecture attendance levels (as low as 20%)
• Poor engagement in seminars (attendance monitored)
• Relatively weak assessment scores (mean ~57%)
• Satisfaction scores acceptable (4.2/5)

CB363 – TBL in context Page 3

Module Learning objectives

• Demonstrate knowledge of the underlying economic concepts, competing

theories and basic principles of macroeconomics.

• Evaluate the effect of the changing/prevailing macroeconomic environment on

business decisions, behaviour and performance.

• Interpret macroeconomic data and develop well-structured lines of economic

argument, offering critical comment on the arguments of others.

• Demonstrate knowledge of the importance of economics in understanding

current business issues in the UK, Europe and global economies.

• Retrieve information from a variety of sources.
• Undertake independent and self-managed learning.
• Draw on social science concepts and theories in decision-making situations.
• Demonstrate ability to communicate information, ideas and solutions

effectively.

Module LOs Page 4

How did we do it?

• 110 students split into teams of 6/7
• Named after famous economists
• Lecture slots used to test the students’ readiness and the

individual scores used as assessments

• Continual assessment at 5 points during the 12 week course
• Highest 3 scores were taken as their MCQ mark (worth 10% in

total)

• This was administered using Turningpoint responseware

devices, registered to individual students.

• 5 MCQs at the start of the hour, followed by breakout time into

teams….

Implementation Page 5

Readiness Assurance Process (2)

• Students were then asked to sit in their teams and discuss

the same 5 questions, reaching agreement on their answer

– Required team debates, decision making, rationalising the chosen answer within the team – Answer revealed via scratchcards – Resulted in immediate feedback from peers and generated continued discussion on more contentious answers – Team results stored and used to award an end of module prize

• Opportunity to challenge

– Students were able to nominate an individual to challenge the answer to any of the iRATs but

RAPs Page 6

Final component of the lecture hour….

• A recap and correction to any misinterpretation
• informed in part by the questions
• Extension into the application of the key

concepts

• Summary of the resources available

Page 7

The Application Exercise

Page 8

• Each team was presented with

a pack containing:

• A Question Sheet
• A menu Stand
• A pack of 5 MCQ letters (A to E)
• Teams were given 20 minutes

to answer the question

• Simultaneous reveal essential
• Each team tasked with

justifying their answer

• NO CHANGING YOUR MIND!

Examples….

• Applications need to meet the 4S criteria (Sibley and

Ostafichuk (2014)

• Significant; Same; Specific; Simultaneous
• Some worked well:
• The more structured, shorter questions
• The ones with RIGHT answers
• Some not so well:

Which of the following is likely to have the biggest impact on structural unemployment? Prepare to explain your answer. a) Growth in the third sector (the Big Society) b) Growth in grammar schools c) Growing global inequality d) Growth in Artificial Intelligence e) An aging population

Page 9

Challenges faced: INSTITUTIONAL

• square pegs, round holes:
• Module constraints
• Assessment patterns fixed and not easily amended
• Timetable constraints
• Traditional Lecture – seminar format
• Students can change seminar slots up to 3 weeks into term
• Rooming constraints
• all KBS lectures are in RHDC and seminar rooms are laid out in rows
• Technological constraints

– Moodle-turningpoint interface

• Led to disjointed delivery of the complete TBL cycle
• Teams were not as cohesive
• Administration was time consuming

Challenges Page 10

Challenges faced: TBL IMPLEMENTATION

• Student expectations (herding cats):
• Student shuffling teams
• Honesty
• The RAP was inevitably quite complicated
• Split across two single hours meant the TBL cycle

was not complete in one sitting

• Small seminars actually worked against creating a

competitive environment

• The compilation of the data was tremendously

painstaking

• Peer review
• How to incorporate it?
• What was it for?

Page 11

The Royal Historic Dockyard Church

Teaching Environment Page 12

The Royal Historic Dockyard Church

Teaching Environment 2 Page 13

So, how did it progress?

• Generally students seemed receptive to the idea of TBL
• Early feedback indicated they were not liking TBL:
• ‘I am not paying £9,000 to teach myself’
• Turningpoint devices were meant to have a plugin to the

moodle site.

• Data had to be extracted and managed using Excel
• Students were using multiple turningpoint devices to

register attendance

– corrected for by taking a register by hand but all of this involved extra work

Implementation of TBL Requires….

• Selling it to the students
• ORGANISATION!
• Online materials

– Topic books containing slides, articles, videos – Readiness Checkers

• Scratchcards

– For the t-RATs

• Seminar materials

– Menu stands, laminated letters

• Feedback

– (at a variety of places through a number of mechanisms)

• Applications

– Applied questions that required greater thought on how to use the material – Not always a right answer – the rationale and reasoning was important

• Peer evaluation

Page 15

Results – Attendance

• Did it work?
• Elements of it, certainly, despite teething troubles

Results Page 16

N=109 (2016) and 76(2015) 88% home students in 2016 compared with 83% in 2015 Around 38% female students in both cohorts

Accounting and management students made up a higher proportion of students in 2015 (around 70% compared to around 50% in 2016) – not shown here Lecture attendance this year averaged 72% (comparative figures for 2015 not available, but it was not uncommon to have less than 10)

Performance

Performance Page 17

Final exam results not yet available

Regression analysis controls for gender, POS, fee and part time status, as well as seminar attendance

Preliminary multivariate results

OLS regressions Page 18

VARIABLES mark cw_mcq cw_ict fees (Home student=1) 0.472 1.538 1.75 (0.568) (2.66) (2.282) Gender (female=1) 0.522

• 0.295

2.554 (0.425) (1.975) (1.704) year (2016=1) 1.378*** 14.36***

• 0.644

(0.427) (1.972) (1.692) sem_attend (%) 6.131*** 19.06*** 20.52*** (1.028) (4.749) (4.1) deg_type (A&M) 0.173 2.397

• 0.569

(0.422) (1.953) (1.682) Constant 13.37*** 48.55*** 43.48*** (1.047) (4.803) (4.148) Observations 175 179 177 R-squared 0.219 0.286 0.144 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1

Any evidence of ‘success’?

• Well, the MCQ marks are higher, but this was

an incentive to get them to play.

• Attendance was necessarily up, but need to

see if this is evidenced through improved performance!

• The ICT marks were the same as last year, but

a subtly different question so hard to evaluate this as an achievement of the TBL method.

• What we can say is that module achievement was

consistent

Student evaluation – what did they think?

• Interim survey
• Module evaluation (end)
• Focus groups
• Teaching quality survey (end of year)

Student evaluation Page 20

Overall student evaluation

MEQ Page 21

..This year Last year..

Focus group feedback

Viewpoint A:

• The delivery of the lectures were (sic) very informative and well structured.
• This module was taught differently which I enjoyed. I enjoyed the team working element which promoted

and encouraged learning and participation.

• I think that I like the structure, because it actually helps me to understand things, because last year I

didn’t really do that well at economics, and this year I’ve actually improved a lot. And I think it’s basically the structure, like preparing before the lecture, then you come in and have the lecture, then the seminar, and then the seminar is based on that lecture that you had. So it just helped me to clarify things, and when I go home I just study, go back over my notes, and then I get a better understanding, rather than wait for a whole week. Viewpoint B:

• Never studied economics before, so layout of this module where you read at home then come in and do

test with no lectures was difficult for me. Difficult to understand terminology.

• The teaching method changed for this module. I personally did not like it because it was not very

productive and felt like i was wasting my time.

• I just think sometimes with the lectures, I would prefer someone to teach at least some of it, as opposed

to me learning it all home by myself, especially when they use technical stuff, technical models, technical frameworks, technical words. If we’d done economics and we’re a bit better on it, so that’s not so much of a problem, but then there are some people who didn’t do economics and English isn’t their first language, so then they get confused.

Page 22

Teaching quality survey

• Of both the TBL cohort and stage 1s doing micro
• Looking at confidence in the subject area:

Are you feeling confident about the exam? Page 23

Stage 1 Stage 2

Conceptual framework of teaching excellence

Perception of teaching excellence

Staff

Students

Implicit assumptions based on sophisticated T&L epistemic belief systems Implicit assumptions based on learner knowledge and experience

Quantitative: performance, demographic, attendance, survey Qualitative: focus groups, survey, student evaluation of module Successful student engagement Didactive/reproductive facilitative/transformative

Teaching and learning environment Evaluation

Perceptual framework of excellence

Other good ideas that cropped up through the process: Co-curricula development

• Feed-forward seminars within the TBL
• Students asked to bring an essay plan for the ICT question

and evaluate their teams plans according to the KBS Generic Marking Criteria (grid)

• Led to discussions around the GMC – was it reflective of

what was required by the assessment?

• To meet learning outcomes
• To reflect the task
• To make sense to students!
• New marking criteria developed and used to evaluate a

plan for the SECOND of the two ICT questions

• Weightings to reflect relative importance of elements to the essay
• Inclusion of separate criteria in relation to economic theory and

use of diagrams.

What I’d do differently…

• Spend more time on preparing the resources
• Students need more guidance on the logic of the order of the

material

• Get the teams working sooner
• limited by the administrative struggles of students shifting seminar

groups because of work patterns

• Work harder at engaging students with the applications
• Quite often the students railed against the application – struggled

to accept there wasn’t a right answer; felt a few individuals dominated; were reluctant to speak out in class

– How to engender a culture change?

• Consider using paper (but computer marked) MCQs
• Consider doing away with seminars BUT bringing seminar

leaders in to the large group session to help facilitate

Lessons Learnt

• Read the books!!!
• Ideally have a two-hour slot
• That is well resourced with a few members of staff to act as

facilitators

• Prepare your facilitators appropriately
• Spend time explaining all elements of TBL
• Especially peer review
• Not be so reluctant to ask for help!

What to do differently Page 27

THE UK’S EUROPEAN UNIVERSITY

www.kent.ac.uk