A Web Based Module A Web Based Module Evaluation System Evaluation - - PowerPoint PPT Presentation

▶
a web based module a web based module evaluation system
SMART_READER_LITE
LIVE PREVIEW

A Web Based Module A Web Based Module Evaluation System Evaluation - - PowerPoint PPT Presentation

Mar 03, 2023 230 likes •406 views

A Web Based Module A Web Based Module Evaluation System Evaluation System 1 , Nikki Rochford 1 Nadine Foster 1 , Nikki Rochford 1 Nadine Foster 2 , Stephen Bostock 3 Barry Smalley 2 , Stephen Bostock 3 Barry Smalley Department of

slide-1
SLIDE 1

A Web Based Module A Web Based Module Evaluation System Evaluation System

Nadine Foster Nadine Foster 1

1, Nikki Rochford

, Nikki Rochford 1

1

Barry Smalley Barry Smalley 2

2, Stephen Bostock

, Stephen Bostock 3

3

Department of Physiotherapy Studies Department of Physiotherapy Studies1

1,

, School of Chemistry and Physics School of Chemistry and Physics2

2, Staff Development and Training

, Staff Development and Training3

3

slide-2
SLIDE 2

Objectives of the project Objectives of the project

  • develop a standard module evaluation form

develop a standard module evaluation form for the BSc ( for the BSc (Hons Hons) Physiotherapy ) Physiotherapy

  • develop a standard course evaluation form

develop a standard course evaluation form

  • further develop the supporting software to

further develop the supporting software to analyse and present web analyse and present web-

  • based summary

based summary data data

  • investigate feasibility of use across campus

investigate feasibility of use across campus

slide-3
SLIDE 3

Background Background

  • Present mechanism of module evaluation

Present mechanism of module evaluation ! ! Relies on paper questionnaires Relies on paper questionnaires ! ! Effective Effective

  • Problems

Problems ! ! Time for analysis and production of data Time for analysis and production of data ! ! Questionable cost Questionable cost-

  • effectiveness

effectiveness ! ! Different forms, so not easily compared Different forms, so not easily compared

slide-4
SLIDE 4

QuestMan QuestMan

Quest Questionnaire ionnaire Man Manager ager

1.

  • 1. Production of computer based forms

Production of computer based forms

2.

  • 2. Students complete the forms

Students complete the forms

3.

  • 3. Data is emailed to special

Data is emailed to special Questman Questman account account

4.

  • 4. Collation in a hierarchical directory structure

Collation in a hierarchical directory structure

5.

  • 5. Responses are made anonymous

Responses are made anonymous

6.

  • 6. Production of results for module leaders

Production of results for module leaders

! ! Web pages of analysis and comments Web pages of analysis and comments ! ! Produces “instant” results, when required Produces “instant” results, when required

slide-5
SLIDE 5

Example web pages Example web pages

  • Menu

Menu

  • A module evaluation form

A module evaluation form

  • The results

The results ! ! Data Data ! ! Summary Summary ! ! Comments Comments ! ! Data for Excel Data for Excel

slide-6
SLIDE 6

Students Evaluation Students Evaluation

  • To develop the new evaluation form

To develop the new evaluation form

! ! Small focus group with students Small focus group with students ! ! Pilot of new evaluation form with 1 module Pilot of new evaluation form with 1 module

  • To compare computer with paper system

To compare computer with paper system

! ! Web or paper for 5 module evaluations Web or paper for 5 module evaluations ! ! Random allocation for comparison Random allocation for comparison ! ! Year 1 and 2 Year 1 and 2

slide-7
SLIDE 7

Q1 Prefer paper or web

0% 10% 20% 30% 40% 50% 60% 70% paper either web web paper

slide-8
SLIDE 8

Q2.Was form design helpful?

0% 10% 20% 30% 40% 50% 60% yes maybe no web paper

slide-9
SLIDE 9

Did the medium make a difference?

0% 20% 40% 60% 80% 100% yes maybe no web paper

slide-10
SLIDE 10

Q2.4/3.3 Happy to give username?

0% 20% 40% 60% 80% 100% yes maybe no web paper

slide-11
SLIDE 11

Q2.3 Need a PC lab session? (web users)

0% 10% 20% 30% 40% 50% 60% 70% yes maybe no

slide-12
SLIDE 12

Staff Evaluation Staff Evaluation

  • Qualitative information

Qualitative information

  • Staff preference

Staff preference ! ! Preferred new evaluation form Preferred new evaluation form ! ! 2 staff preferred computer system 2 staff preferred computer system ! ! 3 stated no preference 3 stated no preference

  • Benefits

Benefits ! ! Time Time ! ! Ease of access of results Ease of access of results

  • Ideas for improvement

Ideas for improvement

slide-13
SLIDE 13

Were the evaluation data different Were the evaluation data different from web and paper forms? from web and paper forms?

Computerised Computerised Paper Paper-

  • based

based Mode Mode No of students No of students Mode Mode No of students No of students giving a low score, giving a low score, giving a low score giving a low score per question per question per question per question Module 1 Module 1 4 4 2.7 (range 0 2.7 (range 0-

  • 22)

22) 4 4 6.7 (range 0 6.7 (range 0-

  • 31)

31) Module 2 Module 2 4 4 7.1 (range 0 7.1 (range 0-

  • 25)

25) 4 4 5.5 (range 0 5.5 (range 0-

  • 22)

22) Module 3 Module 3 4 4 2.5 (range 0 2.5 (range 0-

  • 20)

20) 4 4 3.5 (range 0 3.5 (range 0-

  • 18)

18) Module 4 Module 4 4 4 8.5 (range 0 8.5 (range 0-

  • 22)

22) 4 4 9.7 (range 3 9.7 (range 3-

  • 17)

17) Module 5 Module 5 4 4 5.1 (range 0 5.1 (range 0-

  • 14)

14) 3 3 7.7 (range 1 7.7 (range 1-

  • 15)

15)

slide-14
SLIDE 14

Were the response rates different Were the response rates different for web and paper forms? for web and paper forms?

Number of students completing the evaluations (n=76) Number of students completing the evaluations (n=76)

Web Web % % Paper Paper % % (should have been 38 responses ) (should have been 38 responses ) Module 1 Module 1 36 36 32 32 Module 2 Module 2 38 (100%) 38 (100%) 33 33 Module 3 Module 3 no data no data no data no data Module 4 Module 4 36 36 17 (45%) 17 (45%) Module 5 Module 5 27 27 25 25 Minimum response rate 45% (paper Minimum response rate 45% (paper-

  • based)

based) Maximum response rate 100% (web form) Maximum response rate 100% (web form)

slide-15
SLIDE 15

Further developments Further developments

  • Standard web form now being used for all

Standard web form now being used for all physiotherapy modules physiotherapy modules ! ! Years 1, 2 and 3 Years 1, 2 and 3

  • Standard course evaluation questionnaire

Standard course evaluation questionnaire developed and to be piloted end of May 2002 developed and to be piloted end of May 2002 ! ! Year 3 Year 3

  • Providing summary data to fit with

Providing summary data to fit with requirements of the annual course reports requirements of the annual course reports

slide-16
SLIDE 16

Conclusions Conclusions

  • Standard web form

Standard web form ! ! acceptable to staff and students acceptable to staff and students

  • Summary data

Summary data -

  • easily interpreted

easily interpreted

  • Feasible for wide use if

Feasible for wide use if

! ! Students have access Students have access ! ! Simple means provided to set up a new questionnaires Simple means provided to set up a new questionnaires