UNIVERSITY OF MALTA L - Universita` ta Malta Validation and - - PowerPoint PPT Presentation

UNIVERSITY OF MALTA

L - Universita` ta’ Malta

Validation and Evaluation of Teaching at University of Malta

Quality Assurance (QA): Why the need?

The University of Malta is responsible for ensuring:

• Academic standards and quality

(being the main institution providing higher education on the island);

• That courses are in response to the country’s needs
• That courses are in response to the country’s needs

(being a state-funded organisation);

• That the quality of its programmes and operating standards

are comparable to those of our European (and international) partners (as befits a university working within the Bologna Process).

The Quality Assurance Cycle

External Quality Assurance Internal Quality Assurance Assurance UoM courses Assurance

Internal quality assurance practices

• 1. External examiner system
• 2. Two-stage validation process for new programmes
• 3. Study-unit evaluation of teaching
• 4. Periodic programme review

External examiners

• Peer review where it matters most
• EE are involved in all high stakes

assessments including final written or practical examinations, postgrad practical examinations, postgrad dissertations and doctoral theses

Validation of New Programmes Programmes

Approval Route for New Programmes of Study

Stage 1 (In-principle) Approval

Senate Programme Validation Committee Programme Originators Committee Finance Office

Academic Programme Quality and Resources Unit

APQRU

Approval Route for New Programmes of Study (cont.)

Stage 2 Approval

Senate Programme Validation Committee External Peer Review Programme Originators Finance Office APQRU Faculty Board

Study-Unit Evaluation (“student feedback”)

Why perform feedback?

To ensure that: 1. Students have opportunity to comment on the quality of their courses as required in preparation for and as part of review processes; 2. Academic products on offer are meeting the expectations 2. Academic products on offer are meeting the expectations

• f students;

3. Lecturers are given information that may assist them in improving delivery and/or content of the study-unit.

The Study-Unit Feedback Exercise

• Held via an on-line system on a twice-yearly basis,

at the end of each semester (February and June);

• Feedback is collected after students have been assessed on

that particular unit but prior to publication of results;

• Lecturers do not have access to feedback prior to publication
• f assessment results.
• Study-units are selected for feedback on a cyclical basis, so

that over a 4/5-year period, all study-units would have been reviewed at some point.

Study-unit Evaluation Form

The form includes:

• General questions on study-unit
• Comparison between study-unit description and

actual delivery actual delivery

• Lecturing methodology
• Lecturer attributes
• Method of assessment
• Administration and resources
• Any additional comments

Response Rates

Response rates were as follows: June 2008 38.24% February 2009 47.90% June 2009 38.20% February 2010 37.70% June 2010 28.5% February 2011 35.96% June 2011 30.2% February 2012 37.15% June 2012 33.4% February 2013 37.6% June 2013 33.43%

What Action is Taken?

The results of feedback are communicated to:

• 1. Lecturer/s of the study-units reviewed – to alert about

any problems and to provide opportunity for self- evaluation and improvement;

• 2. Heads of Departments & Directors of Institutes/Centres –

to allow for detection and consideration of any emergent trends concerning departmental teaching responsibilities.

What Action is Taken? (cont.)

• 3. Rector & Pro-Rector for Academic Affairs – any urgent or

serious matters are discussed with Heads of Department, and where appropriate, situations are monitored closely during subsequent exercises. 4. Students (under construction) 5. Relevant authorities (Registrar, Director of Library 5. Relevant authorities (Registrar, Director of Library Services, etc) are advised to address any problems as identified from feedback.

Periodic Programme Review

Why perform Periodic Programme Review (PPR)?

• To ensure relevance, appropriateness and utility of what is

taught at University in relation to the development of the various subject domains; the local economy; the necessity of remaining a relevant University in a global context; global context;

• To

identify examples

• f

good practice for wider dissemination and to detect and modify areas that need improvement;

• To

demonstrate to external reviewers and significant stakeholders that University deploys its teaching provision in a transparent and accountable manner.

Main characteristics of PPR

• Faculty-led process;
• On-going;
• Uses already-available and purposely generated
• Uses already-available and purposely generated

new information;

• Involves significant stakeholders;
• Has effective feedback loops that can affect quality
• f outcomes in teaching and learning.

TEACHING AND LEARNING AT UNIVERSITY OF MALTA

QUALITY ASSURANCE Audit

Collecting information

Review

Analyse the information

Plan

Design an action plan

Implement

Bring about changes according to plan

Typically once every 4 or 5 years depending

• n duration of programme and after the

When to conduct a PPR

• n duration of programme and after the

programme has been completely delivered.

Process of PPR

• 1. Faculty sets up an ad hoc Sub-Committee to carry
• ut the review and identifies main stakeholders;
• 2. Faculty establishes the criteria and standards

against which programme performance is to be against which programme performance is to be assessed;

• 3. Ad hoc sub-committee and dean meet with Pro-

Rector to agree a plan of action and ensure resources are provided for the purpose;

Principal outcome from the ad hoc sub-committee’s work

Self-Evaluation Document (SED) (including supporting evidence) should:

• be evaluative rather than descriptive
• demonstrate in a constructive and self-
• demonstrate in a constructive and self-

critical manner how a Board of Studies has reflected on the operation of the programme since initial approval or last review

• describe implementation plans to maintain

academic standards and enhance the quality

• f the student experience

Data required for the Self-Evaluation Document

Annual Review Report Form (for use by Boards of Studies)

Action points addressed from the previous Annual Programme Review (APR) Actions taken as a result of previous APR (please list) Actions which are currently in process as a result of previous APR (please list) and timescale projected for each action Outcomes of student feedback Study-unit feedback Course Experience Feedback Strengths identified Weaknesses identified Reports of External Examiners Strengths identified Weaknesses identified Issues for concern and actions planned as a result Weaknesses identified Issues for concern and actions planned as a result Issues raised by student representatives during meetings of the Board of Studies Substantive issues raised by students Were any actions or decisions agreed as a result of discussions with students? (please list)

Admissions Student admission numbers against recruitment targets Ratio of applicants to entrants Was there an increase or decrease in the number of applicants and if so by how much from the previous year? Student profile: Age of students (i.e. range and average age) Gender of students Full-time/part-time status Students declaring a disability Performance Indicators Progression and attrition rates for previous academic year Student achievement: Student achievement: Number of re-sits Degree classes awarded Graduate destinations: What sort of employment do graduates of the programme enter? What proportion of graduates is unemployed? What proportion of students proceeds to further studies? Staff feedback on Study-units What were the major points to be considered from the review of study-units? What major changes were processed during the previous academic year (please list and refer to study-unit code)?

Changes to the Programme of Study What changes were effected to the Programme of study in the past year, if any? Please indicate in brackets whether these were major or minor changes What were the factors which brought about these changes? Resources implications arising from the requirements of the Programme of Study Other issues

Self Evaluation Document

• Considers and evaluates, inter alia, following

parameters aims of programme; the learning outcomes; the curriculum; the curriculum; the assessment methods and their fitness for purpose; student progression and academic support; learning resources and effectiveness; stakeholders’ perceptions and attitudes to programme, where stakeholders include current students, alumni and academic staff.

Self Evaluation Document

Executive summary including the concluding assessment of merit and/or worth of programme and the main recommendations for future enhancement of programme enhancement of programme

FACULTY STAKEHOLDERS’ COMMITTEE

advises specific steps appoints submits SED

AD HOC SUB-COMMITTEE (to draw up Self Evaluation Doc)

Appendix I COMMITTEE (containing external agents)

informs

SENATE (through PVC)

responds and/or actions and then informs

Stakeholders’ Committee (SC)

• Dean of Faculty (or Director of Institute)
• Two deans or directors who are external to the

faculty

• Member of staff of a department involved in the

programme being reviewed

• Two external experts who are either academics,
• Two external experts who are either academics,

clinical or professional experts (where appropriate)

• r employers with expertise in area of programme
• Two student representatives of students on course
• Pro-Rector for Students and Institutional Affairs
• Pro-Rector for Academic Affairs

Decision of Stakeholders’ Committee

• relayed to Faculty through Pro-Rector (Academic

Affairs)

• may include:

commendations (areas identified as good practice) recommendations (for actions intended to improve programme to be taken within a practical time period) conditions (for the taking of steps and measures intended to be addressed in a short timeframe as they may be impinging in a serious manner on the quality of the programme)

FACULTY STAKEHOLDERS’ COMMITTEE

advises specific steps appoints submits SED

AD HOC SUB-COMMITTEE (to draw up Self Evaluation Doc)

Appendix I COMMITTEE (containing external agents)

informs

SENATE (through PVC)

responds and/or actions and then informs

FACULTY STAKEHOLDERS’ COMMITTEE

advises specific steps appoints submits SED

AD HOC SUB-COMMITTEE (to draw up Self Evaluation Doc)

Appendix I COMMITTEE (containing external agents)

informs

SENATE (through PVC)

responds and/or actions and then informs

This action closes the loop of the PPR process

Thank you.