about Reports Peter Findlay (QAA) Point of departure Reports are - - PowerPoint PPT Presentation

Some Thoughts about Reports

Peter Findlay (QAA)

Point of departure

• Reports are a form of message
• Reports follow the general rules which govern

all messages

• Communication theory applies here
• We need to think about the communication

process

• It is not simple

Structure of presentation

The sender (panel, Agency)

• Context
• Purpose
• Writing

The message (The report)

• Form
• (Language)
• Content

The receiver (HEI, stakeholders)

• Context
• Reading
• Understanding,

Actions

‘Encoding’

• Encoding means

– Knowing the purpose of the message – Knowing what you need to write about – Knowing how to write it – Knowing who will read it – Knowing how they will read it

‘Decoding’

• Decoding means

– Knowing how to get the message – Knowing how to read it – Understanding it correctly – Knowing what you should do when you have read it

NOISE

‘Noise’ in communication – ‘stuff happens’

All the surrounding events, activities, experiences that take place at the time of the message

• E.g. – the compiler of the report is going to

retire in one month’s time

• E.g – there is an election taking place with new

higher education policies ahead

• E.g - the institution is in financial trouble

Encoding and Decoding

• Choice of language and tone
• Common understanding of the code used (i.e.

language, terminology and expressions

• The level of understanding of the code used

will differ between reader groups

• The code has to be learned
• The easiest commonly understood

code is the simplest language

The report is the outcome of the process

• Agency policy and method
• Agency officer and review team
• Documentation and information
• Visit and confirmation etc
• Drafting the report
• Editing the report
• Finalising the report
• Completing the report is the AIM of all parts
• f the process

Sending the Message

Stages in producing the report

Building the team for the project

The visit and the meetings

Gather the information…..

Doing the analysis

….and now for the report…is it that simple?

Report production - writing

Drafting Compiling Checking Editing Consulting Finalising Publishing

Not that much fun??

And now for the report……it can sometime feel like this!

Challenges in report production

• Individual task, highly responsible
• Weak contributing drafts from panel
• Wrong form and content from panel
• Judgements don’t relate to the main body
• Inadequate evidence-base to support

evaluation

• Getting the language right….
• Unsure about exact purpose
• Unsure about exact readership
• Time management and Deadlines….

Yes – deadlines…….!!

Expectations: Who will read the whole report right through?

1 10 25 2000 5000 50 100 500 1000

How many people do you think should read the whole report (ideally)? How many people do you think will actually really read the whole report?

Who ‘owns’ the report?

• The expert panel?
• The member of the panel who writes it?
• The Agency officer who writes it?
• The Agency committee that approves it?
• The Agency as a legal entity?
• The institution?
• - Can you own part of the report?
• What are the implications?-

The message

Reports – the key considerations

• Length
• Structure
• Content
• Style - language
• Tone

Ideals

• Length – as short as possible
• Structure – as clearly defined as possible
• Style – simple, precise, clear, focused
• Tone – neutral, measured, objective
• Content – minimum description, maximum

analysis and evaluation

• No need to describe detailed procedures

Detailed description? WHO FOR?

Style - virtues

• Well structured paragraphs ‘beginning,

middle, end’

• Sentences which are simple and clear (never

longer than two lines of text!)

• Arguments clearly based on evidence
• Point to relevance to the review

questions/agenda/ framework

• Regular summaries of each section

Style: - avoid

• Over-complex sentences
• Jargon and cliché ‘cutting edge’ ‘pushing the envelope’

etd

• Emotive or strongly judgemental words (‘wrong’

‘chronic’ ‘harmful’ ‘bad’ ‘impressive’ ‘outstanding’)

• Obscure words (‘inchoate’ ‘serendipitous’)
• Ambiguity
• Too much unnecessary detail
• Personality
• Too much reference to the process (the review visit etc)

The jargon of quality

• ‘the quality management system’
• ‘PDCA cycle’
• ‘key performance indicators’
• ‘ feedback loops’
• ‘quantitative data analysis’
• ‘action planning’
• ‘student learning opportunities’
• ‘Constructive alignment’

==Who needs it??

Good words ….QA favoured vocabulary

• confidence
• responsibility
• accountable
• reliability
• soundness
• effective

The receiver (s)

‘Transparency’ for whom?

• The communication of reports is controlled by

the format and language used

• Transparency for one set of readers may be a

limiting device for other readers

• Language can deliberately be used to target

specific reader groups and exclude others

• Is it possible to communicate with all possible

reader groups?

Language – soft or hard?

Diplomatic: “the university may like to consider the advisability of conducting regular student surveys” = Blunt: “The university must introduce student surveys immediately’

Use of Coded Language

• “the university may like to consider the

advisability of conducting regular student surveys”

• “further strengthen the existing

arrangements” = you have not done this properly yet

• Introduce more regular reviews = it’s not

planned

Babel or ‘Euroglish’ Q-Speak?

OR – IS IT ONE TONGUE , MANY VOICES?

?

Language politics

• National languages of small nations don’t have

same impact or comparability (but stay in the culture)

• Translation into the lingua franca (English)

brings risks of mistranslation, confusion, misunderstanding

• We don’t realise how varied our

understanding of key terms in English can be

• Many Englishes …..(US-UK- and others)

Beware! You can’t always be sure what English means to the reader!

Shared English? – Differing readings, Confusions and Misunderstandings…(2006 ENQA WORKSHOP)

• Standards
• Programme
• Accreditation
• Validation
• Experts
• Evaluation
• Audit
• Assessment
• Enhancement

Assessment

Definitions

• ‘the evaluation or estimation of the nature,

quality, or ability of someone or something’ (OED) A general term that embraces all methods used to judge the performance of an individual, group

• r organisation.

(INQAAHE)

Assessment - students

• Assessment in UK HE usage more

generally refers to the assessment of someone i.e. the student

Assessment is a generic term for a set of processes that measure the outcomes of students' learning, in terms of knowledge acquired, understanding developed, and skills

• gained. It serves many purposes. Assessment

provides the means by which students are graded, passed or fail.

(AUQA)

Some terms in student assessment

• Assessment strategy
• Alignment of assessment
• Continuous assessment
• Formative assessment
• Summative assessment
• Self-assessment
• Peer assessment
• Group assessment

Assessment provides information about levels of academic achievement and performance, and is therefore closely related to the determination of academic standards

Institution and programme assessment

CHEA: Assessment A diagnostic form of quality review and evaluation of teaching, learning, and programs based on a detailed examination of curricula, structure, and effectiveness of the institution, its internal review, and quality control mechanisms.\ Bologna/Bergen: Assessment The process of systematic gathering, quantifying and using information to judge the effectiveness and adequacy of a higher education institution or a programme. It implies evaluation of core

• activities. It is a necessary basis for a formal accreditation decision.

UK usage The assessment of an institution or programme is more usually described in

• ther terms in UK HE e.g.:
• Programme review
• Course evaluation
• Institutional audit
• Institutional review

but there are also exceptions….to this rule!

The research assessment exercise (RAE) Teaching quality assessment (TQA)

Aim of the message

What do your report readers need?

• Understanding of complexity?
• Clarity of message?
• Brevity of message?
• Simple signals?

How much attention?

Alternatives to the ‘40 page narrative’

• A checklist approach?
• Nothing but the recommendations?
• Short summary for the public (published) plus

detailed Annex for the institution (private)?

• Agreed Action Plan? – future orientation?

Is a checklist approach the answer?

Language and style of a summary section

Conclusions and recommendations should:

• Be brief and clear
• Signal a pattern of priority (significance,

urgency)

• Always refer back to points made in the body
• f the report
• Should not include arguments in themselves
• Be rooted in recorded evidence

Summaries

• A summary or a variety of summaries must be

essential for the reader of a long report

• Executive summary
• Key background informatino
• Conclusions and recommendations
• Not at the end!

Time for lunch …

But if all this has got you thinking – don’t worry, here’s one solution ….(but you have to understand the code)

Final advice…….