@EQI_DCU Title of Presentation: Inspections of area-based learning - - PowerPoint PPT Presentation

Title of Presentation: Inspections of area-based learning communities in Northern Ireland Conference: Accountability of educational networks; Findings from a comparative EU-study on inspections in polycentric education systems Presenting:

• Dr. Martin Brown, Professor Joe O'Hara

@EQI_DCU

𝑷𝒗𝒖𝒎𝒋𝒐𝒇

• 1. Introduction
• 2. Towards a framework for

Polycentric Evaluation in Education

• 3. Polycentric Evaluation – The

case of West Belfast

• 4. Moving Forward
• 5. Conclusion and a Global

message Inspections of area-based learning communities in Northern Ireland

• EQI is a research centre specialising in the evaluation of educational policies, programmes and
• personnel. Evaluation involves research to make data informed judgments about organisational

and professional performance.

• Our work encompasses schools, other learning centres and the wider public service. It is

primarily concerned with governance and accountability mechanisms including quality assurance processes, school inspection and organisational self evaluation. We also undertake evaluations of educational programmes and projects.

• EQI is a multidisciplinary research group with a diverse membership drawn from within DCU, from

schools and other educational institutions within Ireland and abroad and from a range of other

• rganisations.

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − Who we are

Inspections of area-based learning communities in Northern Ireland

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − What we do

Inspections of area-based learning communities in Northern Ireland

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − What we do

Inspections of area-based learning communities in Northern Ireland

• Core Theory of Polycentric evaluation: When schools reach a certain evaluation threshold, they

can only further improvement not merely through a combination of school self-evaluation and pressure from external inspection but rather, through collaboration between clusters of schools, communities and the inspectorate.

• Northern Ireland: many schools are in the process of asking the question, ‘How do we as

practitioner researchers improve the quality of education not only in our schools but also in our communities’? (Brown, 2011)

• Along with other countries: This requires research into polycentric network evaluations

(Although there is a plethora of research on evaluation and inspection, this is not the case with polycentric networks)

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − 𝐶𝑏𝑑𝑙𝑕𝑠𝑝𝑣𝑜𝑒

Inspections of area-based learning communities in Northern Ireland

• Professed benefits to being part of an educational network: improved learning; the efficient

use of resources; increased innovation capacity; and system-wide improvement, etc. (Chapman and Hadfield, 2010; Provan and Kenis, 2008)

• However, in comparison to single unit inspections, there is very little, if any research relating

to the impact or for that matter, potential impact of inspection on networks of schools in Northern Ireland. Research on the effects of school inspection on the island of Ireland have primarily focussed on evaluation of individual schools. (McNamara and O’Hara, 2012, 2013, Brown, et al, 2014, Brown, 2013)

Inspections of area-based learning communities in Northern Ireland

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − 𝐶𝑏𝑑𝑙𝑕𝑠𝑝𝑣𝑜𝑒

• Difficult to isolate ‘the causal effect of network structure’ (Siciliano, 2012. p.2) in

comparison to single unit inspections.

• Reluctance among many who study networks to discuss formal mechanisms of control.

A common assumption is that, since networks are collaborative arrangements; governance, which implies hierarchy and control, is inappropriate.. (Provan and Kenis 2008, p.230).

Inspections of area-based learning communities in Northern Ireland

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 - Why the lack of research in the field?

𝑱𝒐𝒖𝒔𝒑𝒆𝒗𝒅𝒖𝒋𝒑𝒐 − Why the lack of research in the field?

• There are no studies of the failure rate of networks, either in education or in any other

sector (de Lima, 2010, p.17); and yet, there is constant drive to establish school to school networking becoming an integral part of educational practice.

• There is nothing inherently positive or negative about a network: it can be flexible and
• rganic, or rigid and bureaucratic; it can be liberating and empowering, or stifling and

inhibiting; it can be democratic, but it may also be dominated by particular interests. (de Lima 2010)

Inspections of area-based learning communities in Northern Ireland

Towards a framework for Polycentric Evaluation in Education

Serendipitous networks Interactions among group members (that is, ‘networks evolve haphazardly from the interactions

• f individual actors, without guidance from any central network agent (de Lima, 2010, p. 11).

Goal directed networks All relations between network members are structured in order to achieve network-level goals; an administrative entity plans and coordinates the activities of the network as a whole

Polycentric inspection networks

You’re really asking schools don’t forget here to move from a culture of competition to a culture

• f cooperation. My own view of it is that you don’t move from competition to cooperation. You

have to evolve a new construct (Inspector participant in Brown, 2013, p.124)

Inspections of area-based learning communities in Northern Ireland

School Inspection – The case of Northern Ireland

Inspections conducted by the ETI, whose mission statement is ‘promoting improvement’, utilise a number of inspection modes across the different phases/sectors of the education system of Northern Ireland. Although different inspection frameworks exist, in most cases inspection is focused on individual schools. As stated in ETI (2014: 1), ‘the work of ETI focuses mostly on the inspection of, and reporting on, the overall effectiveness

• f single organisations such as schools, colleges, training and
• ther providers’.

However, in the case of networks in the form of ALC’s a different inspection model (area based inspection) has been tested by the ETI. Inspections of area-based learning communities in Northern Ireland

Area Inspection – The case of West Belfast

Data collection, dissemination and analysis (2014 – 2016) Document Analysis Single unit, Full area and youth inspections since 2005 Parametric / non Parametric Analysis Data relating to terminal examination results, Free Schools Meals entitlement, the number of individuals claiming benefits and also, the destination of students following GCE and A Level education. Interviews and Focus Groups lead inspector who carried out the Area inspection; the education manager for the West-Belfast partnership board; the education officer in the Belfast Education Library Board, School Principals and Teachers.

Inspections of area-based learning communities in Northern Ireland

Background information − West Belfast

• One of the highest levels of people claiming unemployment benefit in Northern

Ireland (NI)

• The highest proportion of people (76%) living in the most deprived Super Output

Areas of NI

• Ranks first on the NI Multiple Deprivation Measure (WBP, 2014, p.6)
• A significant number of the student population are entitled to Free school meals. 35%

at Post-primary level and 60% at primary level.

Inspections of area-based learning communities in Northern Ireland

Background information − West Belfast

An important question of course is whether it can be shown that activities such as polycentric inspection of networks which improve cooperation and provide professional development opportunities for teachers do, in fact, lead to, for many - improved student performance.

WBALC PERFORMANCE 2009/10 TO 2013/14 DE FIGURES GCSE 2010 2011 2012 2013 2014 2015 Percentage achieving 5 or more at grades A*-C 79.2 80.8 84.5 85.7 91.9 91.1 Percentage achieving 7 or more at grades A*-C 60.0 63.6 69.5 70.5 80.2 79.7 A Level 2010 2011 2012 2013 2014 Percentage achieving 2 or more at grades A*-E 92.3 89.9 93.3 91.1 93.8 94.0 Percentage achieving 3 or more at grades A*-C 50.2 51.1 55.0 58.1 66.8 66.8

Inspections of area-based learning communities in Northern Ireland

Centrality of the Inspectorate

The centrality of regular contact with and follow-up by the Inspectorate was emphasised by all parties. I’m not sure without the ETI and the District Inspector that we would have gotten as far as we had gotten in any shape or form because the District Inspector has moulded and put in shape and direction for how we have taken ownership of the findings’.

Inspections of area-based learning communities in Northern Ireland

Centrality of the Inspectorate

• It was widely noted by respondents that the ETI places great emphasis on self-

evaluation and in response, the network and the individual schools within it have sought to develop self-evaluation capacity.

• The appropriate role of external inspection then becomes the quality assurance of the

self-evaluation and data generation processes within a network.

Inspections of area-based learning communities in Northern Ireland

Centrality of the Inspectorate

There is a clear understanding that after a period of time you need to say that this is what we’ve done and this is the evidence of the impact of what we’ve done on the

• ground. Now the ‘first-hand evidence’ that a community group produces would be

different to the evidence that a school produces. And the role of the inspectorate is that the inspectorate can quality assure all of this.

Inspections of area-based learning communities in Northern Ireland

Trust and Collaboration

It was stressed time and time again that the major success of the process, by far, was the gradual growth of trust and collaboration between the network member organisations. The key issue if we peal everything away is relationships and trust. In that context the schools have left something aside which is part of the culture of education which is school against school… Instead of talking about building schools or closing schools, the other way of looking at it is to ensure that we’re accessing the full range of the curriculum open to everyone. How do you do it? You do it through a network of schools.

Inspections of area-based learning communities in Northern Ireland

Cohesion and co−operation

The network and polycentric evaluation act as a catalyst or glue that has led to greater cohesion, communication and cooperation between the different partners. Traditionally we would have been working out of silos in our own school. I keep on going back to the point that it is our community, they are our pupils, and we need to start looking at it from that perspective. I see it as the beginning of a process of looking at our pupils as opposed to our individual school and I think that that’s a process that’s long overdue and it will also begin the debate around what a school effectively is.

Inspections of area-based learning communities in Northern Ireland

Polycentric Inspection – The case of West Belfast

Data collection and analysis – Phase 2 (2015-2017) Regional Survey Survey of Principals and Teachers attitudes towards Polycentric Evaluation in West Belfast Interviews and Case Studies Principals and Teachers in West Belfast Dissemination of initial findings Inspectorate, CCMS, Members of ALC, West Belfast Partnership Board, Belfast Education Library Board, Principals, Teachers.

Inspections of area-based learning communities in Northern Ireland

Towards a Framework for Polycentric Inspection Key Questions

• 1. Stage 1: What are the priorities of the Network?
• 2. Stage 2: Who should be involved in setting standards and

Methodologies used to evaluate the network?

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network

Priorities such as Evaluation and Planning, Curriculum Development and Assessment, Peer Learning and Support and Equity of Access are concerned, for the most part as being either high priority or essential priorities of the network (Tables 1, 2, 3 and 6).

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network – Evaluation and Planning

Not a priority Low priority Medium priority High priority Essential To jointly evaluate an aspect of educational provision (e.g. literacy) in the area. 0.00% 7.14% 7.14% 21.43% 64.29% To jointly develop an improvement plan for an aspect

• f educational provision (e.g.

literacy) in the area. 0.00% 7.14% 7.14% 42.86% 42.86% Table 1: Priorities of the Polycentric Network: Evaluation and Planning

High priority or essential priorities of the network

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network – Evaluation and Planning

Collectively by being part of the network we can also share CPD initiatives. For example, enhancing middle management was something we wanted to improve in our school. So we invited a member of ETI in to talk about Middle Management, and we networked. Our Literacy, numeracy, SEN coordinators sat together, and part of it was how do you do this, what does it look like, and that was a springboard for where we are now because we realised that there’s so much excellent practice around us and about us? High priority or essential priorities of the network

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network – Curriculum Development and Assessment

Not a priority Low priority Medium priority High priority Essential To jointly develop and offer curricula 0.00% 7.14% 7.14% 57.14% 28.57% To develop joint assessments 0.00% 7.69% 53.85% 30.77% 7.69% Table 2: Priorities of the Polycentric Network – Curriculum Development and Assessment

High priority or essential priorities of the network

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

‘The biggest and most positive development from my perspective was the creating of links between primary and post primary. Primary and Post Primary Principals are represented

• n each others committees and this ultimately facilitates the progression of children’.

‘We created a transition pro-forma which was great as a way of making us all aware of the information that we needed to get to make the transition easier’.

Initial Findings: Priorities of the Network – Curriculum Development and Assessment

High priority or essential priorities of the network

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network

While there was strong support for collaborative learning and joint work on teacher professional development there was also evidence of limitations in co-operation and a degree of surviving competiveness. Medium to low priorities of the network

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network - Peer Learning and Support

Medium to low priorities of the network

Not a priority Low priority Medium priority High priority Essential To ensure education

• rganisations learn from each
• ther

0.00% 7.14% 7.14% 21.43% 64.29% To jointly develop and provide continuous professional development for staff (e.g. teachers) 0.00% 7.14% 7.14% 35.71% 50.00% To ensure that strong education

• rganisations support weak

education organisations in improving 0.00% 14.29% 50.00% 28.57% 7.14% Table 3: Priorities of the Polycentric Network - Peer Learning and Support

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network – Infrastructural Resources

Medium to low priorities of the network

Not a priority Low priority Medium priority High priority Essential To share resources: IT 0.00% 35.71% 42.86% 21.43% 0.00% To share school buildings and school grounds 0.00% 42.86% 42.86% 14.29% 0.00% Table 4: Priorities of the Polycentric Network: Infrastructural Resources

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network – Human Resources

Medium to low priorities of the network

Not a priority Low priority Medium priority High priority Essential To share community workers and support services (e.g. school psychologist) 7.14% 14.29% 21.43% 50.00% 7.14% To share staff (e.g. teachers) 7.14% 28.57% 42.86% 14.29% 7.14% To jointly develop and provide initial teacher training 0.00% 42.86% 50.00% 0.00% 7.14% To share resources: HR services 0.00% 46.15% 46.15% 7.69% 0.00% To share financial services (e.g. one financial department and contracting for all schools) 0.00% 42.86% 42.86% 14.29% 0.00% Table 5: Priorities of the Polycentric Network: Human Resources

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Initial Findings: Priorities of the Network

Medium to low priorities of the network Not all sweetness and light, still a ‘healthy’ competition between schools You have to be conscious of not going into areas that might be problematic. We concentrated on areas where we could develop things that made sense for all of

• us. For example we created a process to work with classroom assistants to upskill

them – this worked for everyone as there were economies of scale for us working together to provide training

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

To no extent To a very little extent To some extent To a great extent To a very great extent The inspectorate 7.14% 7.14% 35.71% 35.71% 14.29% Head teachers 0.00% 7.14% 42.86% 42.86% 7.14% Representative of teachers 0.00% 7.14% 42.86% 50.00% 0.00% Governing bodies of individual schools 0.00% 7.14% 50.00% 42.86% 0.00% Representative of parents 0.00% 35.71% 35.71% 28.57% 0.00% Representative of students 0.00% 28.57% 35.71% 35.71% 0.00% Governing bodies of networks 0.00% 42.86% 50.00% 7.14% 0.00% Table 7: To what extent should the following stakeholders have a say in the development of evaluation methodologies.

Evaluation Methodologies Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

To no extent To a very little extent To some extent To a great extent To a very great extent Inspectorate 7.14% 7.14% 25.71% 45.71% 14.29% Head teachers 0.00% 7.14% 42.86% 28.57% 21.43% Teachers 0.00% 21.43% 42.86% 28.57% 7.14% Parents 0.00% 15.38% 69.23% 15.38% 0.00% Students 0.00% 28.57% 57.14% 14.29% 0.00% Governing bodies of individual schools 0.00% 14.29% 50.00% 35.71% 0.00% Governing bodies of networks 0.00% 21.43% 42.86% 35.71% 0.00% Table 8: To what extent should the following stakeholders have a say in the development of evaluation standards .

Evaluation Standards Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Discussion

The benefits of Area inspection and follow up polycentric School Self Evaluation were perceived as many. However, there may be considerable constraints on the level of co-

• peration possible.

Nonetheless there was widespread agreement that a key role and success of the network was to work together to promote educational equity and inclusion across the area.

Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

• In the context of wider debates about the role and function of inspection it is

noteworthy that the role accorded to the inspectorate in this case is one of support and development rather than accountability.

• However it should be noted that while this network has a statutory basis and

gets public money, involvement of schools and other organisations is voluntary which certainly limits the accountability aspect of inspection of the network.

• Also of interest in this regard is the emphasis which the Northern Ireland

Inspectorate itself places on self-evaluation in the schools of the network as being the basis on which inspection of the network rests.

Discussion Towards a Framework for Polycentric Inspection

Inspections of area-based learning communities in Northern Ireland

Moving Forward

Our main task now is to see if this model of evaluation can be applied to communities in

• ther Jurisdictions.

Inspections of area-based learning communities in Northern Ireland

Moving Forward

We also intend to look at the often silent voice of those who are on the receiving end of this mode of evaluation governance – Namely Parents, Students and other members of the school community.

• Do they feel that this approach to evaluation

has had an effect on quality educational provision in their area?

• To what extent are their opinions and their

plans actually making an impact and are actually being taken seriously.

Towards Equal Education Opportunity in a Polycentric Inspection Context

Conclusion and a Global Message

There is probably a role in every jurisdiction for area-based evaluation examining the boundaries between institutions to try to solve problems around student progression, transfer, etc. However, the process in West Belfast goes well beyond that. There, community development partnership structures, which exist independently of the Inspectorate, are central to polycentric inspection being an iterative process in which the Inspectorate is an agent of change, constantly interacting with the network. In summary it seems that the involvement, in a very structured way, of multiple stakeholders is a core requirement in ensuring that polycentric inspection will have a significant impact on the quality of education provided in an area.

Inspections of area-based learning communities in Northern Ireland

• If the next phase of the research turns out to be as positive as West Belfast, we will

certainly be able to emerge from this project saying that this is a very interesting model of evaluation that could be much more widely applied in the future because it has many advantages in the types that were previously outlined.

• An absolute key finding is that it gets schools and other community education

providers and other members of the community working together.

• It also has the potential to takes a considerable deal of resource pressure off

inspection procedures and inspection systems because Schools become more self- governing and self-motivating. A SHARED EDUCATION

Inspections of area-based learning communities in Northern Ireland

Conclusion and a Global Message

Thank you