Browsing Centre for Educational Research and Innovation by Subjects
Now showing items 1-3 of 3
Evaluation of outreach interventions for under 16 year olds: Tools and guidance for higher education providers.During 2017-18, OFFA commissioned research that aimed to understand the nature of outreach activities for under 16 year olds (which were funded through access and participation investment) and how these were evaluated. This document, developed from the research, is intended to act as a resource for pre-16 outreach practitioners and evaluators, drawing both on the data collected by this project and the wider literature around evaluation and outreach. It seeks to recognise the complexity of pre-16 outreach work and eschews a prescriptive approach in favour of establishing important principles and actions that are likely to underpin good practice. Our discussion is broadly positioned within a ‘social realist’ worldview (Archer, 2008; Pawson, 2013) that seeks to understand the fuzzy nature of the cause-and-effect relationships that exist within complex social fields, where individuals construct their own realities in reference to those around them. There is a particular focus on epistemology – the pathways to creating dependable, if contingent, knowledge – as a vehicle for making meaning from data that is usually incomplete, compromised or mediated through young people’s emergent constructions of their worlds. Fundamentally, outreach is predicated on the ability of practitioners to influence young people in a planned way, albeit that the plan will not always work for every young person in every cohort. An important element in this epistemology is that it is not concerned with finding single ‘solutions’ that exist outside time and context. Rather, it is concerned with understanding how young people are influenced by their life experiences – not ‘what works’, but what works in a given context and, importantly, why. It is only through understanding the latter element that practices can become robustly effective in the long-term and potentially transferable to other contexts. This is particularly appropriate to pre-16 outreach work due to the lengthy time lag between activity and application to higher education (HE).
An evaluation of the Derbyshire and Nottinghamshire collaborative outreach programmeIn the East Midlands the NCOP consortia is the Derbyshire and Nottinghamshire Collaborative Outreach Programme (DANCOP) which is a progression of NEMCON (North East Midlands Collaborative Outreach Network) and is comprised from several universities and colleges of further education. DANCOP’s initial two goals were: 1. Raise learners’ motivation to work hard and their understanding of the importance of education in their future: 2. Equip learners to plan for progression and make appropriate choices for post-16 study and HE. This report includes an extensive review of literature on widening access, collaboration and networks and details a formative and summative evaluation undertaken by The International Centre for Guidance Studies (iCeGS) using data collected from February 2017 to May 2019. It reports on the progress made by DANCOP up until April 2019 with respect to: 1. The development of the collaborative network 2. The extent to which schools and learners have been engaged 3. The perceived impact of activities on learners, feedback from learners, teachers and parents and distance travelled with respect to knowledge/attitudes/intentions pertaining to future options and in particular higher education 4. Innovations in collaborative working and widening access The evaluation has captured data from surveys, interviews and focus groups from DANCOP team members, management group members, learners, teaching and school staff, session deliverers and third party providers. KEY FINDINGS 1. The network is well established amongst the HEIs, external stakeholders and FE colleges 2. DANCOP has surpassed its targets with respect to school engagement and learner interactions 3. It took a long time to establish the central and hub teams and recruit college based roles, partly because of the policies and processes inherent in HEIs and FECs but also because of non-competitive salaries and short term contracts. 4. It took a long time to build awareness in schools and develop good working relationships so that activities could be delivered. In short term funded programmes this is a problem. 5. DANCOP could work more quickly if legal, recruitment and financial issues and executive sign off could be facilitated. 6. Collaborative work has been supported by: a. Representation of key partners across different management groups b. The structural and physical location of teams and individuals c. An agile Steering Group and inclusion of further education colleges through the IPG d. ‘Blended Professionals’ who have significant experience, knowledge and skills and are able to cross boundaries to get work done 7. Key innovations have been the IPG, a small but agile steering group and using funding for longer term resources such as skills study coaches in colleges and the STEM Centre. 8. With respect to activities, feedback has been almost entirely positive, and this includes learners, teachers, DANCOP staff funded roles and parents. This has been the case across the wide range of different activities which have been delivered, across year groups and across delivery teams. 9. Activity evaluations show participants report significant increases in knowledge about HE and confidence. 10. Comparisons of knowledge, attitudes and intentions between DANCOP and non DANCOP learners suggest there have been the desired changes in DANCOP learners. They have shown increased likelihood of attending FE and HE, increased sense of academic fit, increased confidence and increased knowledge of HE.
Understanding the evaluation of access and participation outreach interventions for under 16 year olds.The project team was asked to address the following six research questions and these were used to guide the project: 1. What are the intended outcomes for current outreach interventions directed at under 16 year olds from disadvantaged backgrounds where the long-term aim is to widen access to higher education (HE)? 2. What types of outreach intervention activity or activities are institutions using in relation to intended outcomes? 3. What evaluation tools, methods and metrics are being used to measure the intended outcomes? 4. What are the perceived and actual challenges and barriers for different stakeholders to effective evaluation of long-term outreach? 5. What do different stakeholders consider most effective evaluation practice and why? 6. How valid and suitable are the evaluation tools, methods and metrics (identified through the research) that are commonly used? The project was constructed around six interlinked work packages: 1. A quantitative analysis of what higher education providers (HEPs) say about their pre-16 outreach activities (and their evaluation) in their 2017-18 access agreements (as the most recent available). 2. An online survey of HEPs to gather information about the pre-16 outreach activities delivered during the 2016-17 academic year and their evaluation, as well as the structure of their evaluation resources and challenges faced. 3. Case studies of four HEPs identified as demonstrating elements of good practice through their access agreements and the online survey, derived from telephone interviews with key staff and documentary analysis. 4. Telephone interviews with 11 third sector organisations (TSOs) to explore their practices and the evaluation of their activities, providing a counterpoint to the data collected from higher education institutions (HEIs). 5. A synthesis of the four preceding work packages to explore elements of good practice, determine a basis for assessing the quality of evaluations and highlight challenges for the sector and OFFA. 6. An invited participatory workshop for evaluators from HEPs and TSOs identified as demonstrating elements of good practice through the online survey and telephone interviews, to act as a sounding board for the emerging conclusions and recommendations.