Basis of reporting: Sky Sports Living for Sport Social Impact

Basis of reporting: Sky Sports Living for Sport Social Impact

Basis of reporting: Sky Sports Living for Sport Social Impact

Sky plc 01 Social impact | Basis of reporting Basis of reporting: Sky Sports Living for Sport Social Impact Metric: 1. Pre-and post-participation • Percentage of young people that rate at least one skill out of four higher after participating in Sky Sports Living for Sport compared with before. 2. Four to six months after participation • Percentage of young people who considered themselves more confident in applying at least one skill out of four higher, four to six months after participating in Sky Sports Living for Sport. 3. Teachers reporting skills improvement • The percentage of teachers reporting that their students showed improvements in skills as a result of Sky Sports Living for Sport.

Description Sky Sports Living for Sport (SSLFS) is a Sky Academy initiative that uses sport and mentors to build young people’s experience, skills and self-belief. Sky and Youth Sport Trust use a third party research agency to conduct annual research to evaluate the impact of Sky Sports Living for Sport in schools and identify areas for improvement. The survey data reflects young people’s responses and is based on their assessment of how the participants feel they improve in key areas. 1. Pre-and post-participation Calculation methodology Unit of measure The percentages of young people who rate at least one skill out of four higher after the initiative compared with before.

Scope The reported data comes from research conducted between January and June 2015 with young people who have participated in Sky Sports Living for Sport (SSLFS) projects in their school with a sample of 210 young people from 16 different schools. Young people completed a paper-based questionnaire in January, prior to the start of their SSLFS project. They completed a second questionnaire following completion of their project, between March and June. Change in skills levels are derived from the pre and post experience survey of young people taking part in Sky Sports Living for Sport. Young people were asked to rate four skills pre and post project: Teamwork; Communication; Resilience (defined for young people as ‘Keeping going when things get hard’); Planning.

The percentage of young people who rate themselves higher in at least one skill is then reported. Estimations The numbers reported are actuals, no estimations or assumptions have been made. Assumptions Students are asked to rate themselves on a five point scale (with 1 being ‘poor’ and 5 being ‘excellent’). We have assumed that a student who scores higher after the initiative compared with before is showing a positive improvement.

Sky plc 02 Social impact | Basis of reporting Data collection, verification and validation Data collection Data was collected and analysed by Chrysalis Research using a paper-based questionnaire sent to schools and filled out by young people before and after participating in Sky Sports Living for Sport. The data collection steps are as follows: 1. Young people from 39 schools selected at random were sent a paper-based questionnaire to complete in January 2015 prior to the start of their participation in a SSLFS project. Teachers provided young people with a unique ID which they enter on their questionnaire.

The teacher retained the log of young people’s IDs. Young people sealed their completed surveys in an envelope before handing to the teacher to collate and send to Omnisis, the data processing supplier.

2. The same young people completed a second paper questionnaire once their SSLFS project had ended i.e. from March – June 2015. Teachers provided young people with their unique ID which they filled in on their survey, which enables the link between the pre- and post-visit survey to be made. Again, young people sealed their completed surveys and the teacher collated them and sent the package to Omnisis. There were 210 matching pre and post questionnaires from 16 schools (as a result of some schools not completing the pre and post-surveys).

Verification and validation The data are collated and verified as follows: 1.

When questionnaires were received by Omnisis, they were booked in and a staff member entered the data. All data entry was verified to 20% - more than 5 single errors across one batch of 20 questionnaires would result in that batch being re-keyed. This means that 20% of the questionnaires were re-keyed afresh to verify the accuracy of the original data entry. If there is a mismatch between the original and the verification re-key then the whole batch is re- entered. This is the industry standard. Verification was performed by a batch being re-keyed using a special validation feature in the data entry software and was always performed by a different member of the team (in other words the person who does the original entry does not do the verification entry).

2. The data was processed on 15th – 24th June 2015 by Omnisis. They matched the surveys using the unique codes, and school IDs from the return envelopes. The data is validated by putting parameters on answers (limiting input to realistic responses, such as within the age range for the project) and cleaned by removing incomplete or non-matching responses (i.e. where there is no identifiable post survey answer to match a pre survey answer). Pre- and post-responses that do not match are discarded. Base sizes and totals are checked.

3. On the 24th June 2015, the data for these questions was downloaded into Excel by Omnisis and sent to Chrysalis Research who checked the base sizes and totals.

4. Responses to the pre- and post-visit surveys for young people’s rating of their skills in teamwork, planning, communication and resilience were analysed using spreadsheet formula and pivot tables. This gave a percentage of young people who had higher ratings for at least one skill. The final percentage was rounded to the nearest whole number and a report produced.

5. After the report was drafted, a researcher who was not involved in the analysis or reporting checked the data in the report against the source data – i.e. the Excel spreadsheet. This process involved ticking off each figure against the Excel figures and checking the formula. The numbers for the percentage that have improved their rating for at least one skill were then provided to Sky on the 1st July, along with the Excel sheets containing the calculations and the raw data. 2. Four to six months after participation Calculation methodology Unit of measure The percentage of young people who considered themselves more confident in applying at least one skill out of four higher four to six months after participating in Sky Sports Living for Sport.

Sky plc 03 Social impact | Basis of reporting Scope The reported data comes from research conducted between April and June 2015 with a sample of 225 young people who participated in Sky Sports Living for Sport projects in their school. Young people completed a paper-based questionnaire between April and June 2015, four to six months after the end of their SSLFS project. Confidence The change in the confidence of young people is derived from the percentage of young people who considered themselves more confident in applying one skill out of four. This was assessed by asking young people to indicate which of the following statements they agreed with: • I am better at communicating with others as a result of participating in Sky Sports Living for Sport • I am better at working on a project or task with people my own age as a result of participating in Sky Sports Living for Sport • I am better at planning an activity or a task as a result of participating in Sky Sports Living for Sport • I am more likely to see things through until the end as a result of participating in Sky Sports Living for Sport • I do not agree with any of these statements Estimations None.

Assumptions We have assumed that a young person who indicates that they agree with one of the statements about applying a skill better as a result of participating in SSLFS is showing greater confidence in using at least one skill out of four. Data collection, verification and validation Data collection Data was collected and analysed by Chrysalis Research using a paper-based questionnaire sent to schools and filled out by young people four to six months after participating in Sky Sports Living for Sport. Teachers were sent paper-based questionnaires for young people to complete between April and June 2015.

The questionnaires were sent at least four months after their SSLFS project had finished. Young people sealed their completed surveys in an envelope before handing to the teacher to collate and send to Omnisis, the data processing supplier.

Verification and validation The data was collated and verified as follows: 1. When questionnaires were received by Omnisis, they were booked in and a staff member entered the data. All data entry was verified to 20% - more than 5 single errors across one batch of 20 questionnaires would result in that batch being re-keyed. This means that 20% of the questionnaires were re-keyed afresh to verify the accuracy of the original data entry. If there was a mismatch between the original and the verification re-key then the whole batch was re- entered. This is the industry standard. Verification was performed by a batch being re-keyed using a special validation feature in the data entry software and was always performed by a different member of the team (in other words the person who does the original entry does not do the verification entry).

2. The data was processed in June 2015 by Omnisis. The data was validated by putting parameters on answers (limiting input to realistic responses, such as within the age range for the project) and cleaned by removing incomplete responses. Base sizes and totals were checked. 3. On the 1st July 2015, the data for these questions was downloaded into Excel by Omnisis and sent to Chrysalis Research who checked the base sizes and totals. 4. Responses were analysed using spreadsheet formula. This gave a percentage of young people who considered themselves more confident in applying at least one of the four skills.

The final percentage was rounded to the nearest whole number and a report produced.

Sky plc 04 Social impact | Basis of reporting 5. After the report was drafted, a researcher who was not involved in the analysis or reporting checked the data in the report against the source data – i.e. the Excel spreadsheet. This process involved ticking off each figure against the Excel figures and checking the formula. The numbers for the percentage of young people who considered themselves more confident in applying at least one of the four skills was provided to Sky by the 30th July 2015. 3.  The percentage of teachers reporting improvements in skills as a result of Sky Sports Living for Sport Calculation methodology The percentage of teachers reporting that their students showed improvements in skills as a result of Sky Sports Living for Sport.

Scope The reported data comes from research conducted between March and June 2015 with a sample of 482 teachers who led a SSLFS project. The lead teachers completed an online questionnaire between March and June 2015 at the end of their SSLFS project. Skills The percentage of teachers reporting that the young people showed improvements in skills as a result of Sky Sports Living for Sport is derived from the following question: To what extent did your students’ participation in Sky Sports Living for Sport help to develop the following life skills?

• Communication skills • Teamwork • Self-belief • Planning • Resilience (how well they cope with difficulties and problems) The percentage of teachers who selected communication skills, teamwork, self-belief, resilience and planning is then reported.

Estimations None. Data collection, verification and validation Data collection Data was collected and analysed by Chrysalis Research using an online questionnaire that was filled out by teachers after their SSLFS project was complete. Verification and validation The data was collated and verified as follows: 1. The data was collected using an online survey and processed by Omnisis. 2. On the 1st July 2015, the data for these questions was downloaded into Excel by Omnisis and sent to Chrysalis Research who checked the base sizes and totals.

3. Responses were analysed using spreadsheet formula. This gave a percentage of teachers reporting that their students showed improvements in skills as a result of Sky Sports Living for Sport. 4. The final percentage was rounded to the nearest whole number and a report produced. 5. After the report was drafted, a researcher who was not involved in the analysis or reporting checked the data in the report against the source data – i.e. the Excel spreadsheet. This process involved ticking off each figure against the Excel figures and checking the formula. The numbers was provided to Sky by the 30th July 2015.

Sky plc 05 Social impact | Basis of reporting Basis of reporting: Sky Academy Skills Studios Social Impact measurement Metric:  1. Pre-and post-participation • Percentage of young people who rated at least one skill out of four higher after participating in Sky Academy Skills Studios compared with before. 2. Four to six months after participation • Percentage of young people who considered themselves more confident in applying at least one skill out of four higher, four to six months after participating in Sky Academy Skills Studios. Description Sky Academy Skills Studios is a Sky Academy initiative that uses an interactive learning experience to build young people’s experience, skills and self-belief.

Sky use a third party research agency to conduct annual research to evaluate the impact of Sky Academy Skills Studios and identify areas for improvement. The survey data reflects young person responses and is based on their assessment of how the participants feel they improve in key areas. Calculation methodology Unit of measure Percentage of young people who rated at least one skill out of four higher after participating in Sky Academy Skills Studios compared with before. Scope The reported data came from research conducted in December 2014 to June 2015 with 1,062 young people who participated in Sky Academy Skills Studios in West London and Livingston.

Change in skills levels are derived from the pre and post experience survey of young people taking part in the Skills Studios experience. Young people were asked to rate four skills pre and post visit: Teamwork; Planning; Communication and Creativity.

The percentage of young people who rate themselves higher in at least one skill is then reported. Estimations None. Assumptions Young people are asked to rate themselves on a five point scale (with 1 being not very good and 5 being very good). We have assumed that a young person who scores higher after the initiative compared with before is showing a positive improvement. Data collection, verification and validation Data collection Data was collected by Chrysalis Research and administered by Skills Studios facilitators using an online survey administered via a tablet directly before and after the visit as follows.

1. Ten young people from each visit completed an online questionnaire on a tablet at the start of the visit. This was only not undertaken if the group was pushed for time or the young people were not able to complete the questionnaire due to their reading age. Young people were given a wristband with a number on it which they inputted into the survey. 2. The same young people completed a further online questionnaire on a tablet at the end of the visit. Young people inputted the number from the wristband, which enabled the link between the pre- and post-visit survey to be made. 1,062 young people completed the start and end of visit questionnaires.

Sky plc 06 Social impact | Basis of reporting Verification and validation The data was collected and collated by an external third party, Research Bods and verified by Chrysalis Research as follows: 1. The data was collected using an online survey and processed by Research Bods. They matched the surveys using the unique codes and timing of the survey completes. Pre- and post-responses that did not match in terms of their response to the gender and ethnicity questions were discarded, along with incomplete or non-matching survey responses (i.e. where there is no identifiable post survey to match a pre survey).

2. Responses to the pre- and post-visit surveys for young people’ rating of their teamwork, planning, communication and creativity skills were analysed. This gives a percentage of young people who have higher ratings for at least one skill. The final percentage was rounded to the nearest whole number. 3. The data for these questions was downloaded into Excel and the base sizes, totals and calculations checked by Chrysalis Research using spreadsheet formulae and pivot tables. A report is produced that includes the numbers for the percentage that have improved their rating for at least one skill.

4. After the report was drafted, a researcher who had not been involved in the analysis or reporting checked the data in the report against the source data – i.e. the Excel spreadsheet. This process involved ticking off each figure against the Excel figures and checking the formulae. The numbers for the percentage that have improved their rating for at least one skill were then provided to Sky, along with the Excel sheets containing the calculations and the raw data.

5. Sky then checked the reports by looking at the totals in the Excel spreadsheets. 2. Four to six months after participation Calculation methodology Unit of measure The percentage of young people who considered themselves more confident in applying at least one skill out of four higher four to six months after visiting Sky Academy Skills Studios. Scope The reported data comes from research conducted between April and June 2015 with a sample of 298 young people who visited Sky Academy Skills Studios. Young people completed a paper-based or online questionnaire between April and June 2015, four to six months after visiting Sky Academy Skills Studios.

Confidence The change in the confidence of young people is derived from the percentage of young people who considered themselves more confident in applying at least one skill out of four. This was assessed by asking young people to indicate which of the following statements they agreed with: • I am better at communicating with others after visiting the Sky Academy Skills Studios • I am better at working on projects and tasks with people of my own age since visiting Sky Academy Skills Studios • I have improved how I plan activities or tasks as a result of attending Sky Academy Skills Studios • I am better at presenting my own ideas as a result of attending Sky Academy Skills Studios • I do not agree with any of these statements Estimations None.

Assumptions We have assumed that a young person who indicates that they agree with one of the statements about applying a skill better as a result of attending Sky Academy Skills Studios is showing greater confidence in using at least one skill out of four.

Sky plc 07 Social impact | Basis of reporting Data collection, verification and validation Data collection Data was collected and analysed by Chrysalis Research using a paper-based questionnaire sent to schools and filled out by young people four to six months after participating in Sky Academy Skills Studios.

An online questionnaire was also used to collect data for schools who preferred to use this approach. Teachers were either sent paper-based questionnaires or a link to an online survey for young people to complete between April and June 2015. The questionnaires were sent at least four months after their visit to Sky Academy Skills Studios. Young people either sealed their completed surveys in an envelope before handing to the teacher to collate and send to Omnisis, the data processing supplier or completed the online survey on their own.

Verification and validation The data was collated and verified as follows: Paper questionnnaires 1. When questionnaires were received by Omnisis, they were booked in and a staff member entered the data. All data entry was verified to 20% - more than 5 single errors across one batch of 20 questionnaires would result in that batch being re-keyed. This means that 20% of the questionnaires were re-keyed afresh to verify the accuracy of the original data entry. If there was a mismatch between the original and the verification re-key then the whole batch was re- entered. This is the industry standard.

Verification was performed by a batch being re-keyed using a special validation feature in the data entry software and was always performed by a different member of the team (in other words the person who does the original entry does not do the verification entry).

2. The data was processed in June 2015 by Omnisis. The data was validated by putting parameters on answers (limiting input to realistic responses, such as within the age range for the project) and cleaned by removing incomplete responses. Base sizes and totals were checked. Online questionnaires 3. The data was collected using an online survey and processed by Omnisis. For data from paper and online questionnaires 4. On the 1st July 2015, the data for these questions was downloaded into Excel by Omnisis and sent to Chrysalis Research who checked the base sizes and totals.

5. Responses were analysed using spreadsheet formula.

This gave a percentage of young people who considered themselves more confident in applying at least one of the four skills. The final percentage was rounded to the nearest whole number and a report produced. 6. After the report was drafted, a researcher who was not involved in the analysis or reporting checked the data in the report against the source data – i.e. the Excel spreadsheet. This process involved ticking off each figure against the Excel figures and checking the formula. The numbers for the percentage of young people who considered themselves more confident in applying at least one of the four skills was provided to Sky by the 30th July 2015.

Basis of reporting: Sky Academy Careers Lab Social Impact Metric:  1. Pre-and post-participation • Percentage of young people who rated at least one skill out of four higher after participating in Sky Academy Careers Lab compared with before. 2. Four to six months after participation • Percentage of young people who considered themselves more confident in applying at least one or more skills out of four higher, four to six months after participating in Sky Academy Careers Lab. Description Sky Academy Careers Lab is a Sky Academy initiative comprising a full day work-related experience to build young people’s experience, skills and self-belief, and bridge the gap between Sky Academy Skills Studios and Starting Out.

You can also read