Preliminary Findings for the "Winning New Jobs" Programme in Ireland - Centre for Health Promotion Studies The National University of Ireland, Galway
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Preliminary Findings for the
“Winning New Jobs” Programme
in Ireland
January 2005
Centre for Health Promotion Studies
The National University of Ireland, GalwayACKNOWLEDGEMENTS
This report was prepared by Dr. Margaret Barry, who developed and directed the overall
programme evaluation, Colette Reynolds who collected and inputted the evaluation data,
and Róisín Egenton who analysed and reported on the data analysis.
The authors would like to express their sincere thanks to the following:
• The unemployed individuals who participated in the Winning New Jobs programme,
and all those who completed baseline and follow-up questionnaires.
• All the training and employment agencies, health services and community groups
who participated in the Winning New Jobs Planning Group, and who facilitated the
programme evaluation.
• The WNJ programme trainers for their support in administering the evaluation
measures and providing valuable feedback for the adaptation and improvement of
this programme.
• Dr. Jukka Vuori, who has been extensively involved in implementing and evaluating
the JOBS programme in Finland, and Professor Richard Price from the Michigan
Prevention Research Centre at the University of Michigan, who was involved in
developing the original programme. Both contributed to the initial planning as well
as the evaluation of the programme in an Irish context.
• Ms. Mary Duggan and Ms. Therese Lowry, Project Managers with the Rural Health
and Social Well-being Project, and Ms. Ann Marie Nedeljkovic and Ms. Inga Bock,
Project Managers with the Rural Health Mind Matters Project for their assistance and
support in undertaking the programme evaluation.
• The Northern Health and Social Services Board and the Health Promotion
Departments of the North Western Health Board and Homefirst Community Trust
for their support of this initiative.
• The research staff at the Centre for Health Promotion Studies at NUI, Galway for
their help with data collection and support throughout the project. In particular,
thanks to Dr. Saoirse Nic Gabhainn for her advice on data analysis.
The implementation of the WNJ project was funded by the European Union under the
Programme for Peace and Reconciliation 2000 – 2004 (Peace 2) and part-financed by the
UK and Irish Governments. The evaluation of this initiative was funded by the Health
Promotion Unit of the Department of Health and Children, Dublin. We gratefully
acknowledge the support of both funders.
1TABLE OF CONTENTS
ACKNOWLEDGEMENTS 1
I. SUMMARY 3
II. INTRODUCTION 6
III. BACKGROUND 7
IV. CONSULTATION AND PLANNING 10
V. METHOD 14
VI. RESEARCH QUESTIONS REGARDING THE IMPACT OF THE WNJ
PROGRAMME 24
VII. PRELIMINARY RESULTS 25
VIII. DISCUSSION 37
REFERENCES 39
2I. SUMMARY
This report concerns the implementation and preliminary evaluation of the Winning New
Jobs Programme in Ireland. Based on the original JOBS Programme (Caplan, Vinokur &
Price, 1997; Vinokur, Schul, Vuori & Price, 2000), this initiative targets job loss as one of
the most consistent antecedents of depression, and involves the design and evaluation of a
preventive intervention. The objective of the programme is to provide job seeking skills to
promote re-employment and to combat feelings of anxiety, helplessness and depression
among the unemployed. The JOBS Programme, which has produced very impressive results,
has been implemented in a number of countries including the United States, China, the
Netherlands, Poland and on a national scale in Finland (Vuori & Schul, 2002).
The implementation of the Winning New Jobs (WNJ) programme in Ireland was piloted on
a cross-border basis in collaboration with a range of training, employment and health
agencies as part of the Rural Mental Health Project (Barry, 2003; Reynolds, Byrne and
Barry, 2004). Part-funding for the training of trainers and roll-out of this programme was
been provided by the EU Peace and Reconciliation (Peace 2) Programme and the evaluation
was funded by the Health Promotion Unit at the Department of Health and Children.
The WNJ programme consists of five intensive, active and structured half-day workshops,
totalling 20 hours of training, delivered by pairs of male and female trainers over a one to
two week period. To date 31 trainers, from a diverse range of agencies across the health and
training sectors, have been trained to deliver the programme. Some 24 training workshops
were delivered in the Donegal and North West region and the Co. Derry/Londonderry
district over an 18 month period. Based on the feedback from trainers and participants, the
training manual utilised by the trainers has been adapted for the Irish context.
The research evaluation of the programme comprised a quasi-experimental design, with
collection of data from workshop participants (the intervention group) and a comparison
group of unemployed people from the same area who did not participate in the WNJ
programme. In all, data were collected from 210 people in the intervention group, of which
44 were mental health service users, and from 192 people in the comparison group, prior to
the training intervention and at 2 weeks and 4 weeks post intervention. The average age of
study respondents was 34 years, 59.2% were women and the average duration of
3unemployment was 4.2 years with 64% unemployed for12 months or longer. In keeping with the original field studies, a wide range of measures was employed with both participants and trainers, using quantitative and qualitative methods. The evaluation sought to determine the feasibility of replicating the programme in the Irish context with both long- term unemployed people and mental health service users. Employing regression analysis, the evaluation examines the programme’s effectiveness in terms of its impact on re- employment and mental health related outcomes. Comparisons with the findings of the Finnish study are drawn where appropriate. Key Findings: Reaction to the Programme The programme was well received by both trainers and participants alike. The preliminary findings indicate that both trainers and participants responded positively and were convinced of the worth and viability of the programme. Effects of the WNJ on Re-employment At four months follow-up, there was a trend of increasing employment rates for the intervention group with 31% employed in comparison to 20.6% of the comparison group. The programme effect on re-employment approached significance level (p
Effects of WNJ on Psychological Factors The WNJ training showed clear significant (p
II. INTRODUCTION
There is strong evidence suggesting the destructive effects of job loss and unemployment on
social and psychological functioning (e.g., Barling, 1990; Dew, Bromet & Schulberg, 1991;
Feather, 1990; Price, 1992; Vinokur, 1997). Epidemiological studies demonstrate that
unemployment causes significant deterioration in mental health and physical well-being
(e.g., Catalano & Dooley, 1977; Warr, 1983). Further studies show that reemployment can
reverse adverse mental health effects and restore the level of mental health that existed prior
to job loss (Iverson & Sabroe, 1988; Kessler, Turner & House, 1988; Vinokur, Schul, Vuori
& Price, 2000).
In the North West region of the Republic of Ireland, County Donegal experiences the
highest unemployment rate (15.6%) compared with all other counties in the Republic of
Ireland. In addition, a 1998 ADM/CPA report “Educational Disadvantage in the Southern
Border Counties of Ireland” revealed that 42.3% of the population in Donegal left school
without any qualification beyond primary school, compared with 28.6% nationally.
Northern Ireland also experiences high unemployment due to historic troubles and divides
which have spurred low investment levels. Northern Ireland has the highest level of
unemployment in the United Kingdom and the lowest figures for GDP in the UK (BBC
Report May 1999). County Derry/Londonderry is particularly considered an “unemployment
blackspot” with 8% unemployment, compared to the Northern Ireland average of 4.7%.
According to the Northern Ireland Executive, “unemployment is the most profound cause of
poverty. Being out of work impacts directly on income, making it difficult for people to
provide for themselves and their families, and to save for a pension so they can support
themselves comfortably in old age” (Northern Ireland Executive, 2001).
As employment is a protective factor for positive mental health, there is a clear need for an
intervention to address reemployment and positive mental health in both Donegal and
Derry/Londonderry
In this report, we will detail the preliminary results of a programme for the unemployed that
was designed in the United States and implemented on a pilot basis in Counties Donegal and
Derry/Londonderry. We then examine the preliminary findings and the implications these
results have for future rollout of this programme.
6III. BACKGROUND
The JOBS programme was originally developed in 1982 in Michigan, USA, as a response to
the downturn in the motor industry. The programme was intended as an intervention for job
seekers to facilitate their return to the labour market, enhance a sense of empowerment, and
to prevent negative mental health consequences of unemployment.
The JOBS programme consists of five intensive and active half-day workshops, held over a
one to two week period, totalling 20 hours of training. Pairs of male and female trainers
work with groups of 12 – 22 unemployed individuals. The programme is delivered in the
form of workshops to help participants apply problem-solving skills, make decisions as a
group, inoculate against setbacks such as job refusals, encourage social support, and learn
job search skills. Participants spend much of their time practising new skills e.g. role-plays
and composing a curriculum vitae.
This programme is suitable for individuals who are recently unemployed, long term
unemployed, or those who have never worked. Large-scale field studies (Caplan, Vinokur
& Price, 1997; Vinokur, Schul, Vuori & Price, 2000) have indicated that the programme can
produce impressive results including:
• Better quality and higher paying jobs (at 2.5 years follow-up)
• Higher confidence in job seeking ability and improved job search skills
• Lower incidence and prevalence of depression (at 2.5 years follow-up)
• Finding re-employment sooner
• Enhanced sense of mastery and inoculation against setbacks
• Improved mental well-being and positive mental health
• Cost effective in terms of increased economic benefits for participants and
the State
• Extended benefits to the family of the job-seeker
As the JOBS programme was so successful in Michigan, it was replicated in other parts of
the United States and several countries including Finland, Russia, Sweden, China, the
Netherlands, and Poland. However, to date, it has only been systematically evaluation in
Michigan and Finland. Results of the Finnish version of the programme, the Työhön Job
7Search programme, which was implemented at a national level, were used to make
comparisons to the Irish results.
How the JOBS Programme Differs from other Job-Search Programmes
The essential components of the programme that make it different to existing programmes
include:
• Active learning. The learning process is almost entirely active and uses the
knowledge and skills of the participants themselves, elicited through small and large
group discussions, brainstorming and other activities. Participants spend much of
their time practising new skills e.g. through role-plays and giving each other
support. This contrasts with other courses which rely more on passive learning.
• Enhancement of general confidence, sense of control and job-related self-efficacy -
that is, increasing one’s belief that one has the necessary skills to find a suitable job
• A supportive learning environment such that trainers constantly provide
encouragement and positive feedback to participants building on their existing
strengths
• Skilled trainers, as most JOBS trainers have vast experience in group work and skills
training
• Inoculation against setbacks. In order for the benefits of the training to persist over
time, participants are coached in planning for and dealing with setbacks and
obstacles such as job refusals.
Some of the content is similar to that of other job training programmes that currently exist,
yet the style of delivery is very innovative and different. The JOBS programme is very
active and practical, and the training is adequately flexible to allow focus on the needs of
each particular group.
Implementing the JOBS Programme in Ireland
In Ireland, the JOBS programme was piloted as part of the Cross-Border Rural Mental
Health Project (Barry, 2003; Byrne & Barry, 2001; Reynolds, Byrne & Barry, 2004) and
evaluated by the Centre for Health Promotion Studies at the National University of Ireland,
Galway. The Rural Mental Health Project was an action research project concerned with
promoting positive mental health and well-being in rural communities. This project, which
8commenced in 1999, was funded under the EU Peace and Reconciliation programme (Peace
2) and part-financed by the UK and Irish Governments. The project brought together two
communities in County Donegal and County Derry/Londonderry in order to develop a
community model of mental health promotion. Therefore, the JOBS programme was piloted
on a regional basis in the cross-border areas of Counties Donegal and Derry/Londonderry
and in rural parts of the North-West including Sligo and Leitrim.
9IV. CONSULTATION AND PLANNING
Several planning sessions took place in conjunction with the North Western Health Board in
Donegal, the Homefirst Community Trust in Derry/Londonderry, and the regional training
and employment agencies on both sides of the border. Initial dialogue focused on the
practicalities of the programme and the schedule of the implementation and evaluation. The
Michigan Prevention Research Centre (MPRC) at the University of Michigan agreed to
work collaboratively in replicating the JOBS programme intervention, training the trainers,
and providing data collection instruments for evaluation.
Information Sessions
Two guest speakers provided information sessions in order to give a first-hand account of
the programme as it has been implemented abroad and, thus, lead the planning for the Irish
course:
• Dr. Jukka Vuori, who has been extensively involved in implementing the JOBS
programme on a national scale in Finland
• Professor Richard Price from the Michigan Prevention Research Centre at the
University of Michigan, who was involved in developing the original programme
Information Session with Dr. Jukka Vuori
In April 2002, a one-day information session regarding the JOBS programme was held in
Derry/ Londonderry. Many local training and employment agencies were invited with
approximately 40 people in attendance. Dr. Vuori addressed a number of key points
including:
• The programme in Finland had positive outcomes for the long-term unemployed
• Full training of the trainers was provided.
• Despite some participants remaining unemployed after the programme, their mental
well-being improved.
• Those participants most at risk benefit most from the programme.
• There was a low drop out rate among the participants.
A video presentation from the Michigan Prevention Research Centre depicted the JOBS
programme training in action. A question and answer session followed, which helped to
clarify further details about the programme. Finally, participants discussed how the
programme might work in an Irish setting. Everyone that attended this meeting agreed that it
was a very positive experience.
10Information Session with Professor Richard Price
In July 2002, a visit by Professor Richard Price from the Michigan Prevention Research
Centre was attended by members of various training and statutory groups. Professor Price
was involved in the original development the programme. He gave a background to the
JOBS programme in the United States since its inception in 1982 and its growing popularity
worldwide. Professor Price and the attendees discussed several topics, including how to
select participants and trainers and the vital importance of research and evaluation to
establish whether the programme works better than other programmes currently offered to
the unemployed.
Agency Involvement
A wide range of statutory, voluntary and community groups who are engaged in training,
employment and support were encouraged to participate in the JOBS programme training.
The agencies that agreed to participate are listed below.
Action Mental Health
Cookstown and District Women’s Group
Cookstown and Dungannon Women’s Aid
Cookstown and Magherafelt Volunteer Bureau
Cookstown Training
Department of Social and Family Affairs
Department for Work and Pensions
Foras Áiseanna Saothair (FÁS) - Training and Employment Agency
Homefirst Community Trust
Mind Matters Project
Network Personnel
North Western Health Board
National Training and Development Institute (NTDI)
Oakleaf Rural Community Network
Rural Health and Social Wellbeing Project
The Training and Employment Agency
Vocational Educational Committee (VEC)
11Beginning in April 2002, the participating agencies held eleven planning sessions in
Donegal, Derry/Londonderry and Sligo. A number of key issues were discussed and agreed
under the following headings:
Participants
• Participant selection criteria was discussed and agreed. The planning group
determined that a group size of 14 - 22 participants was ideal for two facilitators.
• The group felt that the best way to promote the programme was by word of mouth,
which would build up over time.
• The planning group felt that there would be low attrition.
Trainers
• The group discussed the issue of joint facilitation of training sessions with both a male
and female trainer.
• Participating organisations agreed to co-operate to bring trainers together to deliver this
programme.
The Programme
• The planning group suggested that the JOBS programme could be offered as part of
other existing job-training programmes.
• A schedule for the training was discussed. The planning group agreed that there was
a need to take a long-term view to the implementation of the JOBS programme.
• The group agreed that the programme should involve senior management within the
various organisations to ensure the sustainability.
Training of the Trainers
The training agencies selected a number of their staff to participate in this programme.
Thirty-one trainers in total participated from a diverse range of voluntary, statutory and
community agencies, consisting of 15 trainers from Northern Ireland and 16 trainers from
the Republic of Ireland. These trainers were primarily from training and employment
agencies, the health sector and the relevant government departments. Many of these trainers
have had extensive experience in training, employment and support.
12The training of trainers programme took place over six days in both Draperstown, County
Derry/Londonderry and Letterkenny, county Donegal (23rd September–5th October 2002).
IT was run by Mr. Steve Barnaby from the Century Communications Group (part of the
Michigan Prevention Research Centre) and Ms. Sarah Marsh from the University of
Michigan. Both trainers have extensive experience in conducting training of trainers for the
JOBS programme in the United States.
There were high levels of satisfaction with the training for trainers programme. Participants
stated that the success of the JOBS programme would be not so much due to the content, but
more to do with the methods of delivery and learning involved. Participants felt the JOBS
programme was about empowering trainees, giving them a sense of motivation, and
encouraging them to try to gain new opportunities.
Programme Evaluation Materials
Evaluation materials were selected by the Michigan Prevention Research Centre (MPRC) at
the University of Michigan to allow comparison with the US and Finnish JOBS
programmes. These were modified to suit the Irish context and additional process and
qualitative elements were included. Evaluation materials incorporated participants’
questionnaires, trainers’ measures and semi-structured telephone interviews. The evaluation
was undertaken by the Centre for Health Promotion Studies at NUI, Galway.
Piloting the Winning New Jobs Programme
Several trainers completed at least one pilot workshop before the main programme
commenced in order to gain experience running the programme. A researcher from NUI,
Galway observed these sessions to monitor the training at first hand. In addition, all
evaluation materials were piloted. There were no reported difficulties with either the pilot
sessions or the materials.
13V. METHOD
Sample
Participants were recruited through direct contact with the training and employment
agencies and local employment offices in the Republic of Ireland and Northern Ireland.
These individuals were chosen from persons who were registered as unemployed with the
Department of Social and Family Affairs (RoI) and The Training and Employment Agency
(NI). Potential participants were approached by the training agencies, provided with
information on the WNJ programme, and asked if they wished to participate. Mental health
service users were screened for participation in the programme through occupational
guidance services in the health authorities.
The following selection criteria were agreed with the training agencies as to the suitability of
candidates for participation in the study:
• Unemployed for at least six months
• Age ranging from 18 to 60 years of age
• Looking for a job, training or other similar opportunity
• Being “opportunity ready”: able to commence employment, voluntary work or
training
Research Design and Data Collection
This research comprised a quasi-experimental design. In this case, it meant that the training
and employment agencies chose participants for the WNJ programme, while a comparison
group of unemployed individuals who were not participating in the programme was selected
by the NUI, Galway researchers from each of the agencies. Data from the intervention
group and comparison group were compared in order to highlight differences, and therefore
determine the impact of the WNJ programme.
Data were collected from study participants and trainers throughout the study employing
structured questionnaires, which included a range of measures. Participants were asked to
complete four questionnaires at specified periods:
• T1 – Baseline questionnaire (all study participants, prior to WNJ workshops)
• T2 – End of training questionnaire (intervention group only)
• T3 – 2 weeks post training (all study participants)
14• T4 – 4 months post-training (all study participants)
For returning completed questionnaires, participants were entered into a prize draw for €100
/ £75 cash. Participants were guaranteed that the research was confidential and their
participation was anonymous. Local agencies did not have access to any of the participants’
completed questionnaires, as they were posted directly to the NUI, Galway researchers in
the sealed envelopes provided.
The NUI, Galway researcher sent out reminders to the participants in order to increase the
return of the questionnaires. Individuals who did not return the questionnaires in the post
completed some questionnaires voluntarily over the phone.
Figure 1: Schematic Design of the Study
Contacted & Agreed to Participate
n=416
Comparison Group Intervention Group
Not participating in WNJ Participating in WNJ (T1)
n=192 n=210 Baseline
General Unemployed MHS Users
n=166 n=44
(T2)
n=146 n=38 End of
WNJ
n=123 n=102 n=25 (T3)
2 weeks
(T4)
n=120 4 months
n=127 n=25
Participant Profile
A total of 416 individuals returned baseline questionnaires. 402 valid questionnaires were
used for data analysis: 166 (42.3%) from the general unemployed intervention group, 44
15(10.9%) from mental health service users intervention group and 192 (47.8%) from the
comparison group.
The respondents who returned baseline questionnaires varied in age from 15 to 71 years; the
average age was 34 years. Of the total sample, 238 (59.2%) were women and 162 (40.3%)
were men. On their last job, the respondents worked an average of 36.9 hours per week.
The average salary of the participants in their last job was €290 per week. Their average
duration of unemployment was 4.2 years; sixty-four percent of respondents were
unemployed for 12 months or longer.
The baseline sociodemographic and employment profile of the participants is presented in
table on the following page.
16Table 1: Sociodemographic and Employment Profile of the WNJ Participants at Baseline
(T1)
Mental Intervention Comparison
Health Group Group
Service Users (n=166) (n=192)
(n=44)
Gender
Female 65.1 % 70.5 % 49.0 %
Male 34.9 % 29.5 % 51.0 %
Average Age 34.8 years 34.3 years 32.9 years
min: 17 min: 15 min: 18
max: 71 max: 63 max: 60
Marital Status
Married 35.0 % 26.1 % 28.6 %
Separated 2.5 % 7.9 % 4.9 %
Cohabiting - 5.5 % 4.4 %
Divorced 2.5 % 3.6 % 3.3 %
Widowed - 1.2 % 1.1 %
Single/ Never Married 60.0 % 55.8 % 57.7 %
Socioeconomic Group
Professional - 0.7 % 1.4 %
Managerial & Technical 9.1 % 14.6 % 13.7 %
Non-Manual 18.2 % 24.8 % 28.1 %
Skilled manual 27.3 % 27.0 % 18.0 %
Semi-skilled 45.5 % 25.5 % 36.7 %
Unskilled - 1.5 % 2.2 %
Other / Not applicable - 5.8 % -
Religion
Roman Catholic 75.0 % 80.8 % 82.4 %
Protestant Church of Ireland 15.0 % 10.9 % 5.1 %
Other 10.0 % 3.8 % 1.1 %
None - 4.5 % 11.4 %
Level of Education
No schooling / primary school only 5.1 % 8.0 % 6.1 %
Some secondary education 23.1 % 31.5 % 28.2 %
Complete secondary education 35.9 % 34.6 % 29.3 %
Some third level education 25.6 % 13.6 % 18.8 %
Completed third level education 10.3 % 12.3 % 17.7 %
Avg length of time in previous job 5.03 years 5.05 years 5.20 years
Avg weekly wage rate in previous job € 262.03 € 257.04 € 324.89
Avg hours worked weekly in previous job 36.69 hours 35.56 hours 37.97 hours
Avg length of unemployment 6.40 years 4.97 years 3.09 years
min: 5 weeks min: 1 week min: 1 week
max: 56 years max: 25 years max: 30 years
Previously attended employment 36.8 % 33.1 % 35.4 %
preparation or jobs training programmes
Avg weekly take-home family income
(current)
Less than €150 / £100 21.4 % 25.2 % 21.9 %
€150 - €300 / £100 - £200 31.0 % 29.5 % 36.0 %
€300 - €450 / £200 - £300 14.3 % 7.2 % 12.2 %
€450 - €600 / £300 - £400 7.1 % 9.4 % 9.8 %
€600 - €750 / £400 - £500 - 0.7 % 1.8 %
€750 - €900 / £500 - £600 2.4 % 1.4 % 1.2 %
€900 + / £600 + - 1.4 % 0.6 %
Don't know / prefer not to answer 23.8 % 25.2 % 16.5 %
17Intervention Commencement
Those allocated to the intervention group attended the prescribed programme, which
consisted of five intensive half-day workshops (20 hours) held over a one-week period. Two
trainers, one male and one female if possible, delivered the training.
The workshops were based in health board centres and centres of the training and
employment agencies. Twenty-four workshops (12 in Northern Ireland and 12 in the
Republic of Ireland) were held over a period of 18 months. According to trainer
observations:
• Each workshop session lasted an average 3.39 hours (SD: 0.699)
• An average of nine people attended each session (SD: 3.890, range: 2 to 22).
• On average, 71% of respondents attended the sessions on a voluntary basis.
QUANTITATIVE MEASURES
Questionnaires included measures of sociodemographic characteristics, work and
employment history, job-search intensity, reemployment, and mental health.
Demographics, assessed at T1, included gender, age, education, family income and personal
income, marital status, religious background, number of dependent children and the number
of people that are financially dependent on the respondent. Date of birth was asked in all
subsequent questionnaires to match participants’ answers with those in the T1
questionnaires.
Past Employment and its Quality was evaluated at T1 and included previous occupation,
length of tenure, number of hours worked, the wage rate and length of time since leaving job
(in weeks, months or years).
Reemployment Outcomes were evaluated in follow-up questionnaires at T3, and T4. They
were determined by the question “What is your employment status now?” If the participants
were employed, they were asked the number of hours they work (per day, week or month).
As a measure of quality of reemployment, respondents were asked their wage rate, which
included options for the amount earned per hour, week, month or year. In addition,
respondents were asked whether they had applied to pursue further training or education, as
these may be considered viable alternatives to re-employment.
18Physical and Mental Health Measures were measured at T1, T3, and T4. General Health
was assessed by asking participants to rate their general health on a single five-point scale.
Depression was measured with a subscale of 11 items based on the Hopkins Symptom
Checklist. This required respondents to indicate how much they have been bothered or
distressed by various symptoms in the last two weeks including crying easily, blaming
yourself for things and feeling down or blue.
Job Search Intensity Measures were evaluated at T1, T3, and T4. These included the
following:
1) Job Seeking Efficacy - Respondents were asked to rate how confident they felt about
being able to perform job seeking tasks including: making a list of the skills that they
have; talking to friends and others to find out about potential employers and job
openings; and completing a CV.
2) Sense of Mastery - Respondents were asked how strongly they agreed or disagreed
with seven statements regarding their sense of control and whether they felt their
control was internal or external. Statements included “There is really no way I can
solve some of the problems I have” and “What happens to me in the future mostly
depends on me”.
3) Inoculation against setbacks - This examined how the programme promoted job-
search behaviour. Four statements assessed whether individuals anticipated possible
setbacks or barriers in their job search, and how prepared they were to overcome
them.
4) Economic hardship – This was measured with three statements whereby respondents
rated their current and anticipated economic hardship.
5) Job Search Activities – This scale examined respondents’ frequency of job-search
activities. The five questions included looking for vacancies at the local
employment office, following newspaper advertisements, contacting employers, and
considering other possibilities for employment such as starting one’s own business.
6) Job Search Motivation - This measure looked at subjective norms, attitudes, and
intention to engage in job seeking within the next four months. Subjective norms
assessed what participants’ spouses and friends think about how hard they should try
to get a job. Attitudes toward job seeking measured how useful, beneficial, and wise
19it was to try hard to get a job. Intention to engage in job seeking assessed how
intensely respondents intended to find a job.
Integrity of the Intervention was assessed via the T2 questionnaire, which was administered
only to programme participants. This evaluated the immediate impact of the programme on
participants and included the following elements:
1) Facilitative Learning Process – These questions examined participants’ feelings and
reactions overall during the training including their perceptions of the trainers.
Examples of questions used include “how much have you felt free to participate and
ask questions?” and “how much did the trainers show that they valued your
participation?”
2) Referent Power of Trainers – This investigated the influence of the trainers on the
respondents. Pairs of words (both positive and negative) were presented, for
example, “supportive / unsupportive” and “warm / cold”. Respondents chose the
degree to which each of these words described the trainers.
3) Attraction to the Group – Similar to questions asked concerning the referent power
of trainers, these questions examined respondents’ feelings about others participating
in the WNJ training. Pairs of words were used again to describe the group as a
whole, and respondents chose the degree to which each of these words described the
group members.
4) JOBS-likeness – This last section investigated how well the sessions followed the
prescribed JOBS training programme, looking specifically at how frequently certain
activities occurred during the sessions. These activities included active learning
methods (e.g. often the individual participated in role-plays, discussions and group
work), trainers’ activities (e.g. how often trainers provided examples and encouraged
participation) and the individual’s practice of skills to search for jobs and inoculate
against setbacks (e.g. composing a successful CV).
Trainers’ Questionnaires
Each trainer completed a short questionnaire immediately after each workshop session.
Questions dealt with the trainer’s experience with the workshop, an overall rating on how
well the session went, the programme components that were covered as well as the
attendance rates of the participants. Recommendations were requested so that suggestions
could be incorporated into the training.
20QUALITATIVE MEASURES
Feedback from Trainers
An NUI, Galway researcher conducted semi-structured telephone interviews with a sub-
sample of the trainers to establish trainers’ views of the WNJ programme. The interviews
covered the following themes: trainers’ opinions of how the programme compares to other
job training programmes they have delivered; suggestions for how the training could be
improved; changes they noticed in the participants because of their experiences in the
programme; ways their experience varied in delivering the programme and the factors that
influenced this; and the major challenges in delivering the programme.
Researcher Observations
The NUI, Galway researcher observed a sample of the workshop sessions to examine the
extent to which the programme was delivered as intended (“programme fidelity”). Details of
the programme were documented and the process of implementation was analysed to
examine how it relates to the programme impact and outcomes. Topics included:
• how the session went
• how participants’ job seeking skills changed
• how participants’ self-esteem and confidence changed
• how the trainers presented the information
• how discussions were conducted
• how the group worked together.
Data Analysis
Both regression analysis and descriptive statistics were used to examine the quantitative data
from the questionnaires. Regression analysis is one of the most useful and commonly
reported data analysis techniques. It is used to investigate how much certain “independent
variables” (IVs), assessed at baseline, are able to predict a specific outcome or “dependent
variable” (DV). In this case, regression analysis was employed to predict re-employment,
mental health, and related outcomes (DV) at four months post-intervention using a number
of independent variables (IVs) from the baseline including: intervention vs. comparison
group, demographics, and psychological variables. Of particular interest in assessing the
effects of the programme was the variable relating to intervention or comparison group
impact on the four month outcomes.
21The findings of the regression analysis are presented in Table 2 on the following page.
Significant results are highlighted, and will be discussed in the Results section.
Note: The mental health service users were not included in the regression analysis as their
overall profile was different from that of both the intervention and comparison groups,
which could bias the overall regression outcomes.
22Table 2: Logistic Regression and Multiple Regression Coefficients (ß) of the Effects on the WNJ Intervention, Demographics, Baseline
(T1) Psychological Factors on Reemployment, General and Mental Health Outcomes, and Psychological Variables at 4-month (T4)
Follow-Up
Baseline Predictor T4 Employment Outcome T4 General and T4 Psychological Variables
Mental Health
Re- Job Seeking General Depressive Sense of Job Search Economic Job Seeking Inoculation
employment Activities Health Symptoms Mastery Motivation Hardship Efficacy
Jobs Intervention (0=intrv, -.151† -.044 .101 -.119† -.010 -.117 .121 -.183** -.163†
1=comp)
Demographics
Age (0=VI. RESEARCH QUESTIONS REGARDING THE IMPACT OF THE
WNJ PROGRAMME
Because the intervention in Ireland was designed to replicate both the Finnish and American
programmes, it was important to examine if the Irish programme produced similar main
effects. Based on the main objectives of the Irish Wining New Jobs programme, as well as
the US and Finnish studies, a number of specific research questions were explored.
Question 1: Was the Winning New Jobs programme implemented successfully in an Irish
context?
Question 2: Do programme participants, trainers, and researchers receive the WNJ
programme positively?
Question 3: Does the WNJ programme improve re-employment outcomes, including training
and further educational opportunities?
Question 4: Does the WNJ programme have a positive impact on mental health outcomes?
Question 5: Does the WNJ programme improve psychological variables relating to job
search intensity (sense of mastery, job search motivation, economic hardship, job seeking
efficacy, and inoculation against setbacks)?
Question 6: Does length of unemployment at baseline impact on the effects of the WNJ
programme?
Question 7: Do depressive symptoms at baseline impact on the effects of the WNJ
programme on both reemployment and mental health outcomes?
Question 8: Does the level of job search intensity impact reemployment outcomes?
24VII. PRELIMINARY RESULTS
Question 1: Was the Winning New Jobs programme implemented successfully in an
Irish context?
The T2 questionnaire, administered to WNJ participants at the end of the programme,
assessed how closely the workshops followed the prescribed manual for the programme (the
“JOBS-likeness scale”). Overall, there was a very high comparability, as the majority of
respondents reported taking part in JOBS recommended activities “once a day” or “many
times a day”:
Table 3: JOBS-Likeness Scale, as expressed in percentages for those responding “once a
day” or “many times a day”
Statement: “How often did you…” General MHS
Unemployed Users
n=146 n=38
Practice in pairs 84.2 66.7
Work in small groups (3-5 persons) 73.8 75.8
Take part in role-play 77.8 63.6
Discuss your experiences 93.7 78.8
Trainers return participants’ questions back to the group 90.9 81.8
Trainers say something positive about the discussion 92.5 72.7
Materials appear well organised and clear 92.3 75.8
Trainers give reasons why something a participant said was very
relevant 91.7 93.9
Trainers refer back to participants' answers later during the group
work 89.7 90.9
Trainers encourage the group, particularly more silent members, to
participate 88.6 87.9
Trainers give thanks to speakers in the group 89.0 90.9
Other participants help you in understanding your problems 85.4 78.8
You discover how to find about possible jobs through people you
know 79.7 72.7
Others help you learn how to do a successful job application 81.4 72.8
You practice how to contact employers to get an interview 80.2 72.7
Others help you to learn how to do a successful interview 83.5 75.8
Talk about barriers to presenting your skills & strengths effectively 77.8 69.7
Talk about barriers to finding out about possible jobs through
people you know 70.0 69.7
Talk about barriers to contacting employers successfully 73.4 63.6
Talk about barriers to doing a successful job interview 73.9 60.6
On average, 82.5 % of the General Unemployed participants and 75.8 % of Mental Health
Service User participants found that these programme elements were included in workshops
at least once a day.
25Question 2: Do programme participants, trainers, and researchers receive the WNJ
programme positively?
a) Participants’ Reaction to the Programme
Results from the T2 questionnaire demonstrate that participants responded very positively to
the programme, the trainers, and other group members. According to the Facilitative
Learning Scale, overall, 77% of participants felt that the WNJ programme was very (“pretty
much” or “a great deal”) beneficial, as noted in the table below.
Table 4: Facilitative Learning Scale, as expressed in percentages
General Unemployed MHS Users
Pretty much / Not at all Pretty much / Not at all
a great deal a great deal
I was free to participate and ask questions 81.4 0 84.9 0
Others listened to what I had to say 81.5 0 90.9 0
Other participants shared information 71.9 0.7 72.7 0
Trainers shared information about their 81.3 0.7 0
background 75.8
I felt I myself shared information 49.3 3.4 69.7 3.0
I felt other participants had similar problems 61.0 4.1 0
to me 48.5
The material and the discussion were 78.8 6.2 0
relevant 75.8
The material and discussion were well- 79.4 2.7 0
organised & clear 93.9
I feel enthusiastic about job hunting 73.8 1.4 69.7 0
I feel enthusiastic about the future 50.3 26.5 75.7 0
I felt confused by the discussion 2.7 85.0 3.0 66.7
I was embarrassed by what I said 2.0 84.3 3.0 54.5
I was interrupted when I spoke 29.3 60.5 3.0 84.8
Trainers ignored me 24.5 64.6 0.0 97.0
Trainers made me feel upset 19.7 65.3 0.0 93.9
Trainers show they valued my participation 61.9 30.6 100.0 0
Trainers make me feel happy 47.6 34.7 90.9 0
Trainers made me feel enthusiastic 75.2 5.5 75.7 0
Trainers showed an understanding of my 87.1 4.8 0
problems 87.9
Trainers showed I have an improved chance 76.8 4.1 0
of getting a job 87.9
I think I could be good at things the trainers 76.9 2.7 0
discussed 66.6
Next, participants were asked to rate the trainers on a seven point scale i.e. very supportive to
very hostile, very helpful to very unhelpful. This was in order to determine the Referent
Power of Trainers. As per the table below, more than 75% of respondents felt positively
about the trainers, checking off that they were “very” or “somewhat” for the positive
26adjective. It is notable that the vast majority of mental health service users felt positively
towards the trainers:
Table 5: Referent Power of Trainers Scale- Percentage of respondents reporting “very” or
“somewhat” in regards to positive comments towards trainers
General Unemployed MHS Users
Supportive 83.4 96.9
Sympathetic 77.9 93.8
Enthusiastic 84.0 96.9
Knowledgeable 84.8 96.9
Warm 82.1 96.9
Helpful 82.8 96.9
Sincere 84.0 96.9
Communicating clearly 85.7 93.8
Accepting 79.6 96.9
Similar to questions in regards to the trainers, the Attraction to Group was measured by
asking participants their feelings towards the group as a whole. As per the table below, the
majority of intervention participants felt positively towards the group, with the vast majority
of mental health service users responding very positively:
Table 6: Attraction to the Group Scale - Percentage of respondents reporting “very” or
“somewhat” in regards to positive comments towards the group as a whole
General Unemployed MHS Users
Supportive 75.9 93.6
Warm 79.7 96.7
Sincere 76.0 93.5
Accepting 79.9 93.5
Likeable 60.7 96.7
b) Trainers Reaction to the Programme
Overall, the 39 trainers in the WNJ programme who completed questionnaires on the
workshops reported positive feedback towards each of the sessions. The majority found each
component “extremely satisfactory” or “very satisfactory”.
Table 7: Percentage of trainers reporting “extremely satisfactory” or “very satisfactory” in
regards to individual workshop measures.
Workshop measure Trainers
How satisfactory was the session? 77.7 %
How did the exercises work? 77.3 %
How responsive were the participants? 69.6 %
How adequate was the training? 68.9 %
How comfortable were you with your role? 83.6 %
How difficult was it to follow the manual? 53.6 %
How difficult was it to integrate the programmer into
existing programmes? 67.9 %
27c) Independent Researcher’s Observation of the Programme
The researcher from the National University of Ireland, Galway independently observed each
workshop session from two different workshop groups in order to record impartial
observations of workshop content, trainers, participants, and dynamics of the session.
Employing a structured observation form, the researcher reported the following observations:
• Made critical or negative remarks about participants: never any extent (100%)
• Made insensitive or thoughtless comments to individual participants: never any extent
(100%)
• Exhibited personal care and unconditional positive regard for participants: a very great
extent (75%) / a great extent (25%)
• Displayed enthusiasm for the materials: a very great extent (75%) / a great extent (25%)
• Seemed to appreciate participants’ comments and ideas: a very great extent (75%) / a great
extent (25%)
• Presented useful information and discuss applications: a very great extent (75%) / a great
extent (25%)
• Generated discussions that were spontaneous and unhindered: some extent (75%) / a great
extent (25%)
• Had discussions that seemed to be confusing and take long diversions: never any extent
(75%) / a little extent (25%)
The researcher reported that trainers gave positive feedback towards the group as a whole,
indicating that the participants:
• Created a friendly, congenial and supportive atmosphere: some extent (25%) / a great
extent (25%) / a very great extent (50%)
• Created a feeling of mutual appreciation and respect among the members: some extent
(25%) / a great extent (25%) / a very great extent (50%)
• Created a sense of working together toward a common goal: a little extent (25%) / a great
extent (25%) / a very great extent (50%)
• Brought about interpersonal conflict during discussions: never any extent (100%)
• Actively sought information, evaluations and comments from one another: a little extent
(100%)
• Pushed their own opinion rather than listen to what others had to offer: never any extent
(100%)
28Question 3: Does the WNJ programme improve re-employment outcomes, including
training and further educational opportunities?
At four months follow up, 31% of the intervention group, 24% of the mental health service
users group, and 21% of the comparison group reported being re-employed.
Figure 2: Employment Status over Time – WNJ Participants Employed
35
31.1
30
Intervention Group
25
20.9 23.5 Comparison Group
20 20.0 20.6 MHS Users
[%]
15 16.8
10
5
0
0
T1 T3 T4
The figure above depicts how the WNJ programme had a beneficial impact on
reemployment. This finding was reinforced through regression analysis, which indicated that
the intervention had an impact approaching significance (ß=-.151, pFigure 3: Average Wage Rate (weekly)
350.00
300.00
250.00
Intervention Group
200.00
Comparison Group
150.00
MHS Users
100.00
50.00
0.00
T1 T3 T4
Figure 4: Average Hours Worked (weekly)
38 38
37
36 36 Intervention Group
34 Comparison Group
32 MHS Users
30
29
28 28
26
26
24 25
23
22
T1 T3 T4
The data suggests that participants who achieved employment at T3 and T4 re-entered the
workforce on a part-time basis, as their hours and wages dropped dramatically from their
previous employment, noted at T1. To analyse further, the average wage rate per hour was
calculated for those persons who reported hours and wages at T1, T3 and T4 (n=15).
However, it should be noted that the sample size for this calculation is quite small, and too
few mental health service users reported this information to be included in the comparison.
Figure 5: Average Wage Rate per Hour (in euros), respondents reporting at T1, T3 and T4
12
11
11
10
10
9 9 9
9 Intervention Group
8 8 Comparison Group
7
6
5
T1 T3 T4
30The figure above suggests that the intervention group may enjoy a better quality of
reemployment than the comparison group, as the hourly wage rate increased notably at two
weeks post-intervention (T3). Similar to the re-employment rate figures, the quality of
employment may also increase though it will take a longer time period to see this effect in
full evidence.
Figure 6: Job Search Activities Scale (1 few activities through 4 many activities)
2.4
2.2
2.2
2.0
Intervention Group
2.0 2.0
1.9 Comparison Group
1.8 1.9
MHS Users
1.6 1.7 1.7
1.4 1.5
1.2
1.0
T1 T3 T4
In terms of job search activities, such as asking friends about job openings and checking
employment ads in newspapers, the figure above illustrates that the comparison group’s
activities decreased, the intervention group’s activities remained largely the same and the
MHS Users group’s activities increased slightly. Regression analysis determined that this
effect was not significant.
Figure 7: Percentage of respondents who applied for further training or educational
opportunities
in the last four months
45.0
40.0 40.0
Intervention Group
35.0 34.8
Comparison Group
30.0 29.9
27.3 MHS Users
25.0 26.4
24.1
20.0
15.0
10.0
T1 T4
Figure 7 (above) demonstrates that an almost equal number of intervention and comparison
group participants applied for further training or educational opportunities within the four
months following the T1 baseline questionnaire. A clearer difference between the groups
may become evident with longer-term follow-up.
31Question 4: Does the WNJ programme have a positive impact on mental health
outcomes?
The Depression Scale, illustrated below, shows that the depression levels of the intervention
group and the comparison group remained virtually unchanged across time. The mental
health service users had a higher level of depression overall.
Figure 8: Depression Scale (1 low depression, 5 extreme depression)
2.6
2.4
2.4
2.2 2.1 Intervention Group
2.0 1.9 2.0
1.9 Comparison Group
1.8 1.8
1.8 MHS Users
1.6 1.7
1.4
1.2
1.0
T1 T3 T4
Regression analysis indicated that the WNJ intervention approached significance in affecting
depressive symptoms (ß=-.119, pFigure 9: Job Seeking Efficacy Scale (1 low efficacy, 5 high efficacy)
3.5
3.4
3.4 3.4
3.3 Intervention Group
3.3 3.3
Comparison Group
3.2
3.2 MHS Users
3.1 3.1
3.0
2.9 2.9 2.9
2.8
T1 T3 T4
The figure above illustrates how the WNJ programme had a strong effect on Job Seeking
Efficacy (i.e. confidence to perform job seeking tasks), as efficacy levels for the intervention
group and the mental health service users intervention group increased over time, while the
comparison group remained virtually unchanged. This finding was reinforced through
regression analysis, which determined that intervention group was significantly more likely
to have higher job seeking efficacy than the comparison group at four months follow-up (ß=-
.183, pFigure 11: Mastery Scale (1 low mastery, 4 high mastery)
3.1
3.0 3.0
3.0 2.9 Intervention Group
2.9
Comparison Group
2.9
2.8 2.8 MHS Users
2.8 2.8
2.8
2.7
2.7
2.7
2.6
2.6
T1 T3 T4
The figure above indicates that the level of mastery (i.e. sense of control over problems)
remained basically unchanged for the intervention and comparison groups; however, the
level of mastery steadily increased, albeit by a small amount, in the mental health service
users. Regression analysis found that a relationship between the WNJ programme and a
sense of mastery four months post-intervention was not significant.
Figure 12: Job Search Motivation Scale (1 little motivation, 5 high motivation)
4.4
4.3
4.2
4.2 Intervention Group
4.1 4.1 Comparison Group
MHS Users
4.0 4.0
3.9 3.9
3.8
3.7
T3 T4
Job Search Motivation (i.e. how hard they believe they should try to gain employment), as
depicted above, was measured at two weeks post-intervention and four months post-
intervention only. Note that the results for the intervention group and the MHS Users was the
same, hence both are represented by the pink bar. It appears as though job search motivation
for the intervention group and the MHS Users increased over time, while motivation in the
comparison group decreased. Regression analysis, however, did not indicate that this result
was significant.
34Figure 13: Economic Hardship Means Scale (1 little hardship, 5 extreme hardship)
2.8
2.8
2.7
2.7 2.7
2.6 Intervention Group
2.6 Comparison Group
2.5 2.5
2.4 MHS Users
2.4
2.3
2.3 2.3
2.2
2.1 2.1
T1 T3 T4
The Economic Hardship Scale, illustrated above, shows that the intervention group’s
perception of hardship dropped at two weeks post-training (T3) – this could be due to
immediate “afterglow” effects of the training, inspiring a positive outlook towards obtaining
employment and a perceived reduction in financial hardship. Regression analysis, however,
did not indicate that the results were significant.
Question 6: Does length of unemployment at baseline impact on the effects of the WNJ
programme?
Short-term unemployed (In terms of psychological variables associated with job search intensity, higher depressive symptoms at baseline were significantly predictive of a lower sense of mastery (ß=-.367, p
VIII. DISCUSSION
As evident in this report, the Winning New Jobs programme was well received by trainers
and participants alike. The preliminary results clearly indicate that trainers and participants
responded positively, and were convinced of the programme’s effectiveness and viability.
This study was novel in that mental health service users were a key group targeted for
intervention, and their response to the programme was very positive. The trainers who
administered the workshops can now pass on their skills and experiences to more trainers, in
order to disseminate the programme to a wider audience.
However, throughout the course of administering the workshops, the trainers made it clear
that certain sections of the original American “JOBS Manual for Teaching People Successful
Job Search Strategies” (1992) had to be altered in order to suit the Irish context. An average
of 45% of trainers (range 31% to 67%) reported adding in or leaving out items from the
JOBS manual in each WNJ workshop session. A key issue that was highlighted by the
trainers and agencies was the use of language and examples more suited to an American
audience. In response, at the end of the workshops a full-day meeting was held in which all
the trainers were invited to attend in order to review the programme and re-write the training
manual. Issues discussed included: what works or does not work with each session; what
needs to be changed, added or left out; format and layout of the manual; and availability of
extra resources and materials. Those who could not attend the meeting were asked for their
contributions by email or telephone. As a result of this meeting, a draft copy of the new
manual was produced and made available to all trainers for feedback. Based on this further
feedback, more revisions were incorporated and the newly adapted Winning New Jobs
manual was completed. This manual has now been published and is available for
distribution. It will serve as the basis for the propagation of the Winning New Jobs
programme in the Republic of Ireland and Northern Ireland.
Conclusions
In summary, the impact of the Winning New Jobs programme for the unemployed may vary
depending on the social and cultural context in which the programme is implemented. It is,
however, clear that the results of this initial pilot of the WNJ programme are positive and
encouraging in an Irish context. The programme has had positive benefits in terms of
improved confidence in job seeking skills, inoculation against setbacks and a trend towards
improved mental health and reemployment. The outcomes and impacts of the Irish
37programme are substantiated by the fact that they show many similar effects to those found
in the Finnish Työhön programme.
There were several differences between the Finnish Työhön programme and the Irish
Winning New Jobs programme that must be noted. The sample size in the Finnish study (N=
1,261) was substantially larger than the sample in the Irish study (N= 402), mostly because
the Finnish programme was rolled out nationally and the Irish programme was rolled-out in
only one region. The sample also differed in that there was a random assignment to groups in
Finnish study, while in Ireland the training agencies selected which participants would be
involved in the intervention. The Finnish sample ranged in age from 18-61 years with a
median age of 37; meanwhile, the Irish sample had an average age of 34 with a range from
15-71 years. There was a majority of women in both studies. Most participants in the
Finnish study were short-term unemployed (median duration of unemployment 5 months, M
= 10.7, SD = 17.3) with 28% unemployed for 12 months or longer. In the Irish study, the
average duration of unemployment was 4.2 years with 64% unemployed for 12 months or
longer. Despite these differences, the findings from the Irish study at this stage compare well
with those reported in the Finnish study at six months follow-up. Further analysis of data,
for example a 12 month follow-up, could be used to draw additional conclusions on the long-
term impact of the programme. Finally, the Irish programme may have had other differences
as compared to the Finnish study since each programme was implemented in a vastly
different labour market and social context.
The findings from this pilot implementation suggest that a 20-hour Winning New Jobs
intervention can positively influence both reemployment and mental health outcomes. It
should also be noted that this programme has been successfully implemented with mental
health service users, showing positive outcomes. Such encouraging outcomes support the
call for a large-scale rollout of this programme nationally, as well as a longer-term follow-up
with a larger sample.
38You can also read
