Preliminary Findings for the "Winning New Jobs" Programme in Ireland - Centre for Health Promotion Studies The National University of Ireland, Galway

Preliminary Findings for the
“Winning New Jobs” Programme
           in Ireland

               January 2005

   Centre for Health Promotion Studies
 The National University of Ireland, Galway

This report was prepared by Dr. Margaret Barry, who developed and directed the overall
programme evaluation, Colette Reynolds who collected and inputted the evaluation data,
and Róisín Egenton who analysed and reported on the data analysis.

The authors would like to express their sincere thanks to the following:
   •   The unemployed individuals who participated in the Winning New Jobs programme,
       and all those who completed baseline and follow-up questionnaires.
   •   All the training and employment agencies, health services and community groups
       who participated in the Winning New Jobs Planning Group, and who facilitated the
       programme evaluation.
   •   The WNJ programme trainers for their support in administering the evaluation
       measures and providing valuable feedback for the adaptation and improvement of
       this programme.
   •   Dr. Jukka Vuori, who has been extensively involved in implementing and evaluating
       the JOBS programme in Finland, and Professor Richard Price from the Michigan
       Prevention Research Centre at the University of Michigan, who was involved in
       developing the original programme. Both contributed to the initial planning as well
       as the evaluation of the programme in an Irish context.
   •   Ms. Mary Duggan and Ms. Therese Lowry, Project Managers with the Rural Health
       and Social Well-being Project, and Ms. Ann Marie Nedeljkovic and Ms. Inga Bock,
       Project Managers with the Rural Health Mind Matters Project for their assistance and
       support in undertaking the programme evaluation.
   •   The Northern Health and Social Services Board and the Health Promotion
       Departments of the North Western Health Board and Homefirst Community Trust
       for their support of this initiative.
   •   The research staff at the Centre for Health Promotion Studies at NUI, Galway for
       their help with data collection and support throughout the project. In particular,
       thanks to Dr. Saoirse Nic Gabhainn for her advice on data analysis.

The implementation of the WNJ project was funded by the European Union under the
Programme for Peace and Reconciliation 2000 – 2004 (Peace 2) and part-financed by the
UK and Irish Governments. The evaluation of this initiative was funded by the Health
Promotion Unit of the Department of Health and Children, Dublin. We gratefully
acknowledge the support of both funders.

ACKNOWLEDGEMENTS                                         1

I. SUMMARY                                               3

II. INTRODUCTION                                         6

III. BACKGROUND                                          7

IV. CONSULTATION AND PLANNING                            10

V. METHOD                                                14


   PROGRAMME                                             24

VII. PRELIMINARY RESULTS                                 25

VIII. DISCUSSION                                         37

REFERENCES                                               39


This report concerns the implementation and preliminary evaluation of the Winning New
Jobs Programme in Ireland. Based on the original JOBS Programme (Caplan, Vinokur &
Price, 1997; Vinokur, Schul, Vuori & Price, 2000), this initiative targets job loss as one of
the most consistent antecedents of depression, and involves the design and evaluation of a
preventive intervention. The objective of the programme is to provide job seeking skills to
promote re-employment and to combat feelings of anxiety, helplessness and depression
among the unemployed. The JOBS Programme, which has produced very impressive results,
has been implemented in a number of countries including the United States, China, the
Netherlands, Poland and on a national scale in Finland (Vuori & Schul, 2002).

The implementation of the Winning New Jobs (WNJ) programme in Ireland was piloted on
a cross-border basis in collaboration with a range of training, employment and health
agencies as part of the Rural Mental Health Project (Barry, 2003; Reynolds, Byrne and
Barry, 2004). Part-funding for the training of trainers and roll-out of this programme was
been provided by the EU Peace and Reconciliation (Peace 2) Programme and the evaluation
was funded by the Health Promotion Unit at the Department of Health and Children.

The WNJ programme consists of five intensive, active and structured half-day workshops,
totalling 20 hours of training, delivered by pairs of male and female trainers over a one to
two week period. To date 31 trainers, from a diverse range of agencies across the health and
training sectors, have been trained to deliver the programme. Some 24 training workshops
were delivered in the Donegal and North West region and the Co. Derry/Londonderry
district over an 18 month period. Based on the feedback from trainers and participants, the
training manual utilised by the trainers has been adapted for the Irish context.

The research evaluation of the programme comprised a quasi-experimental design, with
collection of data from workshop participants (the intervention group) and a comparison
group of unemployed people from the same area who did not participate in the WNJ
programme. In all, data were collected from 210 people in the intervention group, of which
44 were mental health service users, and from 192 people in the comparison group, prior to
the training intervention and at 2 weeks and 4 weeks post intervention. The average age of
study respondents was 34 years, 59.2% were women and the average duration of

unemployment was 4.2 years with 64% unemployed for12 months or longer. In keeping
with the original field studies, a wide range of measures was employed with both
participants and trainers, using quantitative and qualitative methods. The evaluation sought
to determine the feasibility of replicating the programme in the Irish context with both long-
term unemployed people and mental health service users. Employing regression analysis,
the evaluation examines the programme’s effectiveness in terms of its impact on re-
employment and mental health related outcomes. Comparisons with the findings of the
Finnish study are drawn where appropriate.

Key Findings:

Reaction to the Programme
The programme was well received by both trainers and participants alike. The preliminary
findings indicate that both trainers and participants responded positively and were convinced
of the worth and viability of the programme.

Effects of the WNJ on Re-employment
At four months follow-up, there was a trend of increasing employment rates for the
intervention group with 31% employed in comparison to 20.6% of the comparison group.
The programme effect on re-employment approached significance level (p
Effects of WNJ on Psychological Factors
The WNJ training showed clear significant (p

There is strong evidence suggesting the destructive effects of job loss and unemployment on
social and psychological functioning (e.g., Barling, 1990; Dew, Bromet & Schulberg, 1991;
Feather, 1990; Price, 1992; Vinokur, 1997). Epidemiological studies demonstrate that
unemployment causes significant deterioration in mental health and physical well-being
(e.g., Catalano & Dooley, 1977; Warr, 1983). Further studies show that reemployment can
reverse adverse mental health effects and restore the level of mental health that existed prior
to job loss (Iverson & Sabroe, 1988; Kessler, Turner & House, 1988; Vinokur, Schul, Vuori
& Price, 2000).

In the North West region of the Republic of Ireland, County Donegal experiences the
highest unemployment rate (15.6%) compared with all other counties in the Republic of
Ireland. In addition, a 1998 ADM/CPA report “Educational Disadvantage in the Southern
Border Counties of Ireland” revealed that 42.3% of the population in Donegal left school
without any qualification beyond primary school, compared with 28.6% nationally.
Northern Ireland also experiences high unemployment due to historic troubles and divides
which have spurred low investment levels. Northern Ireland has the highest level of
unemployment in the United Kingdom and the lowest figures for GDP in the UK (BBC
Report May 1999). County Derry/Londonderry is particularly considered an “unemployment
blackspot” with 8% unemployment, compared to the Northern Ireland average of 4.7%.
According to the Northern Ireland Executive, “unemployment is the most profound cause of
poverty. Being out of work impacts directly on income, making it difficult for people to
provide for themselves and their families, and to save for a pension so they can support
themselves comfortably in old age” (Northern Ireland Executive, 2001).

As employment is a protective factor for positive mental health, there is a clear need for an
intervention to address reemployment and positive mental health in both Donegal and

In this report, we will detail the preliminary results of a programme for the unemployed that
was designed in the United States and implemented on a pilot basis in Counties Donegal and
Derry/Londonderry. We then examine the preliminary findings and the implications these
results have for future rollout of this programme.


The JOBS programme was originally developed in 1982 in Michigan, USA, as a response to
the downturn in the motor industry. The programme was intended as an intervention for job
seekers to facilitate their return to the labour market, enhance a sense of empowerment, and
to prevent negative mental health consequences of unemployment.

The JOBS programme consists of five intensive and active half-day workshops, held over a
one to two week period, totalling 20 hours of training. Pairs of male and female trainers
work with groups of 12 – 22 unemployed individuals. The programme is delivered in the
form of workshops to help participants apply problem-solving skills, make decisions as a
group, inoculate against setbacks such as job refusals, encourage social support, and learn
job search skills. Participants spend much of their time practising new skills e.g. role-plays
and composing a curriculum vitae.

This programme is suitable for individuals who are recently unemployed, long term
unemployed, or those who have never worked. Large-scale field studies (Caplan, Vinokur
& Price, 1997; Vinokur, Schul, Vuori & Price, 2000) have indicated that the programme can
produce impressive results including:

           •   Better quality and higher paying jobs (at 2.5 years follow-up)
           •   Higher confidence in job seeking ability and improved job search skills
           •   Lower incidence and prevalence of depression (at 2.5 years follow-up)
           •   Finding re-employment sooner
           •   Enhanced sense of mastery and inoculation against setbacks
           •   Improved mental well-being and positive mental health
           •   Cost effective in terms of increased economic benefits for participants and
               the State
           •   Extended benefits to the family of the job-seeker

As the JOBS programme was so successful in Michigan, it was replicated in other parts of
the United States and several countries including Finland, Russia, Sweden, China, the
Netherlands, and Poland. However, to date, it has only been systematically evaluation in
Michigan and Finland. Results of the Finnish version of the programme, the Työhön Job

Search programme, which was implemented at a national level, were used to make
comparisons to the Irish results.

How the JOBS Programme Differs from other Job-Search Programmes
The essential components of the programme that make it different to existing programmes
   •   Active learning. The learning process is almost entirely active and uses the
       knowledge and skills of the participants themselves, elicited through small and large
       group discussions, brainstorming and other activities. Participants spend much of
       their time practising new skills e.g. through role-plays and giving each other
       support. This contrasts with other courses which rely more on passive learning.
   •   Enhancement of general confidence, sense of control and job-related self-efficacy -
       that is, increasing one’s belief that one has the necessary skills to find a suitable job
   •   A supportive learning environment such that trainers constantly provide
       encouragement and positive feedback to participants building on their existing
   •   Skilled trainers, as most JOBS trainers have vast experience in group work and skills
   •   Inoculation against setbacks. In order for the benefits of the training to persist over
       time, participants are coached in planning for and dealing with setbacks and
       obstacles such as job refusals.

Some of the content is similar to that of other job training programmes that currently exist,
yet the style of delivery is very innovative and different. The JOBS programme is very
active and practical, and the training is adequately flexible to allow focus on the needs of
each particular group.

Implementing the JOBS Programme in Ireland
In Ireland, the JOBS programme was piloted as part of the Cross-Border Rural Mental
Health Project (Barry, 2003; Byrne & Barry, 2001; Reynolds, Byrne & Barry, 2004) and
evaluated by the Centre for Health Promotion Studies at the National University of Ireland,
Galway. The Rural Mental Health Project was an action research project concerned with
promoting positive mental health and well-being in rural communities. This project, which
commenced in 1999, was funded under the EU Peace and Reconciliation programme (Peace
2) and part-financed by the UK and Irish Governments. The project brought together two
communities in County Donegal and County Derry/Londonderry in order to develop a
community model of mental health promotion. Therefore, the JOBS programme was piloted
on a regional basis in the cross-border areas of Counties Donegal and Derry/Londonderry
and in rural parts of the North-West including Sligo and Leitrim.

Several planning sessions took place in conjunction with the North Western Health Board in
Donegal, the Homefirst Community Trust in Derry/Londonderry, and the regional training
and employment agencies on both sides of the border. Initial dialogue focused on the
practicalities of the programme and the schedule of the implementation and evaluation. The
Michigan Prevention Research Centre (MPRC) at the University of Michigan agreed to
work collaboratively in replicating the JOBS programme intervention, training the trainers,
and providing data collection instruments for evaluation.

Information Sessions
Two guest speakers provided information sessions in order to give a first-hand account of
the programme as it has been implemented abroad and, thus, lead the planning for the Irish
   •      Dr. Jukka Vuori, who has been extensively involved in implementing the JOBS
          programme on a national scale in Finland
   •      Professor Richard Price from the Michigan Prevention Research Centre at the
          University of Michigan, who was involved in developing the original programme

Information Session with Dr. Jukka Vuori
In April 2002, a one-day information session regarding the JOBS programme was held in
Derry/ Londonderry. Many local training and employment agencies were invited with
approximately 40 people in attendance. Dr. Vuori addressed a number of key points
   •      The programme in Finland had positive outcomes for the long-term unemployed
   •      Full training of the trainers was provided.
   •      Despite some participants remaining unemployed after the programme, their mental
          well-being improved.
   •      Those participants most at risk benefit most from the programme.
   •      There was a low drop out rate among the participants.
A video presentation from the Michigan Prevention Research Centre depicted the JOBS
programme training in action. A question and answer session followed, which helped to
clarify further details about the programme. Finally, participants discussed how the
programme might work in an Irish setting. Everyone that attended this meeting agreed that it
was a very positive experience.
Information Session with Professor Richard Price
In July 2002, a visit by Professor Richard Price from the Michigan Prevention Research
Centre was attended by members of various training and statutory groups. Professor Price
was involved in the original development the programme. He gave a background to the
JOBS programme in the United States since its inception in 1982 and its growing popularity
worldwide. Professor Price and the attendees discussed several topics, including how to
select participants and trainers and the vital importance of research and evaluation to
establish whether the programme works better than other programmes currently offered to
the unemployed.

Agency Involvement
A wide range of statutory, voluntary and community groups who are engaged in training,
employment and support were encouraged to participate in the JOBS programme training.
The agencies that agreed to participate are listed below.
       Action Mental Health
       Cookstown and District Women’s Group
       Cookstown and Dungannon Women’s Aid
       Cookstown and Magherafelt Volunteer Bureau
       Cookstown Training
       Department of Social and Family Affairs
       Department for Work and Pensions
       Foras Áiseanna Saothair (FÁS) - Training and Employment Agency
       Homefirst Community Trust
       Mind Matters Project
       Network Personnel
       North Western Health Board
       National Training and Development Institute (NTDI)
       Oakleaf Rural Community Network
       Rural Health and Social Wellbeing Project
       The Training and Employment Agency
       Vocational Educational Committee (VEC)

Beginning in April 2002, the participating agencies held eleven planning sessions in
Donegal, Derry/Londonderry and Sligo. A number of key issues were discussed and agreed
under the following headings:

   •   Participant selection criteria was discussed and agreed. The planning group
       determined that a group size of 14 - 22 participants was ideal for two facilitators.
   •   The group felt that the best way to promote the programme was by word of mouth,
       which would build up over time.
   •   The planning group felt that there would be low attrition.

   •   The group discussed the issue of joint facilitation of training sessions with both a male
       and female trainer.
   •   Participating organisations agreed to co-operate to bring trainers together to deliver this

The Programme
   •   The planning group suggested that the JOBS programme could be offered as part of
       other existing job-training programmes.
   •   A schedule for the training was discussed. The planning group agreed that there was
       a need to take a long-term view to the implementation of the JOBS programme.
   •   The group agreed that the programme should involve senior management within the
       various organisations to ensure the sustainability.

Training of the Trainers
The training agencies selected a number of their staff to participate in this programme.
Thirty-one trainers in total participated from a diverse range of voluntary, statutory and
community agencies, consisting of 15 trainers from Northern Ireland and 16 trainers from
the Republic of Ireland. These trainers were primarily from training and employment
agencies, the health sector and the relevant government departments. Many of these trainers
have had extensive experience in training, employment and support.

The training of trainers programme took place over six days in both Draperstown, County
Derry/Londonderry and Letterkenny, county Donegal (23rd September–5th October 2002).
IT was run by Mr. Steve Barnaby from the Century Communications Group (part of the
Michigan Prevention Research Centre) and Ms. Sarah Marsh from the University of
Michigan. Both trainers have extensive experience in conducting training of trainers for the
JOBS programme in the United States.

There were high levels of satisfaction with the training for trainers programme. Participants
stated that the success of the JOBS programme would be not so much due to the content, but
more to do with the methods of delivery and learning involved. Participants felt the JOBS
programme was about empowering trainees, giving them a sense of motivation, and
encouraging them to try to gain new opportunities.

Programme Evaluation Materials
Evaluation materials were selected by the Michigan Prevention Research Centre (MPRC) at
the University of Michigan to allow comparison with the US and Finnish JOBS
programmes. These were modified to suit the Irish context and additional process and
qualitative elements were included. Evaluation materials incorporated participants’
questionnaires, trainers’ measures and semi-structured telephone interviews. The evaluation
was undertaken by the Centre for Health Promotion Studies at NUI, Galway.

Piloting the Winning New Jobs Programme
Several trainers completed at least one pilot workshop before the main programme
commenced in order to gain experience running the programme. A researcher from NUI,
Galway observed these sessions to monitor the training at first hand. In addition, all
evaluation materials were piloted. There were no reported difficulties with either the pilot
sessions or the materials.


Participants were recruited through direct contact with the training and employment
agencies and local employment offices in the Republic of Ireland and Northern Ireland.
These individuals were chosen from persons who were registered as unemployed with the
Department of Social and Family Affairs (RoI) and The Training and Employment Agency
(NI). Potential participants were approached by the training agencies, provided with
information on the WNJ programme, and asked if they wished to participate. Mental health
service users were screened for participation in the programme through occupational
guidance services in the health authorities.

The following selection criteria were agreed with the training agencies as to the suitability of
candidates for participation in the study:
   •   Unemployed for at least six months
   •   Age ranging from 18 to 60 years of age
   •   Looking for a job, training or other similar opportunity
   •   Being “opportunity ready”: able to commence employment, voluntary work or

Research Design and Data Collection
This research comprised a quasi-experimental design. In this case, it meant that the training
and employment agencies chose participants for the WNJ programme, while a comparison
group of unemployed individuals who were not participating in the programme was selected
by the NUI, Galway researchers from each of the agencies. Data from the intervention
group and comparison group were compared in order to highlight differences, and therefore
determine the impact of the WNJ programme.

Data were collected from study participants and trainers throughout the study employing
structured questionnaires, which included a range of measures. Participants were asked to
complete four questionnaires at specified periods:
   •   T1 – Baseline questionnaire (all study participants, prior to WNJ workshops)
   •   T2 – End of training questionnaire (intervention group only)
   •   T3 – 2 weeks post training (all study participants)
•   T4 – 4 months post-training (all study participants)

For returning completed questionnaires, participants were entered into a prize draw for €100
/ £75 cash. Participants were guaranteed that the research was confidential and their
participation was anonymous. Local agencies did not have access to any of the participants’
completed questionnaires, as they were posted directly to the NUI, Galway researchers in
the sealed envelopes provided.

The NUI, Galway researcher sent out reminders to the participants in order to increase the
return of the questionnaires. Individuals who did not return the questionnaires in the post
completed some questionnaires voluntarily over the phone.

Figure 1: Schematic Design of the Study

                            Contacted & Agreed to Participate

Comparison Group                                                    Intervention Group

       Not participating in WNJ                    Participating in WNJ                 (T1)
                n=192                                      n=210                     Baseline
                                          General Unemployed        MHS Users
                                                 n=166                n=44

                                                n=146                     n=38          End of

                n=123                           n=102                     n=25          (T3)
                                                                                     2 weeks

                n=120                                                               4 months
                                                n=127                     n=25

Participant Profile
A total of 416 individuals returned baseline questionnaires. 402 valid questionnaires were
used for data analysis: 166 (42.3%) from the general unemployed intervention group, 44

(10.9%) from mental health service users intervention group and 192 (47.8%) from the
comparison group.

The respondents who returned baseline questionnaires varied in age from 15 to 71 years; the
average age was 34 years. Of the total sample, 238 (59.2%) were women and 162 (40.3%)
were men. On their last job, the respondents worked an average of 36.9 hours per week.
The average salary of the participants in their last job was €290 per week. Their average
duration of unemployment was 4.2 years; sixty-four percent of respondents were
unemployed for 12 months or longer.

The baseline sociodemographic and employment profile of the participants is presented in
table on the following page.

Table 1: Sociodemographic and Employment Profile of the WNJ Participants at Baseline
                                           Mental           Intervention     Comparison
                                           Health           Group            Group
                                           Service Users    (n=166)          (n=192)
  Female                                   65.1 %           70.5 %           49.0 %
  Male                                     34.9 %           29.5 %           51.0 %
 Average Age                               34.8 years       34.3 years       32.9 years
                                            min: 17          min: 15          min: 18
                                            max: 71          max: 63          max: 60
 Marital Status
  Married                                  35.0 %           26.1 %           28.6 %
  Separated                                2.5 %            7.9 %            4.9 %
  Cohabiting                               -                5.5 %            4.4 %
  Divorced                                 2.5 %            3.6 %            3.3 %
  Widowed                                  -                1.2 %            1.1 %
  Single/ Never Married                    60.0 %           55.8 %           57.7 %
 Socioeconomic Group
  Professional                             -                0.7 %            1.4 %
  Managerial & Technical                   9.1 %            14.6 %           13.7 %
  Non-Manual                               18.2 %           24.8 %           28.1 %
  Skilled manual                           27.3 %           27.0 %           18.0 %
  Semi-skilled                             45.5 %           25.5 %           36.7 %
  Unskilled                                -                1.5 %            2.2 %
  Other / Not applicable                   -                5.8 %            -
  Roman Catholic                           75.0 %           80.8 %           82.4 %
  Protestant Church of Ireland             15.0 %           10.9 %           5.1 %
  Other                                    10.0 %           3.8 %            1.1 %
  None                                     -                4.5 %            11.4 %
 Level of Education
  No schooling / primary school only       5.1 %            8.0 %            6.1 %
  Some secondary education                 23.1 %           31.5 %           28.2 %
  Complete secondary education             35.9 %           34.6 %           29.3 %
  Some third level education               25.6 %           13.6 %           18.8 %
  Completed third level education          10.3 %           12.3 %           17.7 %
 Avg length of time in previous job        5.03 years       5.05 years       5.20 years
 Avg weekly wage rate in previous job      € 262.03         € 257.04         € 324.89
 Avg hours worked weekly in previous job   36.69 hours      35.56 hours      37.97 hours
 Avg length of unemployment                6.40 years       4.97 years       3.09 years
                                            min: 5 weeks     min: 1 week      min: 1 week
                                            max: 56 years    max: 25 years    max: 30 years
 Previously attended employment            36.8 %           33.1 %           35.4 %
 preparation or jobs training programmes
 Avg weekly take-home family income
   Less than €150 / £100                   21.4 %           25.2 %           21.9 %
   €150 - €300 / £100 - £200               31.0 %           29.5 %           36.0 %
   €300 - €450 / £200 - £300               14.3 %           7.2 %            12.2 %
   €450 - €600 / £300 - £400               7.1 %            9.4 %            9.8 %
   €600 - €750 / £400 - £500               -                0.7 %            1.8 %
   €750 - €900 / £500 - £600               2.4 %            1.4 %            1.2 %
   €900 + / £600 +                         -                1.4 %            0.6 %
   Don't know / prefer not to answer       23.8 %           25.2 %           16.5 %

Intervention Commencement
Those allocated to the intervention group attended the prescribed programme, which
consisted of five intensive half-day workshops (20 hours) held over a one-week period. Two
trainers, one male and one female if possible, delivered the training.

The workshops were based in health board centres and centres of the training and
employment agencies. Twenty-four workshops (12 in Northern Ireland and 12 in the
Republic of Ireland) were held over a period of 18 months. According to trainer
   •   Each workshop session lasted an average 3.39 hours (SD: 0.699)
   •   An average of nine people attended each session (SD: 3.890, range: 2 to 22).
   •   On average, 71% of respondents attended the sessions on a voluntary basis.

Questionnaires included measures of sociodemographic characteristics, work and
employment history, job-search intensity, reemployment, and mental health.

Demographics, assessed at T1, included gender, age, education, family income and personal
income, marital status, religious background, number of dependent children and the number
of people that are financially dependent on the respondent. Date of birth was asked in all
subsequent questionnaires to match participants’ answers with those in the T1

Past Employment and its Quality was evaluated at T1 and included previous occupation,
length of tenure, number of hours worked, the wage rate and length of time since leaving job
(in weeks, months or years).

Reemployment Outcomes were evaluated in follow-up questionnaires at T3, and T4. They
were determined by the question “What is your employment status now?” If the participants
were employed, they were asked the number of hours they work (per day, week or month).
As a measure of quality of reemployment, respondents were asked their wage rate, which
included options for the amount earned per hour, week, month or year. In addition,
respondents were asked whether they had applied to pursue further training or education, as
these may be considered viable alternatives to re-employment.

Physical and Mental Health Measures were measured at T1, T3, and T4. General Health
was assessed by asking participants to rate their general health on a single five-point scale.
Depression was measured with a subscale of 11 items based on the Hopkins Symptom
Checklist. This required respondents to indicate how much they have been bothered or
distressed by various symptoms in the last two weeks including crying easily, blaming
yourself for things and feeling down or blue.

Job Search Intensity Measures were evaluated at T1, T3, and T4. These included the
    1) Job Seeking Efficacy - Respondents were asked to rate how confident they felt about
       being able to perform job seeking tasks including: making a list of the skills that they
       have; talking to friends and others to find out about potential employers and job
       openings; and completing a CV.
    2) Sense of Mastery - Respondents were asked how strongly they agreed or disagreed
       with seven statements regarding their sense of control and whether they felt their
       control was internal or external. Statements included “There is really no way I can
       solve some of the problems I have” and “What happens to me in the future mostly
       depends on me”.
    3) Inoculation against setbacks - This examined how the programme promoted job-
       search behaviour. Four statements assessed whether individuals anticipated possible
       setbacks or barriers in their job search, and how prepared they were to overcome
    4) Economic hardship – This was measured with three statements whereby respondents
       rated their current and anticipated economic hardship.
    5) Job Search Activities – This scale examined respondents’ frequency of job-search
       activities. The five questions included looking for vacancies at the local
       employment office, following newspaper advertisements, contacting employers, and
       considering other possibilities for employment such as starting one’s own business.
    6) Job Search Motivation - This measure looked at subjective norms, attitudes, and
       intention to engage in job seeking within the next four months. Subjective norms
       assessed what participants’ spouses and friends think about how hard they should try
       to get a job. Attitudes toward job seeking measured how useful, beneficial, and wise

it was to try hard to get a job. Intention to engage in job seeking assessed how
       intensely respondents intended to find a job.

Integrity of the Intervention was assessed via the T2 questionnaire, which was administered
only to programme participants. This evaluated the immediate impact of the programme on
participants and included the following elements:
   1) Facilitative Learning Process – These questions examined participants’ feelings and
       reactions overall during the training including their perceptions of the trainers.
       Examples of questions used include “how much have you felt free to participate and
       ask questions?” and “how much did the trainers show that they valued your
   2) Referent Power of Trainers – This investigated the influence of the trainers on the
       respondents. Pairs of words (both positive and negative) were presented, for
       example, “supportive / unsupportive” and “warm / cold”. Respondents chose the
       degree to which each of these words described the trainers.
   3) Attraction to the Group – Similar to questions asked concerning the referent power
       of trainers, these questions examined respondents’ feelings about others participating
       in the WNJ training. Pairs of words were used again to describe the group as a
       whole, and respondents chose the degree to which each of these words described the
       group members.
   4) JOBS-likeness – This last section investigated how well the sessions followed the
       prescribed JOBS training programme, looking specifically at how frequently certain
       activities occurred during the sessions. These activities included active learning
       methods (e.g. often the individual participated in role-plays, discussions and group
       work), trainers’ activities (e.g. how often trainers provided examples and encouraged
       participation) and the individual’s practice of skills to search for jobs and inoculate
       against setbacks (e.g. composing a successful CV).

Trainers’ Questionnaires
Each trainer completed a short questionnaire immediately after each workshop session.
Questions dealt with the trainer’s experience with the workshop, an overall rating on how
well the session went, the programme components that were covered as well as the
attendance rates of the participants. Recommendations were requested so that suggestions
could be incorporated into the training.


Feedback from Trainers
An NUI, Galway researcher conducted semi-structured telephone interviews with a sub-
sample of the trainers to establish trainers’ views of the WNJ programme. The interviews
covered the following themes: trainers’ opinions of how the programme compares to other
job training programmes they have delivered; suggestions for how the training could be
improved; changes they noticed in the participants because of their experiences in the
programme; ways their experience varied in delivering the programme and the factors that
influenced this; and the major challenges in delivering the programme.

Researcher Observations
The NUI, Galway researcher observed a sample of the workshop sessions to examine the
extent to which the programme was delivered as intended (“programme fidelity”). Details of
the programme were documented and the process of implementation was analysed to
examine how it relates to the programme impact and outcomes. Topics included:
   •   how the session went
   •   how participants’ job seeking skills changed
   •   how participants’ self-esteem and confidence changed
   •   how the trainers presented the information
   •   how discussions were conducted
   •   how the group worked together.

Data Analysis
Both regression analysis and descriptive statistics were used to examine the quantitative data
from the questionnaires. Regression analysis is one of the most useful and commonly
reported data analysis techniques. It is used to investigate how much certain “independent
variables” (IVs), assessed at baseline, are able to predict a specific outcome or “dependent
variable” (DV). In this case, regression analysis was employed to predict re-employment,
mental health, and related outcomes (DV) at four months post-intervention using a number
of independent variables (IVs) from the baseline including: intervention vs. comparison
group, demographics, and psychological variables. Of particular interest in assessing the
effects of the programme was the variable relating to intervention or comparison group
impact on the four month outcomes.
The findings of the regression analysis are presented in Table 2 on the following page.
Significant results are highlighted, and will be discussed in the Results section.

Note: The mental health service users were not included in the regression analysis as their
overall profile was different from that of both the intervention and comparison groups,
which could bias the overall regression outcomes.

Table 2: Logistic Regression and Multiple Regression Coefficients (ß) of the Effects on the WNJ Intervention, Demographics, Baseline
(T1) Psychological Factors on Reemployment, General and Mental Health Outcomes, and Psychological Variables at 4-month (T4)

Baseline Predictor              T4 Employment Outcome            T4 General and                           T4 Psychological Variables
                                                                  Mental Health
                               Re-              Job Seeking   General    Depressive Sense of      Job Search Economic       Job Seeking    Inoculation
                               employment       Activities    Health     Symptoms Mastery         Motivation Hardship       Efficacy
Jobs Intervention (0=intrv,    -.151†           -.044         .101       -.119†     -.010         -.117      .121           -.183**        -.163†
Age (0=

Because the intervention in Ireland was designed to replicate both the Finnish and American
programmes, it was important to examine if the Irish programme produced similar main
effects. Based on the main objectives of the Irish Wining New Jobs programme, as well as
the US and Finnish studies, a number of specific research questions were explored.

Question 1: Was the Winning New Jobs programme implemented successfully in an Irish

Question 2: Do programme participants, trainers, and researchers receive the WNJ
programme positively?

Question 3: Does the WNJ programme improve re-employment outcomes, including training
and further educational opportunities?

Question 4: Does the WNJ programme have a positive impact on mental health outcomes?

Question 5: Does the WNJ programme improve psychological variables relating to job
search intensity (sense of mastery, job search motivation, economic hardship, job seeking
efficacy, and inoculation against setbacks)?

Question 6: Does length of unemployment at baseline impact on the effects of the WNJ

Question 7: Do depressive symptoms at baseline impact on the effects of the WNJ
programme on both reemployment and mental health outcomes?

Question 8: Does the level of job search intensity impact reemployment outcomes?


Question 1: Was the Winning New Jobs programme implemented successfully in an
Irish context?

The T2 questionnaire, administered to WNJ participants at the end of the programme,
assessed how closely the workshops followed the prescribed manual for the programme (the
“JOBS-likeness scale”). Overall, there was a very high comparability, as the majority of
respondents reported taking part in JOBS recommended activities “once a day” or “many
times a day”:

Table 3: JOBS-Likeness Scale, as expressed in percentages for those responding “once a
day” or “many times a day”

    Statement: “How often did you…”                                         General      MHS
                                                                            Unemployed   Users
                                                                            n=146        n=38
    Practice in pairs                                                       84.2         66.7
    Work in small groups (3-5 persons)                                      73.8         75.8
    Take part in role-play                                                  77.8         63.6
    Discuss your experiences                                                93.7         78.8
    Trainers return participants’ questions back to the group               90.9         81.8
    Trainers say something positive about the discussion                    92.5         72.7
    Materials appear well organised and clear                               92.3         75.8
    Trainers give reasons why something a participant said was very
    relevant                                                                91.7         93.9
    Trainers refer back to participants' answers later during the group
    work                                                                    89.7         90.9
    Trainers encourage the group, particularly more silent members, to
    participate                                                             88.6         87.9
    Trainers give thanks to speakers in the group                           89.0         90.9
    Other participants help you in understanding your problems              85.4         78.8
    You discover how to find about possible jobs through people you
    know                                                                    79.7         72.7
    Others help you learn how to do a successful job application            81.4         72.8
    You practice how to contact employers to get an interview               80.2         72.7
    Others help you to learn how to do a successful interview               83.5         75.8
    Talk about barriers to presenting your skills & strengths effectively   77.8         69.7
    Talk about barriers to finding out about possible jobs through
    people you know                                                         70.0         69.7
    Talk about barriers to contacting employers successfully                73.4         63.6
    Talk about barriers to doing a successful job interview                 73.9         60.6

On average, 82.5 % of the General Unemployed participants and 75.8 % of Mental Health
Service User participants found that these programme elements were included in workshops
at least once a day.

Question 2: Do programme participants, trainers, and researchers receive the WNJ
programme positively?

a) Participants’ Reaction to the Programme
Results from the T2 questionnaire demonstrate that participants responded very positively to
the programme, the trainers, and other group members. According to the Facilitative
Learning Scale, overall, 77% of participants felt that the WNJ programme was very (“pretty
much” or “a great deal”) beneficial, as noted in the table below.

Table 4: Facilitative Learning Scale, as expressed in percentages
                                                       General Unemployed                 MHS Users

                                                     Pretty much /   Not at all   Pretty much /   Not at all
                                                      a great deal                 a great deal
    I was free to participate and ask questions      81.4            0            84.9            0
    Others listened to what I had to say             81.5            0            90.9            0
    Other participants shared information            71.9            0.7          72.7            0
    Trainers shared information about their          81.3            0.7                          0
    background                                                                    75.8
    I felt I myself shared information               49.3            3.4          69.7            3.0
    I felt other participants had similar problems   61.0            4.1                          0
    to me                                                                         48.5
    The material and the discussion were             78.8            6.2                          0
    relevant                                                                      75.8
    The material and discussion were well-           79.4            2.7                          0
    organised & clear                                                             93.9
    I feel enthusiastic about job hunting            73.8            1.4          69.7            0
    I feel enthusiastic about the future             50.3            26.5         75.7            0
    I felt confused by the discussion                2.7             85.0         3.0             66.7
    I was embarrassed by what I said                 2.0             84.3         3.0             54.5
    I was interrupted when I spoke                   29.3            60.5         3.0             84.8
    Trainers ignored me                              24.5            64.6         0.0             97.0
    Trainers made me feel upset                      19.7            65.3         0.0             93.9
    Trainers show they valued my participation       61.9            30.6         100.0           0
    Trainers make me feel happy                      47.6            34.7         90.9            0
    Trainers made me feel enthusiastic               75.2            5.5          75.7            0
    Trainers showed an understanding of my           87.1            4.8                          0
    problems                                                                      87.9
    Trainers showed I have an improved chance        76.8            4.1                          0
    of getting a job                                                              87.9
    I think I could be good at things the trainers   76.9            2.7                          0
    discussed                                                                     66.6

Next, participants were asked to rate the trainers on a seven point scale i.e. very supportive to
very hostile, very helpful to very unhelpful. This was in order to determine the Referent
Power of Trainers. As per the table below, more than 75% of respondents felt positively
about the trainers, checking off that they were “very” or “somewhat” for the positive

adjective. It is notable that the vast majority of mental health service users felt positively
towards the trainers:

Table 5: Referent Power of Trainers Scale- Percentage of respondents reporting “very” or
“somewhat” in regards to positive comments towards trainers
                                   General Unemployed         MHS Users
    Supportive                     83.4                    96.9
    Sympathetic                    77.9                    93.8
    Enthusiastic                   84.0                    96.9
    Knowledgeable                  84.8                    96.9
    Warm                           82.1                    96.9
    Helpful                        82.8                    96.9
    Sincere                        84.0                    96.9
    Communicating clearly          85.7                    93.8
    Accepting                      79.6                    96.9

Similar to questions in regards to the trainers, the Attraction to Group was measured by
asking participants their feelings towards the group as a whole. As per the table below, the
majority of intervention participants felt positively towards the group, with the vast majority
of mental health service users responding very positively:

Table 6: Attraction to the Group Scale - Percentage of respondents reporting “very” or
“somewhat” in regards to positive comments towards the group as a whole
                                    General Unemployed         MHS Users
    Supportive                     75.9                     93.6
    Warm                           79.7                     96.7
    Sincere                        76.0                     93.5
    Accepting                      79.9                     93.5
    Likeable                       60.7                     96.7

b) Trainers Reaction to the Programme
Overall, the 39 trainers in the WNJ programme who completed questionnaires on the
workshops reported positive feedback towards each of the sessions. The majority found each
component “extremely satisfactory” or “very satisfactory”.

Table 7: Percentage of trainers reporting “extremely satisfactory” or “very satisfactory” in
regards to individual workshop measures.
   Workshop measure                                         Trainers
   How satisfactory was the session?                        77.7 %
   How did the exercises work?                              77.3 %
   How responsive were the participants?                    69.6 %
   How adequate was the training?                           68.9 %
   How comfortable were you with your role?                 83.6 %
   How difficult was it to follow the manual?               53.6 %
   How difficult was it to integrate the programmer into
   existing programmes?                                     67.9 %

c) Independent Researcher’s Observation of the Programme
The researcher from the National University of Ireland, Galway independently observed each
workshop session from two different workshop groups in order to record impartial
observations of workshop content, trainers, participants, and dynamics of the session.

Employing a structured observation form, the researcher reported the following observations:
  •   Made critical or negative remarks about participants: never any extent (100%)
  •   Made insensitive or thoughtless comments to individual participants: never any extent
  •   Exhibited personal care and unconditional positive regard for participants: a very great
      extent (75%) / a great extent (25%)
  •   Displayed enthusiasm for the materials: a very great extent (75%) / a great extent (25%)
  •   Seemed to appreciate participants’ comments and ideas: a very great extent (75%) / a great
      extent (25%)
  •   Presented useful information and discuss applications: a very great extent (75%) / a great
      extent (25%)
  •   Generated discussions that were spontaneous and unhindered: some extent (75%) / a great
      extent (25%)
  •   Had discussions that seemed to be confusing and take long diversions: never any extent
      (75%) / a little extent (25%)

The researcher reported that trainers gave positive feedback towards the group as a whole,
indicating that the participants:
  •   Created a friendly, congenial and supportive atmosphere: some extent (25%) / a great
      extent (25%) / a very great extent (50%)
  •   Created a feeling of mutual appreciation and respect among the members: some extent
      (25%) / a great extent (25%) / a very great extent (50%)
  •   Created a sense of working together toward a common goal: a little extent (25%) / a great
      extent (25%) / a very great extent (50%)
  •   Brought about interpersonal conflict during discussions: never any extent (100%)
  •   Actively sought information, evaluations and comments from one another: a little extent
  •   Pushed their own opinion rather than listen to what others had to offer: never any extent

Question 3: Does the WNJ programme improve re-employment outcomes, including
training and further educational opportunities?

At four months follow up, 31% of the intervention group, 24% of the mental health service
users group, and 21% of the comparison group reported being re-employed.

Figure 2: Employment Status over Time – WNJ Participants Employed
                                                     Intervention Group
                       20.9               23.5       Comparison Group
         20                   20.0        20.6       MHS Users
         15                   16.8

              T1         T3          T4

The figure above depicts how the WNJ programme had a beneficial impact on
reemployment. This finding was reinforced through regression analysis, which indicated that
the intervention had an impact approaching significance (ß=-.151, p
Figure 3: Average Wage Rate (weekly)
                                                              Intervention Group
                                                              Comparison Group
                                                              MHS Users
                             T1    T3         T4

Figure 4: Average Hours Worked (weekly)
  38           38
  36          36                                                 Intervention Group
  34                                                             Comparison Group

  32                                                             MHS Users

  28                                                    28
  24                                   25
                        T1         T3              T4

The data suggests that participants who achieved employment at T3 and T4 re-entered the
workforce on a part-time basis, as their hours and wages dropped dramatically from their
previous employment, noted at T1. To analyse further, the average wage rate per hour was
calculated for those persons who reported hours and wages at T1, T3 and T4 (n=15).
However, it should be noted that the sample size for this calculation is quite small, and too
few mental health service users reported this information to be included in the comparison.

Figure 5: Average Wage Rate per Hour (in euros), respondents reporting at T1, T3 and T4
                    9             9                9
   9                                                          Intervention Group
   8                8                                         Comparison Group
                        T1        T3          T4

The figure above suggests that the intervention group may enjoy a better quality of
reemployment than the comparison group, as the hourly wage rate increased notably at two
weeks post-intervention (T3). Similar to the re-employment rate figures, the quality of
employment may also increase though it will take a longer time period to see this effect in
full evidence.

Figure 6: Job Search Activities Scale (1 few activities through 4 many activities)
                                                                    Intervention Group
  2.0                                               2.0
                                                    1.9             Comparison Group
  1.8             1.9
                                                                    MHS Users
  1.6                         1.7               1.7

  1.4         1.5


             T1               T3              T4

In terms of job search activities, such as asking friends about job openings and checking
employment ads in newspapers, the figure above illustrates that the comparison group’s
activities decreased, the intervention group’s activities remained largely the same and the
MHS Users group’s activities increased slightly. Regression analysis determined that this
effect was not significant.

Figure 7: Percentage of respondents who applied for further training or educational
in the last four months
  40.0                                       40.0
                                                          Intervention Group
  35.0                                       34.8
                                                          Comparison Group
  30.0       29.9
                                             27.3         MHS Users
  25.0       26.4
                        T1              T4

Figure 7 (above) demonstrates that an almost equal number of intervention and comparison
group participants applied for further training or educational opportunities within the four
months following the T1 baseline questionnaire. A clearer difference between the groups
may become evident with longer-term follow-up.
Question 4: Does the WNJ programme have a positive impact on mental health

The Depression Scale, illustrated below, shows that the depression levels of the intervention
group and the comparison group remained virtually unchanged across time. The mental
health service users had a higher level of depression overall.

Figure 8: Depression Scale (1 low depression, 5 extreme depression)
  2.2             2.1                                               Intervention Group
  2.0                           1.9                    2.0
                                                       1.9          Comparison Group
  1.8       1.8
                                 1.8                                MHS Users
  1.6                                                  1.7
              T1                T3                T4

Regression analysis indicated that the WNJ intervention approached significance in affecting
depressive symptoms (ß=-.119, p
Figure 9: Job Seeking Efficacy Scale (1 low efficacy, 5 high efficacy)
  3.4                                                      3.4
  3.3                                                                 Intervention Group
                               3.3                         3.3
                                                                      Comparison Group
                 3.2                                                  MHS Users
  3.1                                                      3.1

  2.9                 2.9      2.9
                 T1            T3                    T4

The figure above illustrates how the WNJ programme had a strong effect on Job Seeking
Efficacy (i.e. confidence to perform job seeking tasks), as efficacy levels for the intervention
group and the mental health service users intervention group increased over time, while the
comparison group remained virtually unchanged. This finding was reinforced through
regression analysis, which determined that intervention group was significantly more likely
to have higher job seeking efficacy than the comparison group at four months follow-up (ß=-
.183, p
Figure 11: Mastery Scale (1 low mastery, 4 high mastery)
  3.0                                               3.0
  3.0                            2.9                                Intervention Group
                                                                    Comparison Group
                                   2.8              2.8             MHS Users
  2.8     2.8
                T1                T3           T4

The figure above indicates that the level of mastery (i.e. sense of control over problems)
remained basically unchanged for the intervention and comparison groups; however, the
level of mastery steadily increased, albeit by a small amount, in the mental health service
users. Regression analysis found that a relationship between the WNJ programme and a
sense of mastery four months post-intervention was not significant.

Figure 12: Job Search Motivation Scale (1 little motivation, 5 high motivation)
  4.2                                                                 Intervention Group

  4.1                                         4.1                     Comparison Group
                                                                      MHS Users
  4.0                      4.0
  3.9                                         3.9

                      T3                 T4

Job Search Motivation (i.e. how hard they believe they should try to gain employment), as
depicted above, was measured at two weeks post-intervention and four months post-
intervention only. Note that the results for the intervention group and the MHS Users was the
same, hence both are represented by the pink bar. It appears as though job search motivation
for the intervention group and the MHS Users increased over time, while motivation in the
comparison group decreased. Regression analysis, however, did not indicate that this result
was significant.

Figure 13: Economic Hardship Means Scale (1 little hardship, 5 extreme hardship)
              2.7              2.7
  2.6                                                               Intervention Group
                                               2.6                  Comparison Group
  2.5                                                2.5
                                2.4                                 MHS Users
  2.3              2.3

  2.1                                                2.1
              T1              T3               T4

The Economic Hardship Scale, illustrated above, shows that the intervention group’s
perception of hardship dropped at two weeks post-training (T3) – this could be due to
immediate “afterglow” effects of the training, inspiring a positive outlook towards obtaining
employment and a perceived reduction in financial hardship. Regression analysis, however,
did not indicate that the results were significant.

Question 6: Does length of unemployment at baseline impact on the effects of the WNJ

Short-term unemployed (
In terms of psychological variables associated with job search intensity, higher depressive
symptoms at baseline were significantly predictive of a lower sense of mastery (ß=-.367,

As evident in this report, the Winning New Jobs programme was well received by trainers
and participants alike. The preliminary results clearly indicate that trainers and participants
responded positively, and were convinced of the programme’s effectiveness and viability.
This study was novel in that mental health service users were a key group targeted for
intervention, and their response to the programme was very positive. The trainers who
administered the workshops can now pass on their skills and experiences to more trainers, in
order to disseminate the programme to a wider audience.

However, throughout the course of administering the workshops, the trainers made it clear
that certain sections of the original American “JOBS Manual for Teaching People Successful
Job Search Strategies” (1992) had to be altered in order to suit the Irish context. An average
of 45% of trainers (range 31% to 67%) reported adding in or leaving out items from the
JOBS manual in each WNJ workshop session. A key issue that was highlighted by the
trainers and agencies was the use of language and examples more suited to an American
audience. In response, at the end of the workshops a full-day meeting was held in which all
the trainers were invited to attend in order to review the programme and re-write the training
manual. Issues discussed included: what works or does not work with each session; what
needs to be changed, added or left out; format and layout of the manual; and availability of
extra resources and materials. Those who could not attend the meeting were asked for their
contributions by email or telephone. As a result of this meeting, a draft copy of the new
manual was produced and made available to all trainers for feedback. Based on this further
feedback, more revisions were incorporated and the newly adapted Winning New Jobs
manual was completed. This manual has now been published and is available for
distribution. It will serve as the basis for the propagation of the Winning New Jobs
programme in the Republic of Ireland and Northern Ireland.

In summary, the impact of the Winning New Jobs programme for the unemployed may vary
depending on the social and cultural context in which the programme is implemented. It is,
however, clear that the results of this initial pilot of the WNJ programme are positive and
encouraging in an Irish context. The programme has had positive benefits in terms of
improved confidence in job seeking skills, inoculation against setbacks and a trend towards
improved mental health and reemployment. The outcomes and impacts of the Irish
programme are substantiated by the fact that they show many similar effects to those found
in the Finnish Työhön programme.

There were several differences between the Finnish Työhön programme and the Irish
Winning New Jobs programme that must be noted. The sample size in the Finnish study (N=
1,261) was substantially larger than the sample in the Irish study (N= 402), mostly because
the Finnish programme was rolled out nationally and the Irish programme was rolled-out in
only one region. The sample also differed in that there was a random assignment to groups in
Finnish study, while in Ireland the training agencies selected which participants would be
involved in the intervention. The Finnish sample ranged in age from 18-61 years with a
median age of 37; meanwhile, the Irish sample had an average age of 34 with a range from
15-71 years. There was a majority of women in both studies. Most participants in the
Finnish study were short-term unemployed (median duration of unemployment 5 months, M
= 10.7, SD = 17.3) with 28% unemployed for 12 months or longer. In the Irish study, the
average duration of unemployment was 4.2 years with 64% unemployed for 12 months or
longer. Despite these differences, the findings from the Irish study at this stage compare well
with those reported in the Finnish study at six months follow-up. Further analysis of data,
for example a 12 month follow-up, could be used to draw additional conclusions on the long-
term impact of the programme. Finally, the Irish programme may have had other differences
as compared to the Finnish study since each programme was implemented in a vastly
different labour market and social context.

The findings from this pilot implementation suggest that a 20-hour Winning New Jobs
intervention can positively influence both reemployment and mental health outcomes. It
should also be noted that this programme has been successfully implemented with mental
health service users, showing positive outcomes. Such encouraging outcomes support the
call for a large-scale rollout of this programme nationally, as well as a longer-term follow-up
with a larger sample.

You can also read
NEXT SLIDES ... Cancel