SOCIAL INFRASTRUCTURE PPPs - MEASURING THE VALUE AND SERVICE OUTCOMES OF - Infrastructure Commission
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPs IN AUSTRALIA AND NEW ZEALAND
Infrastructure Partnerships Australia Infrastructure Partnerships Australia is an industry think tank and an executive member network, providing research focused on excellence in social and economic infrastructure. We exist to shape public debate and drive reform for the national interest. This research project was commissioned by Infrastructure Partnerships Australia and conducted with the support and sponsorship of governments from Queensland, New South Wales, Victoria and New Zealand. The research and report were completed by University of Melbourne and Drum Advisory, with support from Infrastructure Partnerships Australia. Contacts at Infrastructure Partnerships Australia: Adrian Dwyer Chief Executive Officer Infrastructure Partnerships Australia P +61 2 9152 6000 E adrian.dwyer@infrastructure.org.au Jon Frazer Director, Policy & Research Infrastructure Partnerships Australia P +61 2 9152 6017 E jon.frazer@infrastructure.org.au Hamilton Hayden Senior Policy Adviser Infrastructure Partnerships Australia P +61 2 9152 6018 E hamilton.hayden@infrastructure.org.au
CONTENTS CEO INTRODUCTION 2 HIGHLIGHTS OF THE RESEARCH 3 AUTHORSHIP AND ACKNOWLEDGEMENTS 4 AUTHORS’ NOTE 5 AUTHORS’ OVERVIEW 6 1. INTRODUCTION 8 2. RESEARCH 12 3. FINDINGS 14 4. INSIGHTS AND OBSERVATIONS 23 5. RECOMMENDATIONS 27 APPENDIX A: ACRONYMS AND DEFINITIONS 28 APPENDIX B: DETAILED LITERATURE REVIEW 29 APPENDIX C: ETHICS PROCESS, DATA AND APPROVAL 38 APPENDIX D: INVESTIGATION, SURVEY AND WORKSHOPS 47 APPENDIX E: WORKSHOP SUMMARY COMPARISON 53 APPENDIX F: REFERENCES 59 ENDNOTES 60 MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 1
CEO INTRODUCTION
Australia has been at a global leader in the delivery of I would also like to thank our partners on this project
infrastructure through public-private partnerships. Since for your support – the Treasuries of Queensland, New
the early 1990s, Australian governments have used South Wales, Victoria and New Zealand. This generous
PPPs to deliver services more efficiently and improve support has not only enabled the research to happen,
value for money for taxpayers. Through PPPs in the but also to make it a partnership that reflects the
social infrastructure sector – in particular, schools, projects this research explores. With the support of the
hospitals and justice facilities – Australians have public sector, brought together with the private sector
gained access to innovative and effective infrastructure through Infrastructure Partnerships Australia and its
services, delivered by leading global providers. membership, and filtered through academic objectivity
of the University of Melbourne, this represents a truly
Infrastructure Partnerships Australia has played an collaborative effort.
important role in the history of Australia’s PPPs. As our
name suggests, partnerships are in our DNA. Since The authors of this paper are responsible for its
our establishment in 2005, we have provided a forum words, research and findings. However, the sector
for the public and private sectors to come together owns its outcomes by virtue of having delivered the
– outside the setting of a deal table – to address projects. The findings reflected through this research
critical issues in the sector, and to find common are overwhelmingly a positive story about the success
ground on solutions. Through this paper, we are of social infrastructure PPPs. This success has been
pleased to continue our focus on how the public and built on the foresight, hard work and commitment of
private sectors can partner effectively to the benefit of infrastructure leaders across the public and private
Australian infrastructure users and taxpayers. sectors who have developed and delivered these
assets and services.
After 25 years of PPPs in Australia, this represents an
opportune moment to reflect on the performance of Infrastructure Partnerships Australia have drawn out
this approach to delivering social infrastructure. While some of the most pertinent findings on the following
many within the infrastructure sector recognise the page. The sector should be rightly proud of this
benefits these PPPs have brought, many in the broader track record. I also encourage you to consider the
community may not see or understand the differences recommendations of this paper, and how those
from regular service delivery. developing and delivering the next 25 years of PPPs
can take important lessons from past experience.
That is why this research focuses on the experience
of service providers and users. By using the data
collected over these projects’ histories, and the views
of the frontline staff and users of PPP facilities, this Adrian Dwyer
research aims to assess whether PPPs have lived up CEO, Infrastructure Partnerships Australia
to their promise, and communicate these findings to a
wider audience.
It is important for this research to be independent
and objective. That is why Infrastructure Partnerships
Australia commissioned the University of Melbourne
to undertake this work. We are grateful to Professor
Colin Duffield and Dr Ali Mohammed Saeed for
their authorship of this paper, underpinned by
objective analysis of projects across Australia and
New Zealand, and made possible by those who
participated in the study.
2 INFRASTRUCTURE PARTNERSHIPS AUSTRALIAHIGHLIGHTS OF THE RESEARCH
Following engagement with service providers of service providers with the product and services
across Queensland, New South Wales, Victoria and being received. Effectively, paying more meant higher
New Zealand, service providers confirmed (95 per levels of services.
cent) that PPP projects do deliver on the service as
promised by government and departments in media
140%
releases, community information documents and
public meetings. All service providers reported that 120%
% concentrated price to PSC
the PPP projects investigated opened for service to 100%
the community on-time, and since that time, they have
80%
performed better than the traditional model.
60%
A clear and overwhelming preference (95 per cent) 40%
was found among service providers for working within
a PPP facility over that of the traditional government- 20%
owned and operated facility. Some respondents also 0%
0 2 4 6 8 10 12 14 16
indicated that experience in the PPP model was a Years since operation commenced
significant career advantage.
The level of satisfaction with the quality of service Clearly, this shows that PPPs in Australia and New
delivered remained high through the years of operation Zealand are delivering on their promise. Governments
investigated. The ongoing high levels of satisfaction should continue to consider and use the PPP model
suggest that the PPP model was a successful means for social infrastructure service delivery as a way of
of achieving and maintaining positive outcomes. bringing greater benefits to service providers and
users, and better value for taxpayers.
5
4
Satisfaction score
3
2
1
0
0 2 4 6 8 10 12 14 16
Years of operation
Service providers were of the view that the PPP
projects were delivering value for money (VFM), but
also constructively identified opportunities for evolution
and continued improvement in the PPP model.
The researchers’ analysis of project documentation
showed that the VFM originally evaluated by
government as a part of the tender process has been
maintained throughout the operating phase of the
PPP agreement with no evidence of price creep nor
of risk transfer back to government. This analysis
demonstrated that 10 out of the 12 case study projects
investigated met or bettered government estimates as
expressed within the Public Sector Comparator (PSC).
Interesting, where the PSC had not been bettered,
there appeared to be a heightened level of satisfaction
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 3AUTHORSHIP AND ACKNOWLEDGEMENTS
Authorship Acknowledgements
This research and report involved a collaborative The research team acknowledges the contributions
effort between The University of Melbourne and Drum of public sector organisations and individuals who
Advisory, with support from Infrastructure Partnerships provided information and freely gave of their time,
Australia. The research team consisted of: experiences and insights for this research. We
acknowledge the contribution of officers from the
Professor Colin Duffield Treasuries of Queensland, New South Wales, Victoria
Department of Infrastructure Engineering and New Zealand, their colleagues from related service
University of Melbourne agencies and officers with responsibility for managing
P +61 383 446 787 ongoing PPP contracts. We also acknowledge the
E colinfd@unimelb.edu.au contribution of the service providers of PPP social
infrastructure projects (the ‘service providers’) for their
Dr Ali Mohammed Saeed
candour and enthusiasm in sharing their views.
Department of Infrastructure Engineering
University of Melbourne The Research Team expresses their appreciation
P +61 403 091 640 to Infrastructure Partnerships Australia and its
E ali.saeed@unimelb.edu.au membership organisations that championed this
research and sought an independent evidence-based
Mr Nick Tamburro
outcome. Special thanks to Nick Hudson for his
Principal
enthusiastic and valuable stewardship of this project
Drum Advisory
in its early stages and later to Hamilton Hayden for his
P +61 409 970 053
assistance in finalising it.
E nick.tamburro@drumadvisory.com
4 UNIVERSITY OF MELBOURNEAUTHORS’ NOTE
There have been many reports published of research the promised uplift in service benefits, advertised by
into Public Private Partnerships (PPPs), focusing the proponents of a PPP social infrastructure project
on matters such as project delivery, risk allocation, to the service providers and the wider community,
financing and time or cost outcomes. For Australian have been achieved.
and New Zealand PPP practices, the findings of these
studies1 demonstrate that PPPs reduce the likelihood This is one of the first in-depth research projects
of time and cost escalation on major projects during investigating the operating performance of social PPP
the design and construction phase. projects from the perspective of the service providers
in Australia and New Zealand. Service providers
The operating performance of mature PPPs as in social infrastructure, compared to economic
experienced by service providers is less well infrastructure, are more likely to physically work in the
understood. While PPP projects have been operating PPP facility over the long term and through full-time
in Australia for some 25 years, there is a lack of employment. Moreover, these service providers are
independent research on the operating performance of more likely to become committed, and in some cases
PPPs in meeting the service objectives of governments emotionally attached, to the PPP facility and its service
and their communities. provision. For example, a school principal, by nature of
their work, may be attuned to the effect the classroom
The term ‘service provider’ is defined in this report as layout and upkeep has on student behaviour, whereas
meaning those employees utilising the PPP capital a toll-road operator, analysing traffic data remotely,
assets to deliver services to their client community is more likely to have a transactional and somewhat
members. They may include school principals, simple fee-for-service relationship.
doctors, wardens, administrative or management staff.
In some PPP models those employees are from the During the authors’ extensive contact with service
public sector and in other cases they are employed providers, many expressed an appreciation of
through the PPP consortium. research that focused on them. They were universally
enthusiastic in engaging with the workshops and
This research investigates whether mature operating providing their experiences and insights with a view to
PPPs are meeting the service delivery outcomes improving future PPP projects, and thereby improving
expected by service providers. It assessed whether the services outcomes to their client communities.
Professor Colin Duffield Dr Ali Mohammed Saeed Mr Nick Tamburro
Department of Infrastructure Department of Infrastructure Principal
Engineering Engineering Drum Advisory
University of Melbourne University of Melbourne
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 5AUTHORS’ OVERVIEW
This research considers whether mature PPP social The following key findings and recommendations
infrastructure projects are meeting the service delivery were made.
outcomes expected by users of the facility. It assesses
whether the uplift in service benefits, promised by the
proponents of the PPP model to service providers,2 Finding 1
and in turn their client community, have been achieved.
Service providers (95 per cent4) stated that
This research and report were commissioned by their PPP project has delivered on the service
Infrastructure Partnerships Australia and conducted promised by the relevant state government and
with the support and sponsorship of jurisdictions delivery agency.5
from Queensland, New South Wales, Victoria and
New Zealand.
The scope of the research was to:
Finding 2
1. assess whether mature social infrastructure PPPs
are meeting the service delivery outcomes for Service providers (95 per cent6) and all contract
service provider and contract manager3 groups set managers participating in the research prefer
out in contractual agreements, media releases and working in a PPP facility and service contract
other community information documents over a traditional government-owned and
operated facility.
2. compare, where data is available, service provider
and contract manager satisfaction with PPP assets
and service delivery to that of traditionally procured
and delivered assets and services
Finding 3
3. identify what factors contribute to positive service
provider satisfaction in PPPs and what factors can Service providers (82 per cent7) expressed a
be attributed to poor service provider experiences strong appreciation of the quality of services
4. assess whether value for money (VFM) is provided by the Facility Management (FM)8
maintained over the long-term operating phase of operator in a PPP facility. Satisfaction level with
social infrastructure PPP facilities, and service quality is strongly influenced by the
experience level and relationship between service
5. provide recommendations for future PPP projects.
providers, contract managers and FM operators.
The research was underpinned by a review of current
literature on trends and use of PPP projects. Specific
project documentation was gathered from participating
jurisdictions and a survey of government contract Finding 4
managers and service providers was undertaken.
All service providers agreed that the PPP
The research concluded with workshops (and post
model provides similar flexibility provisions to
workshop surveys) in Sydney, Brisbane, Melbourne
traditional procurement models.
and Auckland from June to August 2019. The research
covered 12 social infrastructure PPP contracts in
operation from about three to 15 years, involved
11 contract managers and 28 service providers in
workshops, and a total of 58 respondents to surveys.
6 UNIVERSITY OF MELBOURNEFinding 5 Recommendation 1
Service providers and contract managers are a Governments should continue to secure, where
valuable and underutilised source of improvement supported by business case analysis, new
initiatives for future PPPs. There is room to PPP contracts for service providers and their
incorporate their operational phase experience communities to meet current and future social
into the planning phase of new projects. infrastructure needs.
The planning of future PPP projects can be
improved to help achieve better outcomes by:
• involving contract managers in the early Recommendation 2
stages of the procurement process to identify
and assess additional benefits associated Decision-makers in both public delivery
with the choice of a procurement model agencies and PPP proponents should give
a significant voice to service providers and
• engaging with additional service providers
contract managers during the preparation of
during bidding and design refinement
tender documents and while designing and
phases of the project
planning the PPP project.
• improving contract management practices
to ensure a consistent level of expertise
among contract managers within and across
state governments
Recommendation 3
• ensuring that all contracts provide flexibility
provisions to manage future changes as the Both public delivery agencies and PPP
community’s service needs evolve, and proponents should be challenged to evolve
the PPP contractual terms to further focus on
• building strategies in the PPP arrangements
outcome-based service definition, with fewer
that promote open and smooth
prescriptive Key Performance Indicators (KPIs),
communications between service providers
to provide the long-term flexibility service
and FM operators during operational phase.
providers need to best serve their communities.
Finding 6
Recommendation 4
PPP facilities maintain value for money over the
Service provider representatives should be
long term. There was no evidence of price creep
educated on the basics of their PPP agreement
or risk transfer back to the public sector during
and the roles and responsibilities of the various
the operational phase of the case study projects.
PPP parties. Contract management staff and
the FM providers would also benefit from an
education program encouraging continuous
improvement and maximising value through the
Finding 7 partnership aspects of the contract.
Service providers are poorly informed about
the difference between PPP and traditionally
procured facilities, restricting their effectiveness
in the PPP facility. Recommendation 5
Both government and the private sector should
improve the consistency of good communication
in their dealings and in the day-to-day operations
impacting on service providers.
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 71. INTRODUCTION
1.1 Purpose of the research infrastructure offer a particularly rich opportunity to
assess the service provider experience of the services
The primary focus of this research project was on delivered, especially as PPP projects have been
the experience and satisfaction of service providers9 operating in Australia for 25 years.
working in mature Public Private Partnership (PPP)
projects providing social infrastructure and associated This PPP environment enables a thorough assessment
services. It investigated whether such PPP projects to be undertaken of the service provider’s experience
were operating to meet the service delivery outcomes with mature PPPs that are in a “normalised”
promised by governments and proponents of the PPP operational stage. As highlighted in the Authors’ Note,
projects, and consequently, delivering the service mix service providers in social infrastructure, compared
and performance expected by service providers in to economic infrastructure, are typically employed in
servicing their client communities.10 the long term and on a full-time basis within the PPP
facility. Moreover, given their professional interactions
The research sought the direct views of service providers with their client community, these service providers
on whether the uplift in service benefits to them and their are more likely to become committed, and in some
client community, as promised by governments and cases emotionally attached, to the PPP facility and
proponents to be delivered by the PPP model over the service provision. The research also investigated user
traditional model, have been achieved. satisfaction with PPP assets compared to traditionally
procured facilities.
This research and report seek to contribute to
understanding the value for money (VFM) proposition Recently, some social infrastructure portfolio leaders
offered by the PPP model in social infrastructure and have been critical of PPPs for being too rigid in their
improve planning for future projects. contractual outcomes and this may have had an
impact on PPP take-up in new projects. This research
1.2 Context of the research investigated such concerns with service providers,
seeking to identify the source of such issues and
The focus of this research was on the experiences of what factors can be attributed to positive and poor
those using the PPP facility and its related contracted user experiences.
services to deliver services to their client community.
The focus being on employees (‘service providers’) 1.3 Value for money (VFM)
such as senior executives, managers, administrators,
teachers, clinicians, prison wardens, as they service The importance of public infrastructure procurement
their client community, such as students, patients or being able to achieve VFM11 remains a critical
prisoners. The research does not sample members consideration in business case development and
of these client communities directly, rather it sought the prioritisation of projects across all jurisdictions.
to investigate how the PPP facility and its related While the money in VFM is associated with the costs
contracted services enable and contribute to the of project delivery and operation, value in VFM is
performance of service providers in servicing their associated with measures of benefits including service
client community. delivery outcomes and user experiences.
In some PPP models, the service providers are In the PPP market, it appears that any general media
public sector employees (for example, teachers and criticisms associated with service and user experience
clinicians), in other models they are employed through in a few projects can outweigh the greater number of
the PPP consortium (for example, prison wardens). PPPs that have been delivered on-time and on-budget,
and which operate successfully in meeting contracted
From inception, a fundamental characteristic of Key Performance Indicators (KPIs).12
the PPP model in Australia has been a commercial
structure focusing on achieving defined service Public-Private Partnerships typically deliver projects
outcomes through appropriate allocation of risk, KPIs within cost and time expectations, and the transfer of
and payment mechanism. Conceptually, in the PPP risks to the private sector are real. In many examples
model the capital asset is the enabler of the service the public purse has been protected from cost
outcomes that are measured and if successfully overruns13 and the pressure to perform has PPP
delivered are rewarded by government or users. The projects consistently being delivered on-time or early.
Australian and New Zealand PPP model in social
8 UNIVERSITY OF MELBOURNECurrent literature is inconclusive as to whether VFM Partnerships Victoria body within the Department of
is held over the long-term operational phase of PPP Treasury and Finance that developed some of the most
projects. This research considers whether the pricing comprehensive PPP policies and mechanisms used
received in the initial PPP transaction is maintained by different states throughout Australia (English, 2006).
during the operational phase of the agreement. The first use of modern PPP arrangement in New
Zealand can be traced to Auckland’s Hobsonville Point
1.4 Literature review Primary and Secondary Schools in 2012 (Infrastructure
New Zealand, 2017).
A review of PPP literature was conducted as part this
project. The findings were used to determine the focus The PPP model has evolved over time, with different
of the survey and workshop. For the full literature iterations from jurisdiction to jurisdiction and project
review please refer to Appendix B. to project. However, the ‘core’ characteristics have
remained constant, including:
Australia and New Zealand are considered mature
• the involvement of private finance
PPP markets by global standards. While the UK
may be credited as the birthplace of modern PPP • bundling of construction and operation into one
arrangements, governments across the world contract, and
have used a mix of such public and private sector • the use of contracts and risk allocation to align
arrangements to deliver projects throughout history private profit incentives with public service provision.
(Wettenhall, 2005).
Numerous studies examining PPP projects in Australia
Australian PPPs can be broadly classified into two time and the UK are available, however there is a limited
periods: pre-2000 and post-2000. The modern PPP publicly available analysis of New Zealand social
arrangements post-2000s can be accredited to the infrastructure PPP projects.
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 9Various international studies conclude that the PPP typically retained responsibility for the actual delivery
procurement model has stronger incentives to of public services to the community, with the private
minimise whole-of-life costs and improve service quality sector ‘owning’16 and financing capital facilities and
outcomes than traditional procurement approaches providing support services like facilities management
like Design and Construct or Construct-Only models and sometimes cleaning. The general contractual
(Hodge and Greve 2017, Saeed 2018). However, phase used for this style of PPP is ‘Design, Build,
studies have also found limitations of the PPP model Finance and Maintain’ (DBFM). It is this style of PPP
in regard to risk allocation, innovation and operational project that this study concentrates on and a full list of
flexibility (Hodge and Greve 2017, Saeed 2018). the projects undertaken in New Zealand are detailed at
Table B.1 and Australian projects are detailed at Table
There is a gap in the literature as to whether value B.3 in Appendix B.
for money (VFM) is conclusively maintained over the
long-term operational phase of PPP projects. Due Such PPP projects gained wide acceptance until
to the long time-horizon of PPP projects, there has the impacts of the Global Financial Crisis of 2007-
been limited opportunity for assessment in the mature 08, which impacted access to finance. In response,
stages of PPP operation. A number of Auditor General governments sought to optimise the value obtained
Reports14 have confirmed value for money is achieved from PPP structures by retaining the positive features
on contract signing but decline from commenting on of PPPs (for example, transferring ownership risks and
whether this value for money has been maintained FM services) and reducing the long-term debt burden
over the life of the contract. It should be noted that on private financing17 by making contributions to the
many of these reports were published during the early capital cost of facilities once commercial acceptance
operational years of the case study projects and they was gained.
mention that retaining value is dependent on sound
ongoing contract administration. Since about 2015, there have again been examples
where the private sector has taken responsibility not
This research aims to contribute to available literature only for DBFM but also operations. Examples include
by testing previously raised limitations, such as the Wiri Prison in New Zealand and Ravenhall Prison
innovation and flexibility, from the viewpoint of service in Victoria.
providers. The research also seeks to investigate value
for money in the mature stages of PPP operation. All projects delivered since 2000 have adopted
sophisticated contracts whereby service outcomes
1.5 An overview of operational social are driven by the contracts using KPIs or ‘Key Result
Areas,’ and the application of abatement regimes if
infrastructure PPP projects in areas of the required service are not met by the PPP
Australia and New Zealand special purpose vehicle (SPV).18
Early PPP projects, in the 1990s, were organised in There is no single contract or service or contract model
a similar way to the Private Finance Initiative (PFI) for PPP projects that has been used consistently
projects implemented by the UK Government. These over time. Variants or styles of the model are used
early social infrastructure projects included hospitals, by different jurisdictions to suit their individual
prisons and public facilities such as sporting stadia.15 requirements and appetite for risk allocation. Four
These projects passed full responsibility for the apparent styles of Australian and New Zealand PPPs
provision of services and financing the capital cost for social infrastructure are depicted in Figure 1 and
associated with infrastructure to the private sector. detailed below.
Initial concerns with the full outsourcing of public 1. The full transfer to private sector model
services were outweighed by the advantages of the represented a complete transfer of all core and
private sector bringing best international practice, non-core services, including all project risk to the
sound management principles, financing and whole- private sector.
of-life thinking to the delivery of quality infrastructure
2. The DBFM with core services by government
and services.
model represents transfer of all non-core services
New PPP policies were released in Australia and along with facility design, build, finance and
New Zealand from 2000, with a different balance of maintain to the private sector. The projects
government and private sector roles. Governments undertaken before 2000s using this model routinely
transferred most of the project risk of the facility
10 UNIVERSITY OF MELBOURNEand non-core services to the private sector. have examples where justice sectors across both
However, from 2000 onwards, PPP policy and Australia and New Zealand have undertaken
practices were revised to ensure the ‘optimum risk prison projects using this procurement model.
allocation’ principle (a risk is assigned to the party The difference between this approach and
best able to manage it) is applied. that adopted in the 1990s is the greater level
3. The model of DBFM with government delivering of management control through the use of KPI
core services and also making a capital regimes and the potential for a capital contribution
contribution represents transfer of all non-core from government.
services along with facility design, build, finance
Given that the focus of this study is to understand
and maintain to the private sector. The objective
whether the current styles of PPPs are meeting the
of upfront capital payments by government was
original service expectations during their operational
to minimise fiscal risk, lower cost of PPP contracts
phase, it is reasonable that the sample of this study is
and improve public-sector flexibility.
drawn from projects since 2000, and where the facility
4. The full service by private-sector model again has been in operation for at least three years.
represents complete transfer of core and non-
core services to the private-sector. Recently there
Figure 1: Timeline of the four styles of social infrastructure PPPs in Australia and New Zealand since the 1990s19
Full transfer to
the private sector
DBFM with core
services by
government
DBFM, government
core services &
capital contribution
Full service by
private sector &
capital contribution
1990 2000 2010 2020
Year
Hospitals Schools Justice Other
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 112. RESEARCH
This section of the report discusses the overall 5. provide any recommendations for future PPP
research methodology utilised to evaluate mature projects.
social PPP projects in Australia and NZ. Additional
details are provided in Appendices C and D. As Figure 2 illustrates, the focus of the report is to
assess whether the service promise, as made by
2.1 The research brief government to service providers and the general
community, has been filled through:
The scope of work undertaken by The University of • the drafting, negotiation and execution of the PPP
Melbourne and Drum Advisory was agreed upon by contract, and
Infrastructure Partnerships Australia and the Treasury
• the performance of the contractual obligations
departments of Queensland, New South Wales,
(particularly the Key Performance Indicators (KPIs)
Victoria and New Zealand. The scope agreed was:
and the Payment Mechanism (PM)) as managed
1. assess whether mature social infrastructure PPPs by the public agency’s contract manager and the
are meeting the service delivery outcomes for PPP Project Co’s FM operator.
service provider and contract manager groups set
The judgements of the contract managers were
out in contractual agreements, media releases and
observed to be heavily based on formal assessments
other community information documents
of whether KPI targets were achieved and other
2. compare, where data is available, service provider contractual obligations. Conversely, those of the
and contract manager satisfaction with PPP assets service providers were based on a largely ex-
and service delivery to that of traditionally procured contractual assessment of how well their service
and delivered assets and services needs, and those of their client community, were being
3. identify what factors contribute to positive service fulfilled. This less formal type of assessment, which
provider satisfaction in PPPs and what factors can can be expected to closely correspond with the views
be attributed to poor service provider experiences of the general public towards PPP projects, is most
4. assess whether value for money (VFM) is useful to understanding how well the PPP procurement
maintained over the long-term operating phase of model is meeting identified service needs.
social infrastructure PPP facilities, and
Figure 2: The service promise in context
CONTRACT MANAGEMENT
Public agency, with technical,
commercial and legal advisers,
translates the service promise into Delivery of services for the 25 to
contractual terms and conditions 30 year contract period DELIVERY OF SERVICES
GOVERNMENT THE PPP CONTRACT
TO END USERS AND
THE SERVICE KPIs COMMUNITY
PROMISE Is the service promise translated PM 100% delivery of stated and intended ACTUAL EXPERIENCE
with 100% accuracy outputs and outcomes?
DELIVERY ON THE SERVICE PROMISE?
2.2 Scope of the research The total number of PPPs meeting these criteria in the
participating jurisdictions is 28 and the nominated case
The case study projects were selected by the study sample was 12 projects, representing 43 per
participating jurisdictions on the basis that the projects cent of the total. This was considered reasonable given
were representative of their social infrastructure PPP that all four categories of projects were represented,
projects. The projects sought were in the domains of and that the study required access to individuals with
health, education, justice or ‘other’.20 For a project to knowledge of a project during its operational life.
included, it was necessary that: The projects nominated and researched are listed in
• the PPP contracted services had been delivered Appendix C, Table C.1.
for about three or more years of operations, and
• it was representative of the current style PPP
contracts.21
12 UNIVERSITY OF MELBOURNE2.3 Research methodology assessment including time and cost performance,
service outcomes and benefits, and user experience
Figure 3 illustrates the four-phase research and satisfaction including identification of factors
methodology used. The methodology actively used contributing to the positive and negative user
mixed method data collection and analysis approach experience. The review of literature contributed to the
to achieve the desired research outcomes (project development process for the workshops (focus group
scope items 1-6). discussions) and surveys. This process included
developing questions for workshop discussions and
The project began with a systematic review of the surveys, recruiting workshop participants and selecting
literature on social PPP projects and comparable venues, and is illustrated in Figure 3.
traditional projects. The literature review was
conducted on three key areas: a VFM outcome
Figure 3: Research Methodology
Select representative Have relevant jurisdictions
Understand current case study projects that provide project Conduct pre and post online
international position on are in operational phase documents detailing survey and workshops with
social infrastructure PPPs and belong to current original promise and project representatives
style of PPP contracts issues experienced
The case study selection process, detailed in Appendix Project scope items one, three and four required primary
C, ensured that this research included key social PPP data to be provided, while scope items two and five
projects that are in mature operational phase. The required documentation from participating jurisdictions
projects investigated covered schools, hospitals, prisons pertaining to media reports, contract documents, annual
and a general category of social infrastructure projects. reports or other secondary data sources.
Phase two of research methodology included drafting 2.4 Governance of the research project
polling questions and selecting the appropriate
polling tool for use during workshop discussions, This research project was sponsored and administered
with separate questions (and workshops) for contract by Infrastructure Partnerships Australia, with funding
managers and service providers. Ethics approval was from industry and governments. Infrastructure
obtained from The University of Melbourne’s Human Partnerships Australia established an advisory
Research Ethics Committee (ethics ID 1954426) committee that included Treasury officials from the four
enabling the researchers to recruit participants and jurisdictions participating.
begin the data collection process. Survey questions
were administered to participants before the The conduct of the research and the drafting and
workshops took place. finalisation of the Report were the responsibility of The
University of Melbourne and Drum Advisory. This was
Phase three of research methodology included done in accordance with The University of Melbourne’s
overall data collection process using workshop Human Research Ethics Committee approval and its
discussions, live polls and follow up survey. Focus protocols and guidelines.
group discussions were recorded and transcribed for
data analysis. Phase four of the research methodology
included a qualitative analysis to identify critical themes
(factors contributing to positive or negative experience)
and a quantitative analysis to provide a spread
(positive or negative) of service provider experiences.
Qualitative and quantitative findings from data analysis
along were used to draft the Report findings and to
meet the final project scope item 6.
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 133. FINDINGS
This section sets out the main findings of the research service change. This was measured by comparison
as related to the scope of work set in Section 2.1, and of governments’ original estimate of the price of the
are derived from the analysis of the data, which is service compared to the price offered by the winning
provided in Appendices D and E, collected from the consortium. Appendix D details the savings claimed
research participants and project documentation. for the various case study projects. The contract
managers and the project documentation also
The findings presented below are illustrated with the confirmed that it was common for abatements to be
authors’ observations in the workshops. applied if services were not received in accordance
with the agreement.
3.1 The PPP projects have delivered
on the service promised In addition to the straightforward assessment of tangible
physical scale or contract value commitments, the study
also appraised the perceptions of service providers
and contract managers. Participants were tested to see
Finding 1
whether their level of satisfaction with the services of
Service providers (95 per cent) stated that the PPP model were as promised. As Figure 4 shows
their PPP project has delivered on the service the overall satisfaction tends toward participants being
promised by the relevant state government and highly satisfied.22 This aligns with the commitment of
delivery agency. ‘enhanced services’ through the PPP model.
Figure 4: Perceived satisfaction level of social PPP projects23
Both service provider representatives and
governmental contract management staff, were
assessed as to whether their PPP project has delivered
Not satisfied
on the project commitments, through a series of
questionnaires and workshops. The specific service
Partially not satisfied
commitments were made by way of business cases,
key media releases, major project documentation
Partially satisfied 9%
and other relevant official communications such as
Auditor General Reports. The service commitments are
Satisfied 33%
paraphrased in Appendix D.
While commitments were specific to each project, Highly satisfied 58%
there was constancy regarding quality of the facilities, 0 10 20 30 40 50 60
high service standards, whole-of-life expertise, ‘value Participants (%)
for money’ and professional facilities management.
While some commitments were contractually
measurable, others were assessed by service The service providers overwhelming reported that their
provider’s perceived satisfaction level. PPP project has achieved the service and consequent
benefit outcomes promised. Contract managers
Many of the commitments made related to physical
also reported that PPP projects had delivered on the
scale of the project, such as the minimum number
service commitments made by government and others
of hospital beds or capacity of a convention centre
at the stage of project announcement and during
and such obligations were reported to have been
establishment. A common theme was that the PPP
always fulfilled by PPP Co. Indeed, the acceptance
projects are working very well, and on balance were
of these deliverables formed the basis of commercial
providing superior service outcomes for the service
acceptance of the project, and many projects received
providers and their client community.
industry recognition through a range of awards.
Contract managers made references to issues (“rough
Contract managers consistently advised that pricing
edges”) that needed on-going management of the
and risk allocations remained as per the original
contract relationship to ensure optimal delivery of
agreements, and that the private sector had not
services by the FM operator. However, these same
sought variations against the original agreement
contract managers reported their PPP projects
unless government requested a modification or
were delivering to government a good deal and in
14 UNIVERSITY OF MELBOURNEsome cases exceeding expectations. They reported the older PPP buildings look better than some of the
instances of innovations that improved the value new facilities procured by traditional models. One
proposition for government. group of school principals stated that they have not
been on a security or FM related call-out for 10 years.
There was near universal agreement among school- A school business manager estimated that her time
based service providers that the PPP model had devoted to FM issues had reduced by 30 per cent.
delivered on the service commitments made by Service providers generally appreciated the time
government and others at the stage of project and effort in shifting to the FM operator the security
announcement. They reported modern leading edge, clearance of all FM personnel coming on-site.
fit for purpose facilities combined with a flexibility of
operation that allowed for changes to be made. For While service providers felt that the service
example, among school-based service providers a key commitment was met, some felt that the service
commitment met was to significantly free up the time of promise should be built from bottom-to-top, with
school principals for educational leadership. Principals the reality being often top-to-bottom. Most service
reported that when the new school opened significant providers reported that they were not consulted on
extra time compared to the non-PPP school setting the contracted KPIs and the payment mechanism.
was released for immediate pedagogy leadership However, this was not identified as a significant issue
rather than on FM issues. in levels of overall satisfaction.
Typical of the general view, service providers generally
observed that in PPP facilities things get fixed quickly.
Some school-based service providers observed that
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 153.2 Service providers prefer to work and more rewarding professional experience. This
experience was found to be largely positive, with the
in PPP projects quality of the experience being influenced by whether
or not their relationship with the FM operator was
a ‘partnership’, characterised as a good working
Finding 2 relationship where positive and negative news could be
Service providers (95 per cent) and all contract equally aired for discussion and, if necessary, resolution.
managers participating in the research prefer
working in a PPP facility and service contract 3.3 The PPP projects are delivering
over a traditional government-owned and on their contracted services
operated facility.
Finding 3
The authors clearly observed in workshops a
commitment among service providers and contract Service providers (82 per cent) expressed a
managers to the PPP model. This preference was strong appreciation of the quality of services
expressed not only in wishing to continue working in provided by the Facility Management (FM)
a PPP facility but also on in their keenness to highlight operator in a PPP facility. Satisfaction level with
both the strengths of the model and in suggesting service quality is strongly influenced by the
improvements for future projects. experience level and relationship between service
providers, contract managers and FM operators.
Service providers reported that they preferred PPP
projects because they afforded the opportunity to
focus on providing services to their client community. The quality level of services being provided in PPP
To paraphrase one school-based service provider, “I facilities was assessed via contract documentation,
like that in a PPP we have a FM expert that does the and survey and workshop participation. From a
FM and that the educators do what they are experts in. contractual perspective, PPPs unequivocally deliver a
And, I like that maintenance and upgrades are funded high standard of service due to contractual obligations.
and carried out. PPP schools work better for students, The perceptions of service providers and contract
and with better facilities the students take pride and managers also confirmed a high level (82 per cent) of
treat the buildings better”. service being provided during the operation stages
of the PPP facility. It was found that perception levels
One principal stated that “another advantage of a PPP were highly influenced by the quality of the relationship
school was a significant decrease in vandalism”. It between FM operators, service providers and contract
was also stated that “higher student attendance levels managers, as well as the relative experience level of
were evident in PPP facilities”. The theme being that the various parties.
maintenance is funded and carried out in a timely
manner, and that students’ response to this service uplift Service providers and contract managers reported,
with an understanding that they are the beneficiaries. and this was confirmed in review of contract
documentation made available by jurisdictions,
The one exception to this majority view was a service that the PPP projects were overwhelming delivering
provider that reported difficult relationship with their satisfactorily on their contracted services.
on-site FM operator. The result being contractual
obstacles being quoted to stop or delay FM services As expressed by one contract manager, in PPP
being provided, and the bounds of good manners projects there is more pressure on getting the service
being occasionally crossed. Interestingly, this on-site right, with greater accountability than in a non-PPP
FM operator was employed by the same FM operator project. This resulting in a heightened focus on the
that employed other on-site FM operators receiving FM operator performing to the requirements of the
high praise. This comment drew attention to the Contract. This contract manager thought that while this
influence relationships have on actual and perceived may also make for a risk-averse approach, the PPP
levels of service. model does allow for innovation and to react quickly
with focus on providing the contracted and even
Contract managers expressed the view that they improved services to the service provider.
preferred working in PPP projects as it gives a richer
16 UNIVERSITY OF MELBOURNEGiven the long-term nature of the multitude of When pressed for details, school-based service
contracts investigated, it was reasonable to expect providers were generally of the view that they were
cases of KPIs not being met or of other contract best serviced when the on-site FM operator:
breaches resulting in abatements to payments. When • was responsive and engaged with their activities
contract managers were asked to comment on such and the educational objectives of the school
cases in their projects, it was found that the application
• approached the role with good-will and a “can
of contractual abatements range from “regularly” to
do” attitude rather than seeking to hide behind
“never,” for fear of relationship damage. However, the
the contract24
fear appeared unfounded as those contract managers
that did abate reported changes leading to better, • acted in a manner responsive to maintaining and
mature and positive relationships. These contract advancing the school environment that mirrored a
managers reported that abatements were managed diligent caretaker in a non-PPP school, and
through contractual processes and the services were • had a natural aptitude for service.
quickly brought back on track.
The service providers were also of the view that such
Most contract managers indicated that constant desirable characteristics of the on-site FM operator
attention is needed by both themselves and the FM need to be enabled by the head FM operator. The
operator to ensure optimum service delivery to the FM operator needing to be engaged in enabling a
service providers. This was a view shared by service strategic, rather than transactional, service. A service
providers. It was also observed that as the contract should be responsive and tailored to their specific
managers and the FM operators gain experience needs and their way of operating. Some service
and expertise, and new generations of PPP contracts providers, for example, stated the process of logging
are entered into, operations and ease of contract jobs can get in the way of them servicing their client
management improve. community in a timely manner.
Some service providers reflected that the performance Another clear theme emerged that superior outcomes
of the PPP project was enhanced by the maturity of with PPP facilities and services are linked to not only a
personnel (both public and private sector) to deal with strong on-site facility management team but also open
the reality of operations, and manage the relationship lines of communication with the PPP Project Co. This is
clauses in the PPP contract. They felt that mature discussed further in Section 4.1.
discussions lead to speedy resolution of issues. One
contract manager observed that 70 to 80 per cent of It is worth noting that, while service providers were
issues and their solution in the PPP facility and service not shy in highlighting areas of improvement for
contract are similar to those faced in a non-PPP facility. FM provision (such as maintenance, upgrades and
new works) and the service provided by their on-
Service providers and contract managers both felt it site FM operator, most also quickly acknowledged
was important not to allow performance and attention when prompted that their on-site FM operator had a
to service quality to drift over time. Some expressed service-friendly orientation (“they are here to help and
the view that the drift can be towards “what the they do help”).
contract says” rather than working the relationships
to optimise mutual benefit. All were of the view that 3.4 The PPP projects deliver service
contract management was not “set and forget”. outcome to expectations
Contract managers also observed that in social
infrastructure PPPs they needed to develop strong
service provider relationships, more so than compared Finding 4
to managing economic infrastructure PPPs. Some All service providers agreed that the PPP
suggested that management of social infrastructure model provides similar flexibility provisions to
PPPs projects required more processes and structure, traditional procurement models.
including the escalation of issues, to ensure that
the multitude of KPIs, stakeholder action items,
outstanding FM matters and other issues were Service providers stated that the PPP model was
addressed in a timely manner. not usually inflexible when seeking change. Service
providers were generally of the view that overall PPPs
MEASURING THE VALUE AND SERVICE OUTCOMES OF SOCIAL INFRASTRUCTURE PPPS IN AUSTRALIA AND NEW ZEALAND 17are “flexible”, being in this regard no better and no 3.5 The research provides lessons for
worse compared to non-PPPs. As a service provider
stated, “we get changes done in PPPs and there
planning new projects
is a process to follow,” noting that in the traditional
setting there were other processes and other types of
obstacles to overcome. Finding 5
Service providers and contract managers are a
However, a few service providers reported that some
valuable and underutilised source of improvement
details in the PPP contract were applied in a way that
initiatives for future PPPs. There is room to
restricted their flexibility unduly. One such example was
incorporate their operational phase experience
the restriction in painting a mural on an external wall
into the planning phase of new projects.
(see Section 4.5). Another example was the ability of
schools to refresh furniture and equipment through the The planning of future PPP projects can be
sale of old stock to partially fund new stock. improved to help achieve better outcomes by:
Flexibility can be improved through “no-fault, • involving contract managers in the early
no-blame” changes to KPIs and minimising the stages of the procurement process to identify
administrative cost or effort of changing KPIs and assess additional benefits associated
(especially where contracts have approximately 100 with the choice of a procurement model
KPIs). The view from service providers and contract • engaging with additional service providers
managers was that the PPP arrangements should be during bidding and design refinement
outcome-focused and less prescriptive, so that the phases of the project
“small stuff” can be resolved on-site and not take a • improving contract management practices
long time. to ensure a consistent level of expertise
among contract managers within and across
A contract manager, whose comments were confirmed
state governments
by others present in the workshop, stated that the
‘service’ was better at the PPP facility compared to an • ensuring that all contracts provide flexibility
non-PPP, and although over time management usually provisions to manage future changes as the
make various changes to the PPP facility (as indeed community’s service needs evolve, and
happens in the non-PPP), this does not change the level • building strategies in the PPP arrangements
of satisfaction experienced by the service providers. that promote open and smooth
communications between service providers
and FM operators during operational phase.
18 UNIVERSITY OF MELBOURNEYou can also read
