The design and construction monitoring structure often involves the use of key interim milestones as either progress monitoring or payment incentive tools. The interim milestones can assist the Procuring Authority to monitor the progress of the works all the way through construction. The use of interim milestones can be particularly helpful where there are separable parts of a major project that can commence operations early.

The interim milestones may introduce additional payment mechanisms on the completion of an agreed section of work to incentivise performance.

The use of interim milestones may be more important on large and complex projects that consist of a number of discrete packages of work. The application of interim milestones enables the Procuring Authority to gain an early indication of any delays that can be mitigated before they affect the overall completion deadline and compromise the delivery of the public service.

Example – Interim milestones on large and complex projects

The Procuring Authority on a light rail project in a developed market noted that the lack of interim milestones combined with the complexities of the work made it more difficult for the Procuring Authority to monitor the construction schedule. Milestones can also be associated with lump sum payments as an incentive for the Project Company, which was not present on this project.

A common tool used to monitor KPIs during operations is a Quality Management Plan, which is developed by the Project Company and then reviewed by the Procuring Authority. This document sets out who will do the operations and maintenance work, how it will be inspected and how the findings will be reported by the Project Company back to the Procuring Authority. The PPP contract also typically sets out specific reporting requirements.

The Procuring Authority should rely on the Project Company’s reporting to some extent but should also make itself comfortable that the performance data provided is accurate. A variety of methods are available for this, including user satisfaction surveys, spot checks and testing, inspections, and reviews of complaint logs and help desk records.  

The key is to ensure that the level of detail, format and frequency of performance reporting contractually required and operationally requested from the Project Company is adequate for the Procuring Authority’s needs, and can facilitate independent monitoring and verification. Sometimes the information provided (by the operations contractor through the Project Company) is deficient, and parties should be encouraged to meet and review this process.

For complex projects, the parties may also jointly appoint an independent consultant to assist with the performance monitoring during operations.

Compliance with some KPIs can be automatically generated by the software that controls the asset’s operation. For example, some KPIs associated with delays on a rail project are automatically generated by the software that controls the operation of the rolling stock (times of arrivals and departure in all stations, speed, location of the rolling stock, etc.).

Example – Monitoring KPIs

The Procuring Authority on the Zaragoza Tramway project in Spain monitors the KPIs very carefully. The KPI regime is comprehensive, covering quality and availability measures, and four employees work full-time on this task. The Procuring Authority considers that this approach leads to high quality service delivery.

For more information, see the Zaragoza Tramway case study.

The Procuring Authority should not take a ‘soft stance’ on the enforcement of payment deductions as they relate to KPIs. KPIs embody the level of service and the allocation of risk that was agreed to in the PPP contract and so should be appropriately managed to ensure the Project Company is performing in accordance with its contractual obligations. The approach taken by the Procuring Authority should be ‘strict but fair’.

It is not uncommon for there to be some disagreement between the Procuring Authority and Project Company over KPIs at some stage during a project lifecycle, with the study data showing 20% of disputes in PPPs globally involve KPIs. Disputes are detailed in Chapter 5 (Disputes). At a fundamental level, the incentives of each party are divergent with regard to KPIs, as the Procuring Authority wants the highest level of service for the lowest price to deliver the greatest value for money to the public, and may be incentivised to apply payment deduction strictly. On the other hand, the Project Company wants to deliver what is required in the most cost effective way for the highest revenue. These opposing drivers increase the likelihood of a disagreement or dispute. For example, in the research there were a number of instances where the Project Company perceived KPIs as too onerous or unrealistic, while the Procuring Authority saw them as a means of ‘keeping pressure’ on the Project Company to perform.

Both parties should acknowledge the inherent divergence of interests, and approach KPIs with an open mind to work together to resolve any operational difficulties. Application of payment deductions has the strong potential to damage the relationship between the Procuring Authority and Project Company. The main objective is to ensure proper service delivery and not to use payment deductions as punitive measures, as this puts the relationship at risk and will not improve long-term value for money. Stakeholder management with respect to the Project Company is detailed in Section 3.3 (Stakeholder management).

In certain circumstances the Procuring Authority may decide not to enforce its contractual right to impose a payment deduction or a penalty if it considers there is an overriding interest for it not to do so. Similarly, it may decide to apply it at a lower level than contractually entitled. In these circumstances it is important to communicate clearly that the inaction is deliberate, and to clarify the grounds for the decision should similar circumstances arise in the future. Waiving rights under a contract should only be undertaken after receiving legal advice, to ensure an appropriate waiver is effected (i.e. that the Procuring Authority is waiving only what it is intending to waive and not waiving any other rights under the PPP contract). The Procuring Authority must weigh up the risk of damage to its relationship with the Project Company with the financial gain and precedent-setting of strictly applying the deductions.

Example – Pro-active management of KPIs

There was an issue with excessive noise on the Brabo 1 Light Rail project in Belgium. The mitigation, was proactively managed by both parties. Data was collected during noisy periods and appropriate mitigations were developed and implemented.

For more information, see the Brabo 1 Light Rail Case Study.

KPIs are created during the procurement phase and agreed to in the PPP contract. However, they have to be considered from an operational, real life point of view. The Procuring Authority should be in agreement with the Project Company about what is required and work to ensure that KPIs are mutually understood. This needs to be addressed as early as possible once KPI measurement and assessment starts.

The Procuring Authority should regularly assess any KPIs that may be ineffective and, with the long-term success of the project in mind, decide if:

• certain KPIs (e.g. performance standards, rectification periods) or payment deductions should be amended for the benefit of the project
  
  
• a practical solution can be agreed with the Project Company as a way to manage difficult KPIs.

As is detailed in Section 3.1 (Transitions), ‘bedding-in’ periods are also common at the outset of the operations phase to allow the parties to become familiar with their operations phase obligations.

Example – Early collaboration on review of KPIs

The Procuring Authority and Project Company on the Port of Miami Tunnel project in the USA began collaborating with the operations contractor a year before operations were due to begin, to review KPIs and predict any challenges. The Procuring Authority assessed the issues raised by the contractor and concluded that one KPI relating to response times was not workable. All other KPIs were kept as described in the PPP contract.

For more information, see the Port of Miami Tunnel Case Study.

KPIs as drafted in the PPP contract may be unrealistic, ambiguous, difficult to implement in practice or no longer relevant. Procuring Authorities should not take advantage of unclear KPIs to the detriment of the Project Company and the Procuring Authority’s relationship with the Project Company. Instead, the intended application of the KPIs should be clarified between the parties.

KPIs are generally defined in the PPP contract at financial close. For projects that can run for 20 to 30 years, this means that the indicators can become out of date due to external factors. For example, contracts that were drafted in 2005 are unlikely to have greenhouse gas emissions targets, yet a Procuring Authority may be more likely to have emissions targets defined in its current policies. Similarly, an airport may need additional capacity earlier in the contractual period, with the focus shifting to improved customer service later in the period.

Different approaches to dealing with this have included creating KPIs that are flexible to circumstances, such as factoring in specified review points on the original KPIs.

Example – Outdated KPIs

The Central Berkshire Waste project in the UK was signed in 2006, at a time when the primary objective was to direct waste away from landfill.  The Project Company was given the autonomy to achieve this however it saw fit (e.g. through incineration of waste). Current policy is now more focused on recycling and KPIs therefore do not correspond well to the new goals of the Procuring Authority. At the time of writing, the Procuring Authority is considering how to address this challenge.

In addition, the Procuring Authority realised that there were certain aspects of reporting that weren’t covered clearly in the contract, however were still of interest. It therefore developed an informal audit. Conducted with the Project Company, it covers issues that may be more subjective and are not as clearly defined.

For more information, see the Central Berkshire Waste Case Study.

One of the key elements of the PPP model is the transfer of risk to the Project Company; however, it should be emphasised that there is a range of public interest risks that will remain with the Procuring Authority regardless of the risk allocation. Aspects such as environmental issues, health and safety, or community engagement, where the actions (or lack of action) by the Project Company have the potential to affect the public, must be considered by the Procuring Authority. These issues have the potential to affect the reputation of the Procuring Authority, regardless of how the risk in question has been allocated.

The Procuring Authority has an obligation to monitor and support the Project Company, both to protect its own reputation if an incident were to occur, and for the fundamental reason that a government body should be concerned with the welfare of its citizens.

Issues regarding stakeholder management and engagement may present risks for the public and for the Procuring Authority, particularly on large and high-profile projects, which PPPs often are. Construction works have a significant effect on those who interact with the project in any way, whether through noise or vibration associated with the work itself, or the inconvenience of road closures. A lack of engagement will make the experience of those affected more difficult, as well as affecting the reputation of the Procuring Authority. Stakeholder engagement is detailed in Section 3.3 (Stakeholder engagement).

Where the risk and responsibility for stakeholder management and consultation is passed to the Project Company, there is a risk that the work done on that stakeholder engagement by the Procuring Authority pre-bid will be lost. The Procuring Authority should give careful consideration to remaining involved in stakeholder management to assist in ensuring appropriate behaviour of both the Project Company and the stakeholders.

There are several other matters that may threaten the ongoing provision of services that have the potential to become public interest issues. For example, when the Project Company experiences financial difficulties, even where they are entirely the result of the materialisation of risks allocated to the Project Company, if they lead to insolvency then services may be halted, which will affect the public. The issue of insolvency is detailed in Chapter 6 (Insolvency).

Example – Environmental issues

The Project Company on a road project in a developed market agreed to pay compensation towards local community projects after it allowed water that had been polluted by construction works to enter the local waterways. The underlying cause of the incident was related to the unusual soil composition in the area (leading to higher than average settlement time and consequently the overflow of settlement ponds in heavy rain). The construction contractor was not accustomed to these conditions, and the Procuring Authority concluded that it could have emphasised this soil challenge more than it did and mitigated the risk.

Example – public engagement issues

On a light rail project in a developed country, there were issues that required third party stakeholder consent, and were therefore critical. The Procuring Authority took a proactive approach in facilitating these consents and managing the implications for the project. The Procuring Authority established its own stakeholder management team, and is considering retaining this risk for future projects to avoid the costs of duplicating the work with the Project Company.

On the Port of Miami Tunnel project in the USA, the construction contractor faced some challenges in terms of compliance with federal labour laws. It highlighted the importance of the Procuring Authority ensuring that the Project Company and its contractor are fully aware of the relevant laws. On this project, the Procuring Authority was also liable for fines if any of its projects were not compliant with relevant laws and regulation.

For more information, see the Port of Miami Tunnel Case Study.

The Procuring Authority should recognise the broader value in performance data, and should collect performance data for a variety of reasons: as benchmarks for other projects, to inform policy development, and to feed into reporting requirements to the regulators or central government. To the extent performance data is available from other projects, that data can also be used to better assess the performance of the Project Company on a particular project, as it can benchmark against similar projects. Information management is detailed in Section 3.4 (Information management).