Reimagining Transport Project Peer Reviews

Key takeaways:

• Independent peer reviews of transport projects are designed to improve project outcomes, including the accurate estimation of costs, timescales, and benefits.
• However, a significant proportion of transport projects still come in significantly over budget, behind schedule and with fewer benefits than originally envisioned.
• Although the reasons for this go far beyond the peer review process, there are significant issues with peer review that there are now opportunities to address. These issues include:
  1. Peer reviewers are limited in their knowledge and experience.
  2. Peer reviewers' recommendations are often ignored.
  3. Peer reviews often occur after significant project decisions have already been made.
  4. Peer reviews can suffer from the principal-agent problem, where project proponents constrain reviewers.
  5. Peer reviewers can be susceptible to optimism bias.
• Several changes to peer review could significantly improve the process:
  1. Create a global transport project dataset and use it to train an AI support system that peer reviewers use.
  2. Utilise the AI support system throughout a project, not just at periodic milestones, such as gateway reviews.
  3. Independent agencies, not the proponent, should employ peer reviewers.

What next?

What steps can you take to strengthen the peer review process?

Introduction

Independent peer reviews are designed to improve the outcomes of transport projects by rigorously assessing estimates of costs, schedules, and benefits before investment decisions are made. Yet, the persistent reality is that many transport projects still come in substantially over budget, are delivered late, and fail to achieve their promised benefits, a clear indication that the peer review process is only having, at best, a limited impact.

The problem isn't that peer review is fundamentally flawed; it's that the current system is operating with outdated methods, structural conflicts of interest, and limited data. While peer reviewers bring valuable expertise to the table, they're often constrained by biases, information gaps, and institutional pressures.

However, by combining artificial intelligence with truly independent oversight, embracing transparency, and learning from a wealth of global transport project data, we can transform peer review to contribute more to project success.

Why Peer Review Often Falls Short

Despite their intent, peer reviews face a range of challenges that undermine their effectiveness:

Limited Reviewer Knowledge
​Peer reviewers are highly experienced professionals. However, the best peer reviewer in the world will still only have experience in a very limited number of projects and circumstances that they have learnt from. This means they inevitably have blind spots in situations where they lack experience.

Recommendations Ignored
​Even when peer reviewers make strong recommendations, these are too often overlooked by decision-makers who are committed to predetermined outcomes or constrained by political and financial pressures.

Timing Issues
​Many peer reviews are conducted after crucial project decisions have already been made, making it difficult for their recommendations to be adopted without significant disruption to the project.

Principal-Agent Problem
​Reviewers are often selected or paid by project proponents, making it difficult for them to challenge powerful interests or raise uncomfortable truths. This structural flaw leads to constrained critique and limited independence.

Optimism Bias
​Just like those managing projects, peer reviewers can succumb to optimism bias, overestimating benefits and underestimating costs or risks, especially when reviews rely chiefly on subjective judgment rather than factual benchmarks.

Lack of Transparency
​Peer reviews are not made available for research or for others outside of the associated governments to learn from.

The Path Forward: Building a Better Peer Review System

A few practical changes can improve the integrity and impact of peer review in transport projects:

Create an AI Support System Based on Global Data
​To provide peer reviewers with insights based on a much broader knowledge base, a globally accessible transport project dataset should be developed and used to train an AI support system.

Creating this knowledge base requires governments to be willing to share their project data, including peer reviews and reports. Gaining access to the AI support system and improving projects provides a significant incentive to achieve this.

Government should also implement robust post-completion project reviews, feeding actual project performance back into the global knowledge base and AI engine, continuously improving future project evaluation frameworks.

This AI support system would be best built and maintained by an organisation with a global footprint and an interest in supporting transport projects, such as the World Bank.

Of course, this means that peer reviewers must learn to use the AI support system to supplement their expertise, enhancing the quality and depth of their reviews.

As is well known, AI has its limitations. Therefore, it needs to be acceptable to disagree with the AI support system. Guidance is needed on when this is appropriate and how these decisions are documented. Decision makers need to be informed when there are differences and their rationale.

Implement Continuous AI-Driven Review Of Projects
​A significant benefit of the AI support system is that it can be utilised effectively beyond peer reviews. It can provide an invaluable resource throughout the project's life cycle. This allows for dynamic review and feedback that can prompt early interventions and course corrections before mistakes get embedded into projects and become costly to reverse. When done well, peer reviews should identify far fewer items of concern.

Create Truly Independent Reviews
​Moving forward, peer reviewers should be employed by agencies that are independent of the project proponent or deliverer, thereby reducing the risk of compromised critique and enhancing accountability.

Conclusion

Building an AI support system for transport projects that can be integrated into the peer review process and used as an ongoing support tool for project teams, combined with creating a genuinely independent peer review processes, should contribute, hopefully significantly, to improving the success of transport projects, bringing a much higher percentage of projects in on time and on budget and delivering the benefits desired.