ITT was commissioned by the Kenya National Highways Authority (KeNHA) to provide consultancy services for an impact assessment of investment in the roads sub-sector. The project aimed to identify and demonstrate the additional impact that road investments have on Kenya’s development. At the policy level, it was perceived to support to the three pillars of Vision 2030: Economic, Social and Political Governance, by enabling the development of world-class infrastructure and services.

In undertaking the assignment, key activities included the selection of Key Performance Indicators (KPIs) to identify what impacts should be monitored, selection of representative case studies, extensive stakeholder engagement and development of a spreadsheet framework (KENREF – the Kenya Roads Evaluation Framework) that would enable KeNHA, the AfDB, other interested parties, as well as local independent consultants, to monitor the effect of road investments across a range of impacts. KENREF is the first of its kind to become embedded in KeNHA’s service delivery.

A three-day virtual KENREF training workshop was conducted by ITT from 6th to 8th April, 2021, via Microsoft Teams. The  workshop was attended by a range of stakeholders from different road sector agencies in Kenya, development partners and countries within the East Africa region.

Day 1

The focus for Day 1 was on a detailed description of how the project was undertaken and the outputs that resulted. The project review comprised six elements:

1. The impact of road investment on local economies;

2. The Impact of road investment on social factors;

3. The impact of road investment on the environment;

4. The impact of road investment on road safety;

5. Institutional and Administration Reforms; and

6. Review of assessment and evaluation methods.

The review set out to identify the key issues and impacts that the construction of new and improved roads can generate across a range of socio-economic and environmental factors. These then lead to the development of the KPIs to be monitored as part of the evaluation process. Fieldwork studies revealed that the improvement of roads had led to vehicle operating costs (VOC) savings and contributed significantly to reducing travel time between commercial centres in the case study locations, especially for newly constructed roads. Along rural roads, the average traffic level had increased by 1-5% and in urban and national trunk roads the levels increased by 5-10%. In the agricultural sector, the construction/improvement of the roads had lowered the costs of farms’ inputs: 56% of the households reported that the roads had played a critical role in improving agricultural productivity by allowing locals to access markets. However, due to the increased volume of traffic and modal conflict, most of the sampled roads were found to also see higher numbers of crashes, accounting for an increase in injuries and fatalities as a result.

Day 2

Day 2 covered the KPIs and details about KENREF. KENREF sets out guiding principles for designing and implementing impact evaluation in the road sub-sector. This rollout phase comprised four elements:

1. The KENREF Spreadsheet;

2. The associated Guidance (User manual);

3. The Data Strategy (contained within the guidance); and

4. The Capacity Building Programme (i.e., the training of KeNHA staff and others to operate the spreadsheet).

It was demonstrated that KENREF is applicable for all road investments by KeNHA, the Kenya Rural Roads Authority (KeRRA), the Kenya Urban Roads Authority (KURA) and the Kenya Roads Board (KRB). It was suggested that KENREF is probably most applicable to larger road investments, such as new road construction and major highway improvements. There was acceptance that KENREF can help inform decisions regarding future road investments since it can demonstrate the impact of road investments across a range of economic, social and environmental effects, enable trends in the impacts to be monitored over time and measure progress towards the achievement of targets or standards.  It was explained that KENREF KPIs are grouped five strategic KPIs:

1. The impact on the transport economy;

2. The impact on the local economy;

3. The impact on the social economy;

4. The impact on the institutional management of the road system; and

5. The impact on the environment.

The strategic KPIs are further broken down to sub-indicators. Overall, approximately 50 KPIs covering the wide range of potential impacts can be measured and are included in the KENREF spreadsheet.

Day 3

Day 3 of the training was short and mainly covered the data management strategy for KENREF. It was emphasised that baseline evidence/data was an important resource to monitor and evaluate, in order to be able to compare the current or future impacts of a road investment done in the past against the pre-intervention status quo, and counter-factual, i.e., what difference has the investment intervention made since or make projections of what the trend might have been without the intervention. Following the workshop proceedings, there was consensus that anticipating future evidence needs at the early road development/improvement stage enable lifecycle impact evaluations to be done faster and easier, and of much higher quality. This is because the evidence base is there to build upon as opposed to road officials or consulting firms attempting to back-analyse the impacts years after investments have been made. It was agreed that a major constraint to back-analysing impacts is the potential loss of institutional/sectoral knowledge needed to guide the process. As an example, it would be extremely difficult to retrospectively capture baseline or pre-intervention evidence for what the public attitudes were several years ago.

Workshop Outcomes

Overall, the workshop successfully demonstrated the need for KeNHA and other road agencies to move from a reactive process of identifying impacts from the numerous road investments to a proactive model supported by a well-planned transport data management system. Key takeaways included:

– Plans should be made for how and when a road project will be monitored, evaluated and reviewed.

– Road agencies should consider committing to a periodic or yearly review by committing resources (human resources and money) to the process.

– Using the KENREF and existing baseline evidence would help to provide the basis for planning what is important to measure and report on as KPIs through the implementation of road investments.

– KPIs should be evaluated or reviewed periodically.

As a final remark, Impact Evaluation does not happen by itself; it needs planning, resources, and skills.