A View from SiB – Monitoring

Seeing is Believing and Monitoring – How the SiB team monitor projects

Many thanks to all of you for supporting the Seeing is Believing (SiB) team here in London to ensure smooth project reporting. We have noticed a continued improvement in reporting over the years / tranches.  And I can report that SiB utilise all the data you send us!

SiB realises that in some projects, SiB funds have had to support data collection within the project itself – through support to HMIS, specific staff training on M&E issues etc…. But SiB appreciates that without the foundations being robust, it is very problematic to produce accurate and clear data for monitoring the project outputs and achievements – including quality issues. And without good reliable data it is difficult at the end of the project to see the progress and impact that the project, with SiB funding, has achieved.

Monitoring has to start at the project planning stage – and be integrated into the planned budgets. When all new projects commence we spend time (usually a skype call) to discuss all the reporting formats (narrative, excel and case studies) with the team who is implementing the project, to ensure that from the start you are collecting the appropriate data and confident using the SiB formats to make the reporting back to SiB as smooth as possible for local, country and head office teams.

SiB receives reports from each project at 6 month intervals, but due to 2 reporting cycles to manage our workload here at IAPB, we are engaged with project monitoring across the whole year.

The main ways we use the data is:

·  At the project level.

We collect and collate the 6 month project reports.

As well as reading the narratives very closely to inform ourselves about your specific project, and its progress – both the planned, as well as the unplanned / unexpected – we also compare the narrative with the data in the excel sheets to make sure that we have a clear picture of how the individual project is progressing, and the issues that you are experiencing locally.

We are interested in both the quantitative data, but also issues around quality of the intervention.

We compare your progress with your original plans. We also review the spend (planned and actual) with the outputs reported.

We undertake some additional reviews, primarily utilising the data in the excel sheets.  We produce several key SiB documents / summaries from your data:

·  Project progression sheets

The project progression sheets – outputs and spend – are produced for each individual project. These map progress against your original plans. They are presented both as tables as well as graphs, and are also cumulative over the life of each project. Clearly not every project progresses exactly as planned, but periods with high or low performance should be balanced with periods of “catch-up” or “smoothing” of project activity and spend.

These sheets are sent back to each project, they are also shared with the Bank and sent out to the countries.  And we hope that staff are all now receiving these at the project level.

·  Risk registers

The risk register is an overview of each individual SiB project.

To create this we use 4 parameters: * spend, * outputs (both of these are compared to +/-90% of expected),  * sustainability, and * having a clear plan to return to the agreed targets (should the project be deviating).

The risk register is updated every 6 months to provide a clear overview for each project and its progress against plans for senior SiB managers and Committee members.

We use a “traffic lights system”.  Any projects in higher risk categories (red) will have been contacted directly.

We do not commence the risk register system after the very first 6 month report, as we have found many to deviate widely in this initial report period (plus for some projects this first period is not a full 6 months)

·  Output sheets

The output sheets are a summary of all your output data. Centrally these outputs are summed by Phases, to assist the Bank in measuring the progress of SiB.

From your original proposals we had tabulated your expected outputs, and the actual output sheet are updated at each reporting cycle and sit alongside the expected outputs for each project. These include clinical outputs, as well as capacity building, other infrastructure / capital improvements, etc.  This creates a summary at the programme level (all the projects) and allows SiB to map all outputs and progress across the globe.

·  Collect audit reports

We also collect auditors’ letters from the countries to confirm that SiB projects are meeting standards in their project and financial management at the country level.

·  Visits

The Bank in-country is tasked with visiting the projects at least twice per year to inform themselves  directly about the progress of the SiB project. This is also a great opportunity for project teams to educate the Bank staff about the project and the issues that are faced – both for eye care in general in the country, and also success and challenges for the SiB project team.

Also the SiB team from IAPB will visit every project at least once during its lifetime (generally around its mid-life) to be able to capture key learning. The visits also gain ideas for sharing the successes and learning of the SiB projects.  Visits are also good at reviewing quality issues with the project teams, and sharing how best to capture / highlight this in the reports. It also enables the IAPB team to understand better the local situation, and expected impact of the project in that country and its health system.  Learning is primarily shared among SiB partners at the moment, but SiB hope to encourage more sharing across the eye care sector.

·  Evaluation

An evaluation is more than the sum of monitoring. It is an opportunity to reflect on, and learn from, achievement and also the processes involved in delivering a quality project. Also a good evaluation enables key impact of a project to be captured.

All Phase V projects are now required to undertake a structured evaluation at the end of the project’s life, to review the achievements and cost-effectiveness of the projects , to ensure that best practices are captured, learning is shared across all stakeholders. .

The evaluation reports that SiB are accumulating as projects close-out we hope to utilise both to review the full impact of Seeing is Believing, and also to help review key issues at country and also thematic and global levels.

Sally Crook

Seeing is Believing Programme Manager, IAPB, London