BHVI Sri Lanka – Final evaluation

The Standard Chartered Bank Seeing is Believing – Colombo Comprehensive Urban Eye care project was established in 2010 and sought to increase both access to eye health services and to affordable corrective devices in marginalised communities in the capital city of Colombo in Sri Lanka. An estimated 54% of the population of the city lives in slum areas.

The goal was: To contribute to reducing avoidable blindness in Sri Lanka, specifically, targeting the growing problem of uncorrected refractive error in poor urban communities.

Four key areas of intervention were identified, school eye health, primary eye care, refractive services and affordable devices (e.g. spectacles and low vision devices), Sightsavers International (SSI) focused on school eye health and primary eye care and the Brien Holden Vision Institute (the Institute) focused on provision of refractive services and affordable spectacle supply.

The key objectives were:

1.  To strengthen access to primary eye care services among poor urban communities. (Sightsavers)

2.  To strengthen school eye health services. (Sightsavers)

3.  To establish within the communities the sustainable delivery of refractive services through vision centers, optical labs and workshops. (Brien Holden Vision Institute)

This report looks only at objective three and the activities of the Brien Holden Vision Institute.

The project design was to develop two community based vision centers and an optical workshop and to support the provision of spectacles at the national eye hospital via establishing an optical workshop, training of staff and the provision of frames and lenses.

Typically refraction in Sri Lanka is conducted by Ophthalmic Technologists who have two years training at a government training school and are deployed to the government hospitals, or private sector optometrists and technicians who have been trained via apprenticeships and/or the Sri Lankan Optometric Association’s training course. Therefore, the project also supported the up-skilling of Ophthalmic Technologists to provide quality comprehensive eye-exams.

The project was implemented as planned although time delays meant that some of the activities started considerably later than expected and some aspects of the project were then extended beyond the planned finishing date in order to ensure targets were achieved. There was also considerable staff changes across the life of the project at all levels, the national eye hospital, the Institute’s Sri Lanka and Australian offices, at the vision centers and within corporate services at the bank. These also contributed towards delays in project implementation and also disruption of services to patients, which in turn affected output numbers. However despite this, the project has essentially succeeded in delivering against the project objective and the key outputs.

Another key achievement is the strong relationships that have been developed with various stakeholders in Colombo, within the public, NGO and private sector which have had several positive flow-on effects, including increasing awareness of the need for marginalised communities to have access to eye health.

This has translated into positive initiatives by the Sri Lankan Optometric Association, Rotary, the Lions Club, religious leaders and several large-scale commercial businesses, resulting in regular outreach services being provided by the vision center staff, which has significantly increased the number of people receiving care beyond just those attending the vision centers.

These activities have been regarded positively and have helped in strengthening and extending the relationship between Brien Holden Vision Institute and the national Vision 2020 committee and the National Ministry of Health, which in turn has facilitated their support for further eye health initiatives in the country. Seeing is Believing banners are displayed at outreach and promotional events to promote the support of Standard Chartered Bank to partners and to the community at large.

Key Learning and Recommendations

Program Learning:

1.  Strong relationships are critical for providing a foundation for the program.

2.  Committed leaders championing and driving action, this has been particularly important in achieving the project outcomes in the NEH optical workshop and the allocation of a MOH budget for the continued provision of services.

3.  Government awareness of program benefits is central to garnering future commitment and support.

4.  Quality of service is critical – it can differentiate the service from others, engender confidence and overcome barriers to patients accessing services. Perception of quality of services and communication skills of practitioners can influence patient uptake and garner support from key decision makers in government, professional bodies and communities.

5.  Recognise that quality and sustainability are dependent on adequate training of staff and that this requires resources and time. This may result in a slower build of services but theoretically a solid foundation will be built resulting in better outcomes overall.

6.  The importance of a complementary approach to program design to ensure projects build upon and also support existing services and infrastructure. The initial ‘stand-alone’ nature of the proposed services and training was confronting to existing service providers, the final design worked within accepted structures, systems and norms and was therefore more acceptable to stakeholders, garnered the required support and was successful.

7.  Increased stakeholder engagement and maintaining dialogue led to increased project outputs.

8.  Additional strategic inputs and resources may be required to reach vulnerable and marginalised communities. Outreach is not always factored in as a key strategy for achieving awareness, education and reach.

Recommendations:

Although recommendations have been made in the project specific areas these can be summarised under the broad categories below. These are applicable to this particular project and to most eye care projects that may be developed.

Situational analysis: Future situational analyses should include engagement with a broad range of potential stakeholders, including existing professional bodies, a range of eye care cadres, other NGOs and the private sector where relevant, as each will bring different perspectives and pertinent issues to be considered in the program design.

Stakeholder feedback: The project has been very good at holding stakeholder consultation meetings, but I recommend that, as part of the stakeholder engagement process, future projects get formal, documented feedback via surveys or structured interviews. In the early stages this can be used to understand stakeholder needs, desires and challenges and to ensure there is an alignment between the expectations of stakeholders and what the project can deliver, and then repeated periodically during implementation to measure ongoing satisfaction and alignment. During the project, it is also important to get beneficiary feedback via patient satisfaction questionnaires for example. In addition, it can be very valuable to do simple surveys, or interviews, after some outreach services, this should be done with both the organisers of the outreach and with the beneficiaries to determine satisfaction with the way the service is operated, what were the expectations, needs and were they met, etc.

Develop a more detailed sustainability plan: as sustainability is key, the mechanisms by which this will be achieved and the indicators and milestones for achieving it should be identified, in detail, at the beginning of the project, monitored and adapted as need be throughout the project. The sustainability plan should be both for the project as a whole but then also broken down by each vision centre. Each center will have different dynamics and influencing factors that need to be taken into account. The project situational analysis at macro and micro level should inform this process. Ensure that the data you are collecting monthly and the way in which you are analysing it allows you to determine if you will meet your sustainability milestones.

Formally monitor the impact of sustainability driven interventions: this is particularly important when testing out new models and also determining what works in what setting and informs replicability and potential for scale up. If a significant change is made, e.g. opening hours, staffing levels, pricing, frame ranges etc, then the impact of this on patient flows, sales and customer satisfaction should be monitored, and the outcome of this should determine if other adjustments need to be made. Typically project staff are well aware and able to articulate what impact or otherwise interventions have had, but formal monitoring allows information to be shared with stakeholders in a meaningful way and provides tangible evidence to inform future project design or scale up.

Learning framework: From the Project Design Documents draw out a project specific learning framework to guide data collection and analysis and reporting at different levels, both strategic and operational. This would define the parameters, criteria and processes.

Identify operational research opportunities: often there is a wealth of information within ‘everyday’ project activities or data collection that could be extracted or analysed for either publication or shared learning within and without the organisation. Within this project an example is the analysis that was done surrounding the impact of a gender focused approach, two other simple analysis that could be undertaken just with desk based work, and based on data already collected, would be to track return patients – patients are told to come back each year for a repeat visit, the centres have been established long enough now to be able to monitor to what extent this is happening (and if intervention is required, i.e. a reminder phone call), and if so, does the intervention increase repeat visits. Tracking of referral uptake, what percentage of those referred keep their appointments, reasons why and why not, are interventions required, etc.

Embed Quality Assurance in the project structure: this project demonstrated very good training programmes, both clinical, management and customer service, it would be useful to work with the Institute’s education team and the Sri Lanka project team to develop this into a framework that could be replicated for other projects.

Draw from in-country partner and stakeholder expertise: This project utilised existing expertise from both the SLOA trainers and the personnel at the bank to capacitate project staff, this both reduced potential training costs and strengthened relationships. Allied to this is the importance of creating relevant in-country training capacity to minimize the need to bring trainers from overseas.

Source equipment and other resources locally: where possible as this usually minimizes delays due to shipping and clearing of goods and maintenance contracts are easier to secure. If equipment is donated or significantly cheaper if purchased elsewhere, then a) factor in realistic delays for shipping and clearing (explore benefits and costs of using experienced commercial agents rather than relying on under-resourced MOH staff to do this), also include a budget for maintenance contracts for providing in-country training to MOH equipment engineers and technicians (a train the trainer approach).

Evaluator:  Daveena Brain

Sumrana Yasmin

Regional Director, South East Asia & Eastern Mediterranean

Brien Holden Vision Institute

s.yasmin@brienholdenvision.org