All posts by ashleygood

Transcript

Good evening. My name is Charity Ngoma. I’m from Lusaka, Zambia.

I work for a project called PROFIT. PROFIT is a USAID-funded program started in 2004 but we only started to go out and do interventions in 2005.

For those that are from Zambia, I think we have heard a huge publicity about PROFIT and what it has done to improve the private sector. It is perceived as a successful project.

But I want to draw the crowd this evening to a different thing: How did we get to where we are? It hasn’t been very rosy. I think people read our reports and they come to PROFIT and say “Oh, you guys are doing a great job.” But it has not been that great.

In 2005, when we started – I mean after being interviewed, given the job, the package was good, they sent in for a grant, we were young, from university, just thinking we were going to change the whole industry – we wanted to change Zambia.

We started and went out with our big books and talked to farmers about what we thought we were going to do. It was good because we had all the energy. We had all the incentives waiting for us and with good pay and vehicles and everything. So we were all out to do a good job. But it didn’t work like that.

In 2005, as sector coordinator, I was supposed to help vets and farmers come up with an intervention that was going to help promote vet services. It worked well. We talked to the vets. The vets bought our idea. We talked to the farmers. The farmers were saying “Oh great, finally we are going to save our animals.” And we said to the vets, “You can sell this package, and it will give you lots of money.”

But that package had one thing: farmers needed to pay for the vet service well in advance, a year in advance. Twelve months in advance they pay for the services and all the vet does is go in and do the services.

The first year, the first month, farmers signed up. Farmers paid, some for six months; they didn’t trust us enough to pay for a year, but they paid for six months. Some paid for a year. We had a good number join. But the following year, 2006, all those that had paid didn’t want to pay again. So we only had a few people want to buy that service.

But did we stop? No.

We kept on saying “This is the best solution for you. You need to buy this service. You need to keep your animals alive. You’re doing it all wrong. This is how you should do it.” Six months later we only had a few farmers again, most of the farmers had dropped out of the package.

And here we are: How do I go back and say “It’s not working.”?

How do I go and tell my supervisor, or go back to my quarterly report when I am writing to USAID saying “Oh, we are not getting the numbers, farmers are not buying our interventions.”?

It was hard. We didn’t want to do it. So we kept on pushing it. It meant – personally, on me – it meant I was not doing enough. You’re not going in the field enough. You’re not talking to farmers enough. You’re not having enough meetings.

I was also putting the pressure on the field staff because I was supervising a good number of field staff. I was putting the pressure on them: “You’re not going out enough. You’re not selling this thing. You’re not talking to service providers.” So another two years of pushing a service that farmers didn’t want to buy.

But we had a very good organization. They still kept telling us: “When you go out in the field, please come back to us and let us know what is not working so that we can work on that.”

But because admitting that it is not working means you are a failure, that you haven’t done a good job, no one wanted to do it. So we go out and still market it, come back, still market it, and come back. Until such a time that we really got problems from the supervisors: “Why are we not getting the numbers? You guys have been busy at this for two years and still only five vets are paying for services. The vets that paid last year do not want to pay for it. Why is this happening?”

But then we came to the problem, and this has to be done very well: You need to have very good people because it’s hard to tell them it’s not working, because they sign your paycheck. You’re thinking “If I say it’s not working and my contract is up for renewal, am I going to get the job?” So you’re in the middle of “Should I tell them ‘It’s not working’?” or “Should I pretend it’s working and it’s going to work if we push it more?”.

But with that lesson, with that problem from the supervisor, we reached a stage where we said “It’s not working. Guys, we have failed. The farmers are just not buying this. It’s not working for the service providers. It’s not working for the vets.”

And it’s only when we reached the stage when we were admitting that it was not working that we started to learn. Because all this time we were closed to learning about what wasn’t working, because we just thought “This is the best thing. This is it. No one has any other solution than this.”

Only when we sat down and we said “Guys, it’s not working. Let us re-look at this. Why are farmers not paying for this service? Why does a vet still not see this as a business? Why is it so hard for the farmer to come and pay for the service without us forcing him to do it?” That’s when we started to learn.

I want to say tonight: failure is painful to admit. And a lot of people that are in development work push an intervention just because they need to keep the numbers on and they don’t admit it in a report on the table. But sometimes, even though they know it’s not working, it’s a very difficult thing to admit.

When we sat down and re-looked at it, and we asked the question “Why is it not working?”, then we started to look at other ways in which this intervention could still be brought out to the farmers without it being a very difficult thing to buy.

The first thing we had to look at is: Why are they not paying for it? We forgot that these communities were not conventionally driven; they were trust driven. So their thought was: “If I pay a vet for a year, is he going to come out and vaccinate my cattle? My money is going to go, so I’ll lose out.”

They were not paying because they just didn’t trust to pay up front. And I would also not do it. I think we were asking too much from the farmers who have never met the vet, to pay him for a year – the vet who comes from 400 kilometers away – and pay him for a year so he’d come in and do the services.

When we admitted that this is not very good, we looked at it, and then we sat down to redesign it. When we looked at the why it was not working, we broke down that package into individual services and farmers buy as they want the service. So the farmer will buy the service that he needs at that particular time.

We just saw the way that it shifted: farmers started to buy it, and now we’re talking about thousands of dollars in sales to vet firms or vet service providers. That wouldn’t have happened three years ago.

So what I would like to say tonight is: A lot of donor-driven organizations, we have to answer to our donors, of course. Money is coming from somewhere. We’re evaluated based on the numbers: “What have you done with the 17 million dollars that have been given to us?”

But also, this is a lesson to people that are implementing projects. They need to build in their organization a culture of learning. A culture to allow their staff to admit that “No, this is not working.”

We need to get a conducive environment. Our supervisor – he was from somewhere else – but he made that environment a viable one for us. He said: “You guys, if it’s not working, come back and tell us it’s not working.” We just didn’t want to do it because we thought “Oh, this is going to take my job away.” But there was an incentive for us to come back and say: “It’s not working.”

And this is the lesson that a developmental organization needs to put in. They need to create their environment. They need to let their staff know that it’s OK to fail. It’s OK to say “This thing is not something that will work for this community” so that those staff come back and they tell them the truth and they’re able to change that in the intervention.

And now – between that same period, 2005 to 2010, we can safely stand up and say “the farmers were able to buy the vet services from the vets without any of us going out and pushing them because now it’s working for them and vets are also able to do business with the farmers without feeling “Oh we are being pushed to push the service to the farmer”.

At the end of this I want to say failure is not the end. The worst thing is – you as an implementing organization – you fail and you do nothing about it.

[Instead] You fail and ask why.  When you answer the why question – why is it not working? –  and look at other reasons and ways in which it would work, get up and do it because if you fail and say “it didn’t work so I’m just not going to try anything else”  it won’t help anyone. That is the lesson I would like to let people know.

Maybe some people have not been to Zambia or Africa. It looks so easy but if you are working there you know that I need to keep this job. It’s an environment that is very difficult to work in and say “I am not doing a good job.”

Thank you.

Ka-Hay Law (Engineers Without Borders Canada):

Sorry, I’ve had the opportunity to work with Charity for three and a half years in Zambia and her voice needs to be heard more. Her voice needs to be heard more, and more of her voices and her peer’s voices need to be heard more. But something is stopping it and we need to do something about it.

Failure

When I was still new in Kenya, I was at a meeting about energy efficient stoves, reducing firewood consumption by around 50%. I had read about solar cookers that totally eliminate the need for firewood, so after some time I could not contain myself anymore, but remarked that it seemed unnecessary to still be using 50% firewood, when that could be reduced to zero.

“You must be very new to Kenya”, I was told, and indeed over the following months I realized that very rarely will the societies of Kenya or Eastern Africa in general accept to cook food in the open, for everyone to see, and with no fire. Indeed, one time while camping, we were physically threatened for cooking our food in the open – it is a strong taboo for several tribes, and with good reason – in areas and times of near starvation, showing others that you have food is not good manners.

Failure

Around 2002, I donated a $700 Katadyn water filter to an orphanage in Haiti. I instructed them on the use of the filter, and asked them to be sure and filter water for the neighbours of the orphanage too… to build up good relationships with their community. They seemed amazed that such a small piece of equipment (around 10 pounds) would actually work to create clean water, but after I drank water purified by the unit, they were convinced.

The filter was rated for 20,000 gallons, so I told them whenever they used it up, to let me know and I would send them another cartridge. I thought we were golden when I left.

Because the filter was so expensive, after I left, it got put into a locked closet. It was rarely taken out and used. I don’t believe it was ever used for the neighbors and rarely for the orphans.

Eventually, someone broke into the closet, or an employee with a key, not sure which, stole the water filter and sold it. The money did not, to my understanding, go to serve the orphans in any way. I have always hoped that someone was getting clean water from it, but who knows?

Failure

If you’ve followed charity: water over the last five years, you have probably heard us say, Tweet or write: $20 can provide clean and safe drinking water to one person for 20 years. But earlier this year, we removed the “20 years” part from that messaging.

As with any retraction, this sparked a discussion with our staff about how we deal with failure. We didn’t necessarily “fail” in providing water for 20 years to the people we serve—not only are we not there yet, but we’re also adamant that we do what it takes to make sure each of our water projects last at least that long.

However, we knew that if we continued to promise that each $20 donation would provide one person access to water for two decades, we’d be using a number we’re not certain about. In effect, we’d be failing the faith of the public and our mission to “reinvent charity”—to restore peoples’ trust in charitable work. And that’s been an ambition of ours from the start.

So as we changed the messaging about 20 years across our site, we immediately took to our blog with an explanation, hoping to spark a dialogue around the word “sustainability” and divulge what goes into the numbers we use to explain our work.

Here’s the breakdown:

where’d charity: water get $20?

It comes down to simple math: $20 is the average cost per person to implement a charity: water project. That includes funds for sanitation, hygiene training and our partners’ existing maintenance models.

The technologies we fund depend on the region, the local culture and the program of our local implementing partner. Construction in some places can be relatively cheap; in others, even getting out to the project site in the first place costs a fortune. Here’s the breakdown of the average costs per country we work in, to give you an idea of just how much the cost of building a project can vary from program to program:

where’d you get 20 years?

Four years ago, the accepted average lifespan of many of our water technologies was 20 years. Since then, charity: water—and the water sector as a whole—has been reevaluating what “sustainability” really means. We’ve always known that $20 per person covers the implementation of the water project on the ground. But we became unclear about the cost to maintain our water projects over time; so we didn’t want to continue to tell our supporters or the general public that $20 can cover the cost of water for one person for 20 years.

A $20 donation to charity: water can still provide one person with the initial access to safe water, since it pays for construction of the project and early engagement with the community it serves. But keeping the project running over the next 20 years could cost more—we aren’t sure yet. This will depend on the technology, which maintenance model works best and how (and when) the community fully takes ownership of their water project.

how will charity: water projects last?

For each charity: water project we fund, from drilled wells to household BioSand filters, we work with our local partner to include some type of maintenance component. Just like the cost of building projects, this also varies; in some countries, we form and support local Water Committees to look after the projects. In others, we fund training for individual families to learn how to maintain their projects or set up a scheme where the village pays an available repair team to help.

We’re also dedicated to innovation in water project sustainability. The water sector as a whole is shifting its focus from the number of projects built to the longevity of these water sources. It’s an exciting time; new opportunities that have come up in just the last few years have potential to drastically increase accountability for water projects and monitor their sustainability.

We’re already piloting or supporting new systems to oversee our projects in the field. Here are a few examples:

Public-Private Partnerships in India

We’ve supported the establishment of Public-Private Partnership (PPP) Centers in two urban districts of India, Uttar Pradesh and Bihar. The program trains local youth and women to repair and maintain hand pumps. This provides jobs, ensures a repair option for locals and best of all—the center is a business, so it sustains itself. The PPP centers serve as demonstration sites, whose best practices can then be replicated by local government, with our partners indirect involvement and support.

Field Level Operations Watch (FLOW) with Water For People

One of our implementing partners, Water For People, has created an innovative visual data system to make monitoring projects more transparent and reliable. They upload data—GPS coordinates, populations served, state of the water project—from the field on mobile devices (usually smart phones). This data is then available online for anyone to assess the status of projects. Since we already prove every charity: water project using GPS and photos, we’re hoping FLOW helps us get more information on our projects and get it faster, too.

Clustering in rural Ethiopia

Monitoring projects in remote areas is very challenging. Our local partners in Ethiopia have adapted by “clustering” many of their charity: water projects to concentrated areas. This makes gathering data easier, as it’s all in one place. It also fosters region-wide accountability; communities learn best practices from others who are taking care of their projects.

Failure

Sometimes, even when we know the right thing to do, we fail to do it. We do this with seatbelts, diets, speeding, and love, and as it turns out, we sometimes do this with PEPY programs too.

Recently one of our programs faced a failure which should have been avoidable but which will hopefully help us set better systems in place to avoid similar problems in the future.

You might have read about our “Saw Aw Saw” program, the arm of PEPY which partners with communities to help them create and implement plans to improve their government primary schools.

To build more long-term sustainability into the program (learn how we define “sustainability” at PEPY), SAS includes a small business development component. The idea is that if schools are able to generate additional income on their own, they can use this income to further develop their school beyond what the government or other fundraising efforts provide.

Last year one of the SAS partner schools decided to start a small mushroom growing business. It did quite well, as there was no other local supplier of these nutritious mushrooms, and their first rounds of sales went very well. Eventually, it became too difficult to source mushroom spores and the program stopped.

This year, two schools decided to start a spore-growing program, as spores typically generate a high net profit and in this way they could support local families in improving their nutrient intake by affordably growing their own mushrooms at home. This sounded like a great plan!

BUT we rushed into this program to try to get it started before the end of the school year. We didn’t do enough research, or support the communities with the tools and networks to do this themselves and we also didn’t have the in-house technical expertise to understand the threats to this agriculture program.

Part of the SAS model provides support for the one-off training costs which go into business development. We sent representatives from both schools to a course on mushroom growing. In addition to poor research, we made another big mistake, which goes against the lessons we have learned:

We paid for this in full. The school support committees did not have to invest funding into this project, only their time. As such, if there was a financial waste, they had very little incentive to point it out or prevent it.

We didn’t send any PEPY staff to the training, which would have helped us to understand the program into the future and might have also prevented us from wasting funds on unnecessary equipment. You see, the key to growing spores, it turns out, is a sterile working environment. We had researched this enough to know the very basics, but when signing community members up for the course, we failed to research what technical tools, apart from the training component, would be required for the success of the program. When the community came to us with a proposal to go to a nearby training on spore growing, we accepted the proposal without doing enough research on how the training would work.

It turns out that part of the training included how to use one of the key tools in spore growing. This sterilization device is, you guessed it, electricity-powered. We had sent two people who live in remote communities with no electricity to a training about how to use an electronic instrument, just because they had asked.

Big oversight.

Failure

Médecins Sans Frontières (MSF) is an organization that thrives on critical debate. Across the five MSF operational centres, we readily point out each others’ errors and shortcomings. When this dynamic is kept within the range of “healthy tension,” it serves the operational teams and the patients we treat. Everyone has to defend or adjust their actions in response to the criticism of colleagues. This pushes program quality to the forefront and reinforces accountability.

Added to this, MSF headquarters staff regularly visit field teams to ensure that our medical action adheres to the standards and goals of the organization. We also do formal evaluations, particularly after major emergencies. Some within MSF will say we don’t do enough evaluations. Having been with the organization for almost 20 years, I believe that we tend toward the opposite.

We do so many evaluations, alongside the regular supervisory visits, that our teams tend to be swamped with recommendations that risk being lost over time. Trying to prioritize and follow through on the multitude of well-intentioned insights can be overwhelming.

Learning lessons in MSF is not a problem. We learn lessons easily. Unfortunately, the same lessons are sometimes learned by different teams at different times. Our challenge is to swiftly integrate what we have learned across the vast MSF movement – more than 27,000 MSF aid workers served patients in over 60 countries in 2010.

As an organization that works largely in sub-Saharan Africa, historically we have focused on treating people with infectious diseases found in low-resource settings. This was reflected both in our clinical guidelines, and in our emergency medical supply kits containing standard drugs and medical supplies for the most common diseases.

In 2003, I was supervising MSF operations in Iraq. There, many of the illnesses people suffered were non-communicable, like heart disease and diabetes. In the aftermath of the U.S. invasion, our teams quickly positioned themselves with the usual emergency medical supply kits. Armed with malaria drugs and antibiotics, teams were not well prepared to encounter the kinds of health needs associated with middle-income countries, namely non-communicable diseases.

It was a frustrating situation. Not only were we unprepared, but our teams struggled with the change in focus. We needed substantial additional supplies to make our medical action fit the main needs. Even more frustrating was when we realized that an evaluation of our previous work in Kosovo had already highlighted the importance of being ready to treat chronic, non-communicable diseases in middle-income settings.

But still we failed to learn our lesson, and repeated the same mistakes after the 2010 earthquake in Haiti. From “Haiti One Year After” (2011), the report on our response:

“Furthermore MSF did not have the appropriate medicines on hand in the emergency phase to care for patients suffering from non-communicable conditions such as hypertension, diabetes, and epilepsy. Of 850 patients treated in one location between March and September, there were 72 cases of hypertension. Recognizing this shortfall in the package of available care in some MSF medical structures, the organization is already evaluating the feasibility of including chronic disease kits in the emergency preparedness stocks it maintains in different countries.”