Crossing Borders

I spent the better part of the past week in western Kenya checking out the scale-up of IPA’s Safe Water Program.  This is one of the development interventions that IPA has evaluated and declared a “Proven Impact” project.  One of the things a lot of folks don’t realize about IPA is that when something does work we’re pretty big on seeing it go to scale – so instead of reaching a study sample of maybe 500 or even 5,000, taking the same intervention and trying to reach entire districts, countries, or even regions.  It’s exciting stuff but brings with it it’s own set of challenges.

Those four days were really the first time I’ve spent in Kenya since I studied there in 2008.  It was really nice to be back – I guess I can’t say whether it’s just a little change of pace from Uganda or whether there’s actual some intrinsic difference but it was refreshing nonetheless.  I can understand maybe a third of the Kiswahili spoken around me, which is a third more than I understand of any local language in Uganda; the food is definitely better; tuk-tuks are awesome; fewer calls of mzungu in the village; and a little bit more of a spring in the step.

One interesting thing – crossing the border from Uganda to Kenya you fill out a 5 by 7 inch immigration card which they barely glance at before picking up the rubber stamp and attempting to punch a hole in your passport.  However, returning to Uganda through the dusty little border town of Busia you now receive the same high-tech treatment promised at any international arrivals terminal in the States.  Metal detectors, fingerprint scanners, the little camera, 3M passport scanners, computers, the whole works – which works fine until the internet goes down, which of course it does at regular intervals.  It’s also just amazing to see what the government of Uganda is spending funds on these days.

Perhaps the biggest indication that I was back in Uganda was getting cut in the queue by a full grown man who then looked at me and said, oh, I’m sorry, as if  he hadn’t noticed that he had been standing behind me for the last five minutes.

The Uganda Rural Development & Training Program (URDT)

Tucked away in Kagadi Town, Kibaale District, is the Uganda Rural Development & Training Program (URDT).  I first heard of URDT through my mom after she was invited to a fund-raising event in Boston.  I had checked out their website but Kagadi is a bit off the beaten path so I hadn’t gotten a chance to visit until late last week when I traveled from Hoima to Fort Portal – a casual trip of 200km and 5 hours.  The website suggests they run a girl’s school, a radio station, a farm with progressive and even organic practices, a technical institute, a microcredit project, a land rights center, a recently established “rural university”, and more.  From all this I was a little doubtful, it just sounds like t0o many initiatives for one NGO to do well, and arrived expecting to find a local NGO with a comprehensive if half-way plan for addressing all the various aspects of rural development issues.

I was blown away; my idea of a 30 minute walkabout turned into a 24 hour layover.  Enoch, the Farm Manager, gave me what he called the “marathon tour.”  Not only does URDT actually have each one of those programs but the staff and faculty that direct the various initiatives are exemplary leaders, are passionate about their professions and rural development, and truly buy into the URDT ideology.  URDT is founded on the belief that rural development should focus on achieving goals rather than solving problems.  Sounds simple but in reality that is the difference between single-entity project oriented development and a comprehensive systems-wide approach to transforming lives.  The first says, people don’t have access to clean/enough water – let’s fix that problem and development will follow.  It’s about removing constraints.  The second says, this 13 year old girl comes from a very poor household and she wants to go to university and then go on to do [fill in the blank].  How can we make that dream a reality?  The answer to that question lies in the multitude of complimentary programs and initiatives that URDT has put together both on their beautiful 80 acre campus and in collaboration with the families and communities from which their students come.

The Girls School is home to 30 students per class, grades P5 to S6 (essentially middle & high school), all of whom attend completely free of charge.  Girls are selected based on their needs and the willingness of the family to buy into what they call the “Two Generation Model” where the girls come home during each break with a specific project or series of lessons to pass on to their families and communities.  Anecdotally it appears to be working.  The new African Rural University will open it’s doors this September to a class of 30 young women who want to become rural development professionals.  It’s an attempt to keep talent in rural areas and prepare young women for jobs that exist in the context of rural development.  The faculty has been involved in a five year curriculum development effort and are all excited to welcome the first official class of students.  I sat down with the University Secretary Jacqueline Akello and essentially asked, how’d you guys do all of this?!

While a good portion of the funding comes from outside sources, URDT was founded by two visionary Ugandan men and an Italian woman and started with a small office in Kagadi town in 1987 and they’ve been taking one small step after another ever since.  The Rural University is the capstone in their vision of creating a center that promotes rural development for girls, young women, local professionals and the community as a whole.  Co-founder and CEO Mwalimu Musheshe is an Ashoka Fellow and recently was asked by the Government of Uganda to serve as the director of the National Agriculture Advisory Services (NAADS) in an effort to clean up one of the historically corrupt institutions of this country.

One story that hit home for me: every morning the entire campus comes together for a one hour “Foundation Course” led by one or two members of the faculty, staff or student body.  Topics can cover just about anything related to URDT initiatives or rural development in general.  That morning the course was covering the different appropriate technologies being studied and promoted by the URDT teams.  It was an interesting summary of technologies, including modern bee-keeping techniques, rainwater harvesting, a bio-gas installation, homemade pesticides and a handpump mechanism for collecting water from underground tanks.  At the end of the hour there was a Q&A session and one man stood up and said Thank you for this, but many of us are already aware of these technologies.  Now we need to discuss how these technologies can be used for development, how we can integrate these technologies into the community.  That is what is really important. It was an effort to push the conversation to the next level and to me it spoke of the passion and determination of these individuals to really promote rural development in a way that many groups only talk about.  The question, I later learned, came from the man that directs the solar electrification courses at the technical institute.  Yeah, they do that too.

Check out their website but I will say in their defense that they are well aware that the site is dated and could use some work.  It sure beats a flashy website and a hollow NGO – and there are certainly plenty of those.

Work Update (Part 1)

It’s been over a month since I last bored you all with details of my work but believe it or not it’s not all safaris and cultural immersion these days.  So here’s what has actually been occupying the vast majority of my waking hours of late.  As I previously mentioned  I have created a lengthy survey which focuses on a few broad themes:

First, Household Roster – who lives here?  Second, Employment Details – what do they do for a living?  how much do they work and how much do they earn? Third, Consumption & Assets – what do they buy and what do they own?  Fourth, Water water water – what sources do they use? how far away is each?  how do they collect?  who collects water and how much time do they spend collecting?  how much do they use for various activities? and how does rainwater play into all of this?  Fifth, Social Networks – who knows who? who likes who? who is widely respected in the village and who is active in the community?  Finally, Behavioral Games – simple scenarios that try to elicit risk and time preferences, much in the same vein as my senior work at Middlebury.

That’s a gross oversimplification of a survey that has over 1,000 questions (when you considering asking many questions about each of the 10+ people in a given household, and asking other questions about each of 40 other households in a village sample) and will take about 90 minutes to administer.  So that’s kept me on my toes.  But what’s been the real focus for the last three weeks is moving this beast of a survey from a clean and simple Excel spreadsheet into a software program called Pendragon Forms which is used to conduct surveys on PDAs such as the Palm Pilot or in our case the HP iPAQ series device.  Forms is a very powerful tool in that it is relatively simple to program but still gives the survey designer lots of freedom – if they can figure out the tricks.  Once on the PDA the survey functions much like an online survey that you have likely seen before – questions and pre-coded answers that allow you to “select one”, choose “yes or no”, rate something on a scale of “1 to 10”, etc.  The difference is in our case the enumerator is the one inputting the answers, not the respondent.  The value of electronic data collection is hotly debated in this industry – even within IPA some professors swear by it and others won’t even consider it.  Pros and cons:

Pro – Complete control over the movement from one question to the next.  In all these surveys there are many skip patterns, ie: if the answer is A, go to question 5, if it’s B, go to question 10.  With a paper survey you can only hope that your enumerators are paying attention and following instructions.  PDAs give the programmer complete control over what action is taken every time a response is entered.

Pro – Erase the potential for illegible or incomprehensible survey responses.  The ink doesn’t run and the 5s never look like 6s.  That’s nice.  Also, if an enumerator accidentally enters an age of 150 instead of 15 I can set the program to catch this unlikely observation and prompt a correction on the spot.

Pro – Eliminate the total headache that is data entry.  The only thing more stressful then preparing for a survey is dealing with sloppy data entry companies, or so I’ve heard.  Fortunately I’ll never have to find out – at the end of each day the data from completed surveys are transferred directly to my hard-drive and immediately backed up (about a dozen times!).

Pro – Cut costs.  Printing 3,240 copies of a 40 page survey costs real money.  As does paying the sloppy data entry teams to enter each of those 130,000 pages, twice.  Twice not because they will mess up the first time (which they will), but twice because double entry is the only acceptable standard and the only way to conduct error rate checks, which usually results in us sending them back for round three.  Either way the manual entry of 260,000+ pages of survey data eats up a good chunk of change.  Even when you consider the cost of purchasing new PDAs for a large survey team the savings are real.

Personally those four reasons seemed pretty good so I threw my weight behind electronic data collection and we decided to take the plunge.  While I still believe this is the right thing to do and should pay off in the long run, using PDAs frontloads much of my work and demands a level of preparation above and beyond locating a safe dry place to store stacks of paper surveys  for a later date (ok, there’s a lot more too it than that!).  After three solid weeks of Pendragon Forms programming what I can say is that there is a very real reason why the program has a small band of IPAers around the world contemplating ways to rig a skeet launchers to toss up PDAs for target practice.  One example – the creators of Pendragon Forms decided that, as if correcting hundreds of little screens in a row wasn’t punishment enough, every time you hit the Backspace bar they would cheerily remind you of your mistake with a short sharp BEEP!  Thousands upon thousands of BEEP!  Another, there is nothing really intuitive about the program, you just kind of have to learn it step by step.  Where are those steps located?  In the user manual.  The 599 page user manual.

So those are the cons.  The pros still have it, but they are mostly long term while the cons are immediate.  I hope to be bragging about smooth sailing soon.

How hard can random be?

Work update:  I’ve had a busy week or so doing all sorts of things we describe using ambiguous and / or otherwise unintelligible words such as Stratifying, Sampling, Mobilizing and Pre-testing and now I’m preparing for a big Listing exercise.  So here it is, a week in the life of:

Imagine (purely hypothetical) you wanted to study the patterns of technology adoption among rural dwellers in a developing country.  Well who are you going to talk to?  These folks, the study participants, collectively make up the sample.  Sampling is a really creative verb used to describe the process of choosing who to include in the study.  Ok, so we’re going to sample a whole lot of people – makes for a strong paper – how are we going to find them?  Well, we’ll go to the villages and get a list of villagers (that’s called Listing, don’t worry, full explanation to come).  You’re going to do that in every one of the thousands of villages in Uganda?  No, obviously not, we’ll work from a study area – in this case the district of Kamwenge.  Ok, so you’re going to go to all 615 villages in Kamwenge district and get a list of households?  No, that’d cost a lot.  Oh, so which villages are you going to study?  Well we’ll choose some number of them at random in an effort to select a group that is representative of the broader study area.

Sounds good – but wait, does new technology reach really remote villages as quickly as it reaches trading centers located near the main road(s)?  Perhaps not.  But aren’t the vast majority of the villages in this area a good ways away from the main road?  Yep, looks like.  Ok, so maybe we should take some villages that are near the road, and some that are further away – that’s called Stratifying, or Oversampling, when you pick your sample to include various characteristics in proportions that do not exist in nature.  Ok, no problem, let’s grab the map of the entire district with all the villages on it and just – wait, we haven’t had an up-t0-date map for nearly a decade due to some impressive gerrymandering efforts.  Fine, let’s find some proxy for the size of each village and use that to oversample larger villages: plug some commands into the right statistical software and out pops a nice clean list of villages.  (I stopped to smile at this point)

Ok, so we’ve picked a whole bunch of villages, some near the road, and some further away.  So now we go march on in there and ask for a list of residents?  Naturally the records are well kept and there’s always someone in the office ready to assist.  Maybe not.  Mobilization is essentially doing a quick trip to each village to introduce the study to the local leaders, collect contact information, and explain to them what we’re going to do next.  It’s the prep work necessary for a successful Listing exercise.  The looming Listing exercise involves going to each village and conducting a small pow-wow with local elders and generating a list of each household in the village.  Sounds simple, but when we tried it out the other day the council told us “this is silly, we know there are 68 households in our village, here’s the list”.  We sat there until they came up with 76 households and then we went and did some spot checks to ensure that they’re all real and we didn’t miss any others.

Don’t worry, I didn’t forget Pre-testing: the word used to describe the constant process of revising the survey through trial and error.  Which survey?  The Baseline survey – what we usually consider the first step (ha!) in studying the patterns of technology adoption among rural dwellers in a developing country.

The Office & the Dorm-room

I haven’t really posted at all on what exactly it is I’m doing in that office for the better part of every day, so here goes: the first step of any evaluation is conducting a baseline survey to understand the current state of affairs.  A good baseline needs to ask questions about every aspect of a family’s life that one suspects might be impacted by the intervention under investigation.  For instance – if we want to understand the impact of a rainwater storage device on school attendance, we need to collect precise attendance records from both before and after the intervention.  That’s the easy part – the harder part is trying to build a picture of a rural family’s level of economic activity when many are at least partly self sufficient or are not paid in cash wages.

The first step to conduct a baseline survey is to build a survey, and that is what I am doing.  Typical baselines can run anywhere from 5 to 50 pages, from 50 to 400 questions, and need to pass approval by a Human Subjects board at whatever university the research is affiliated with (Yale in our case).  This process is the same (or very similar, not sure actually) for psych, medical, sociology, or any other type research involving surveys or experiments on people.  Although the approval process can be a pain, I think it’s actually a good check to ensure that questions are properly worded (not leading, not offensive, etc.), but that said, you really don’t want to raise any flags when you send in a survey for approval.  So, I am in the process of reviewing a wealth of previous surveys that were successfully conducted by IPA around the world and using the best sections from each to create one stellar survey.  I have also been able to observe a training of enumerators (survey conductors) and will hopefully get out to observe the actual survey process in the next week or so, just observing the process once or twice will undoubtedly help me organize my own survey in due time.  Once we have a polished draft of a survey and submit it for review it’ll probably time for me to head to Kamwenge to find a place to live, start talking to local officials and set up logistics.

For the last week I’ve been coming home to Nana Hostel – which IPA is slowly taking over, there’s four of us in here now.  It’s a pretty large establishment, 5 floors, probably 40 doubles on each floor, located just a 5 minute walk from Makarere University’s campus.  Randomly most of the university students that I’ve met are from Kenya (not representative at all); there seems to be something going on where kids come here from Nairobi because school is cheaper but arguably just as good, and then they go right back because there are more and better jobs in Nairobi than there are here.  I’ve noticed some subtle and not so subtle tension between these Kenyan guys and the locals on more than one occasion.  The rooms are pretty big and cheap, each has it’s own little balcony.  Laundry service for $2 a week was a blessing, except my socks came back about 18” long!?  There are stoves on each level but I am still lacking my own fork much less cooking utensils and pots, so it’s been a lot of street food and small restaurants for dinners – very much looking forward to my own place with a kitchen and cooking my own meals.

The sounds outside this place at night are nuts – as I started to write the little kiosk selling CD rip-offs was blasting some Jay-Z just as the call to prayer came out from Gaddafi mosque right up the hill.  The windows over the door to the balcony are just screens, so at night I hear: from 10 to 1am – general hoopla, loud music, the regular trucks/matatus/bodas sans mufflers, etc.  From 1 – 4 or so I’ve awoken to dog fights nearly every night – ok, maybe it’s just dogs howling, I haven’t investigated, but it sounds like a bloodbath.  And then about 6 to 7 every morning it could be VT – just roosters, cows, and the occasional old truck rumbling by.  That said, I’m actually sleeping alright.

K, time to go stake out a seat for the US –Ghana game, this is going to get heated.  Tonight we’re not the local favorites for the first time since the tournament began.  But there’s plenty of mzungus where we’re headed – but that’s for another post…

The Details

I have so far skirted around the meat of the issue – so what exactly am I doing in Uganda?

I’m the field researcher for an evaluation of a development project which will introduce a rainwater storage device to villages across several districts in rural Uganda.  The story goes as follows:  Relief International (RI), a US based NGO, has designed a new device which holds rainwater collected off the roofs of households via gutters and downspouts (hence, rainwater storage device…).  The Gates Foundation is the primary donor, and before they write the big check they want to know what the project is really going to cost and what the impact is really going to be.  Professor Mushfiq Mobarak (my boss) gets a call from the Foundation to do a rigorous evaluation using an experimental, or randomized, protocol.  Mushfiq is affiliated with Innovations for Poverty Action (IPA), a group of development researchers, who collectively hire a bunch of over-eager recent econ undergraduates like myself (and 2 of my roommates, as it were) to go administer a couple thousand household surveys and conduct a “randomized evaluation” of various projects around the world (more details on “randomization” to come in the future).

The Rainwater Storage Device:  Think really big rubber/plastic bag encased in an even bigger woven plastic bag (about 5 feet across & 5 feet tall when full).  The device is capable of holding 1000 liters of water (about 260 gallons) – estimated to meet the basic needs of a family of five for 10 days.  This capacity is not going to provide a year round solution, but rather during the rainy seasons and for a period stretching into the dry seasons, villagers will be spared countless trips to the creek or borehole (actually, we’re probably going to count them…).  After purchasing the device (not a give-away)  the villagers are responsible for constructing a simple earthen foundation to hold it upright and the gutters to collect and direct the flow of water into the bag.  A simple spigot at the bottom of the device allows users to withdraw water.

The Evaluation:  The evaluation is one part market analysis and one part impact assessment.  Using an experimental protocol we will be conducting a market analysis to determine an efficient price point for the device.  To do this we will be randomly allocating rebates of different values across different villages.  The adoption rate among those with each level of rebate will allow us to understand the demand for the product and inform future price points.  In addition, various marketing strategies will be used across villages and we will be able to compare the results of each.  Then, using household surveys from before, during and after the introduction of the device we will be able to understand the impact of purchasing a device on household welfare.  Specifically we will be looking at the impact on women and children who are primarily responsible for gathering water.  From a development standpoint the ultimate impact would be higher school attendance, allowing women to be more economically productive, less water-bourn illness, etc.

This is still a very rough sketch, but I’ll wait to fill in the details in real time as they become relevant.  Anything that is unclear above would probably make a good blog-post topic, so let me know!

About

A collection of observations, thoughts and experienced going back to 2010.