The Revolution Will Not Be Aggregated

I’ve been talking to people about how they do fieldwork (see here, here). In particular, I’ve been looking for people who do it right, dollar for dollar the most effective folks in the field. No field offices, minimal administrative staff, but lots of expert researchers cranking out huge volumes of up-to-date, local information. They have author-to-manager ratios above 50:1, which I call a distributed NGO.

And then I talk with managers at these bleeding-edge projects, and they say completely insane things.

Like this: “We collect all our data in Microsoft Word.” Or “By the time we’re done, the .DOC files are around 20 MB.” Or “Eventually all the .DOC files get corrupted, so we print them out [!!!] and retype the data we need.” How many data points do they retype by hand? Oh, around 10,000 or so. Retyped. By hand.

A majority of the institutions I talked to use .DOCs or .PDFs to collect quantitative data.

You can get similar stories from me. I once had to split the Global Integrity Report results into 35 separate Microsoft Access databases, which I emailed to dozens of people, then spliced back together in a 12-hour white-knuckle hacking session. This is not best practice.

And these are the smart people. Why do they do things like this?

The short answer is because it works and it’s what we have. We scrounge and hack and piece together insane solutions because distributed NGOs are out on the leading edge of rapid innovation, where start-fast-fix-it-later trumps just about everything else. When your strategy is risky and new, off-the-shelf tools become more appealing.

The folks I’ve been talking to are not necessarily the best supported non-profits or even particularly well-known ourside of their respective niches. The people that have been helping me develop the Indaba fieldwork platform are not a cross-section of the NGO sector or development community at large. Instead, I see them as the vanguard of a movement that is searching for ways to sustainably scale fieldwork without having to constantly add staff or funding each time someone wants to replicate or refine a successful project. No field offices, no fancy mission statements, very little overhead, but lots of original data from diverse groups of people loosely connected by threads of copper and light.

These innovators are battling back a degree of complacency and inertia in the NGO sector (and the social sciences in general) that dates back generations and is frighteningly unchallenged today. In the classic “development” space, there is plenty of junk data that takes on an authoritative air simply because people haven’t put the effort into gathering better information. “Risk” indices that originate as marketing tools for management consultancies become the basis of ostensibly serious regression analysis. Perception surveys of loosely defined concepts benchmark billions in foreign aid. A handful of success stories get repeated at conference after conference regardless of their relevance or potential to be replicated. Good luck vetting source data for any of it: it’s frequently proprietary. UNCAC implementation assessments aren’t even published. How convenient.

Our insurgent NGOs are instead starting with a very simple concept: find the people that already know what’s going on. Connect online. Ask objectively verifiable questions, and then open the raw results to the world. There are some common values. Numbers should have commentary explaining the scores. References should be checkable. Peer review disputes should be published. Methodologies should be simple and repeatable. Indicators should focus on concrete things we can change, not abstractions like “rule of law.”

It seems obvious to me, but historically it’s rarely been done that way.

Last week, I sat in a room with people who — if they know it or not — have invented the distributed NGO. Their tools are hacked together with whatever software is lying around. And despite these simple, mostly inappropriate tools they’ve opened a firehose of original and useful information that is raising the bar for public interest research. It’s beautiful to witness.

The Indaba fieldwork platform is built for them. They already know how to do this. I think Indaba can make them better: quicker, cheaper, more sustainable over the long haul. But there’s more to it — I think the Indaba platform can serve as the catalyst for a community of practice, a rallying point for a new kind of fieldwork, and over time, a tougher set of expectations for the information that the development community, in particular, uses to make decisions.

Let me know what you think in comments below, or join the Indaba email list here.

– Jonathan Eyler-Werve

– Mumbai market photo by Akshay Mahajan (CC by/sa/nc)

– Title credit to Gil.