Technology's Humanitarian Purposes

The expanding use of technology has many implications for responding to humanitarian crises. How is technology being leveraged in these instances and how can it be most effective? Laura Walker Hudson and Patrick Meier discuss the application and innovation of technology for rapid humanitarian response.

GRAFF: Good afternoon, everyone. Welcome to the Council on Foreign Relations for today's meeting. I'm excited to have these two people join me for today's conversation. My name's Garrett Graff. I'm a term member at the Council and editor of Washingtonian magazine here in D.C.

And joining me onstage today is Laura Hudson, who is the chief executive officer of the Social Impact Lab Foundation, which is the maker of Frontline SMS, which is an open-source technology used in lots of ways she is about to explain to us.

And then also today, we have Patrick Meier, who is the director of social and humanitarian innovation at the Qatar Foundation Computing Research Institute.

Patrick, we'll start with you today. Give us a little bit of a broad picture to frame today's conversation on technology and the response to humanitarian crises. Talk about the Pakistan earthquake that we've seen this fall and the role that technology played in the response there.

MEIER: Sure, happy to. Thanks for being here. Looking forward to your questions, as well.

So maybe I should qualify. And this is the conversation we had, I think, last week about what we mean by technology and so on. What I'm going to focus on, really, is information and communication technology and the use of advanced computing to make sense of information. There are other types of technology, such as using 3-D printing for developing shelters and building shelters during disasters and so on. That's not my area of expertise, so I'm going to part that and let somebody who's more intelligent on that particular front to speak about that.

So on my end, it's looking at the role of technology to strengthen humanitarian action and information. So one of the challenges that we've had and is a recurring challenge over the past sort of few years is the overwhelming amount of information that becomes available via social media in particular during -- during crisis, so this vast volume of user-generated content that gets posted.

And what we've realized over the past few years, in partnership with a number of humanitarian organizations, is, well, the overflow of information during a disaster can be as paralyzing to humanitarian response as the absence of information. So we were in a world of scarcity, where there was never enough information, disaster happened, it took us days to find out what -- really, who'd been affected and how and how badly and where and so on. And now we just had this explosion of information.

So the work that I sort of focus on is developing humanitarian technologies that can make sense of big data or big crisis data during disasters. The point about the Pakistan earthquake, the U.N. Office for the Coordination of Humanitarian Affairs a few weeks ago, within a few hours of the earthquake, asked us to collect all the tweets that had been posted during the first 48 hours and then identify which tweets referred to infrastructure damage, as well as urgent needs.

And there were about 35,000, 40,000 tweets, and we had to go through those very, very quickly, and we used something called microtasking, which is an example of human computing. It simply means that we had volunteers go to a website, and the website would list one tweet at a time, and the volunteers would basically tag that tweet as saying, OK, that has to do with infrastructure damage or not. So it's really single-click sort of volunteering that helps you really try and manage big crisis data. So that's an example, and I can go on to more examples.

But maybe I'd put a caveat to that. Of the 30,000 tweets that we went through in a couple days, none of them were actionable and referred to infrastructure damage or needs. That's because we're talking about southwestern Pakistan, very, very little to no social media footprint, so that's another sort of disclaimer here, that these technologies are not going to work everywhere. They're only going to work in certain situations.

GRAFF: So, Laura, talk to us a little bit about the role that Frontline SMS plays in responding to disasters.

HUDSON: Yeah, sure. So we come at things from a very different angle from a technology point of view. I suspect that there are very many smartphones in this room. I certainly have one. There may be a few feature phones, which are the ones that are not smart that still have a color screen. There may not be very many with the black-and-white screen that we remember from years ago.

SMS, or text messaging, is the most widespread digital communications platform that we've ever seen. We have new research that came out I think just last year or this year that says that, although there are 7 billion mobile phone connections, about -- there were actually only about half the number of individual subscribers. So half the people in the world don't have phones at all, and those that do -- or the overwhelming majority don't have access to mobile data and they don't have access to smartphones, either, because they're expensive and they're expensive to run.

But what people do have is low-end mobile phones. This is technology that people already have in their pockets. And so we have built and distributed Frontline SMS for nearly eight years now. Actually, eight years, it's October, so it's been out for eight years. And we've just released a new cloud-hosted version of the software.

And what it does is it lets you send, receive, and manage SMS. And you don't have to have the Internet for it to work. So why is this important? Because in places where nothing works, where there's no good Internet connection, where electricity is patchy, SMS becomes really critical technology, including in emergencies, where perhaps infrastructure may be down, including mobile networks, but we saw in Haiti, as they came back up pretty quickly.

So SMS becomes critical. And what you can do in the situation where nothing else works is you can run your business with it or you can run your local response organization. You can send and receive information. You can manage your staff. You can manage deployment and disbursement of materials. And SMS then very quickly becomes very important.

And what we tried to do is make managing SMS in a professional way very accessible. You can just download our software from our website, get yourself a modem and a laptop, and away you go.

MEIER: If I can connect that, because it's a perfect set-up, because we were just talking before -- before the presentation. So one challenge that we had on the SMS side, we were both involved in the earthquake in Haiti, right, three years ago, where we're receiving tens of thousands of text messages coming in. And we used microtasking to translate the text messages, so the Haitian diaspora went online and they'd be -- they'd see one text message in Haitian Creole at a time, and they translate it.

Then we had to triage it, right, find out which text messages were urgent, life and death, and the truth is, we had a growing backlog every day of thousands of text messages that we never got to, because we had a dozen volunteers using microtasking, but it wasn't enough. And ultimately, throwing more volunteers at the problem is not the situation.

And during the search-and-rescue phase, we had Digicel, the telecommunications company, offer to send out a blast SMS to their 1.4 million subscribers across Haiti to let them know to text us if they had urgent needs to report and so on, and we said absolutely not. No way in heck should we do this. First of all, there's managing expectations that comes, this huge, huge issue, but we could barely manage several thousand text messages coming in per day, right?

So how do we -- that's what I want to do with Laura is collaborate and find a way to develop advanced computing solutions so automatic classification of text messages -- there's already a precedent in Uganda with the UNICEF project -- and apply that and bring it to other places with free and open-source software. And that's what I'd love to do. That's very selfish of me, but I just wanted to say it.

GRAFF: Well, I think you raise an interesting point here about managing expectations in terms of the ability of people who are victims in a natural disaster to be able to now summon help or think that they are summoning help in a way that's possible that someone is never actually going to see their equivalent of a 911 call.

How -- as a responding agency -- do you help to ensure that you are not raising the bar beyond which the fields can actually provide?

MEIER: Hard question. You want to take that, Laura? I'm not in the -- I'm not working currently for a humanitarian organization. I can tell you, in Haiti, what we did is we went on local community radio in Port-au-Prince. We had partners there, Internews and Reuters. We went on Haitian diaspora television and radio, as well, to let people know, listen, there is no guarantee of a response. We're sharing this with humanitarian responders at the time. It was really the U.S. Marine Corps, and they're going to prioritize the response. We cannot guarantee anything.

I would never want to do what we did in Haiti again. That's another caveat, is I think that was interesting, but it could have gone wrong in many, many different ways in terms of managing expectations.

HUDSON: I mean, what I would say is rather to slightly move the frame of the discussion, if I may, because -- so I have a background in humanitarian, and I used to work for an international response agency and policy staff in London. But what we remember -- remember the Red Cross movement and the great privilege of being -- of working with the Red Cross is that you see how powerful and how important, actually, the national societies are at a national level.

And I think if you look -- if you go to something like ReliefWeb, which is a website that shows you, you know, what's going on in disasters globally and you look at the list of -- even from the last two months, say, you pull out very quickly sort of 11 or 12 emergencies, sort of cholera in one place and floods in seven or eight countries that don't hit the news. We don't hear about it. International agencies don't deploy a response to these emergencies. It's not because they're not important or because people aren't affected. It's because they don't need to, because the people on the ground are already responding, as we do in the U.S., right?

So you have infrastructure, you have local responders, and it's a combination of government, state and local resources, but also civil society organizations, the American Red Cross, church groups, sort of local unions, and other entities come together in a time of crisis to respond.

So when we look at emergency response, we look at it much more as we understand that, you know, most emergencies don't make the news, but they all get responded to by local responders. Even in Haiti, it was very difficult for people to get in for a good 24-48 hours, because the infrastructure to bring in people and material just wasn't there any more. So you always have local responders there first, and empowering them is an important thing to do.

And it's also important to do that before something happens, right? If you are deploying a technology in an emergency, it's too late. It needs to be there beforehand. And a great example of what we try and do is an award-winning use of Frontline SMS by an international agency called ActionAid who you might have heard of in Kenya, in Isiolo, which was experiencing drought and famine a couple of years ago. ActionAid Kenya, the national chapter, were responding to that, and they implemented Frontline SMS to do staff management and primarily to communicate with the community, which is important, also, to say, right? The community are themselves first responders. Many first responders -- like organizations are drawn from the local community, and also communities are resilient in and of themselves and can help themselves.

And so ActionAid were communicating with the community such that they knew when they were going to turn up with food, and the community could plan around that, which is -- it sounds like a minor thing, but actually this wasn't happening before, so the community was waiting all day for a disbursement of material that might happen in the afternoon. And this changed the power balance and made it much more of a -- you know, they were doing -- they were in it together, so much so that as often happens in these places, something else happened, so there was suddenly insecurity in Isiolo, and it became unsafe for people to work in the area, and the community was forced to flee their village.

But they texted ActionAid and said, "OK, we've had to move, but we're here now, and so the food distribution moved." Now, that's pretty groundbreaking. I'm not aware of many -- many times that happening, but they're still using Frontline SMS, and so it becomes much more of a community sort of engaged response, which is what we want to see. We want to see people being able to use SMS to communicate with people, even, as we say, in the last mile.

GRAFF: So, Patrick, how -- you raised earlier in talking about Haiti and talking about the Pakistan earthquake the problem of the volume and velocity of data that you are sort of hit with in responding to humanitarian crises. How do you as a responding organization or someone sort of managing the technology behind the responding organization try to sort the good data from the bad? I mean, when -- this is something that we saw even just in Hurricane Sandy in the United States a year ago this week, where you have just such a tremendous volume of social media discussion that not all of it is at all relevant to actually responding to help.

MEIER: It's a great question. I'm trying to keep it short, right? So the challenge -- to use sort of an analogy -- is the needle-in-the-haystack problem, right? You've got a giant haystack. That's what big data is. If there wasn't big data, there would be no haystack, and finding the needle would be straightforward, but big data means we're facing vast amounts and growing stacks of information during disasters.

So, number one, how do you find those needles, not six months later for a research paper you're presenting at a conference, but within minutes? That would be really the idea, not to say there's anything wrong with conference papers. We benefit a lot from these conference papers to develop the technologies. But, obviously, it's not going to be useful for disaster affected communities.

So the first challenges is the velocity volume. How do you extract meaningful, actionable, informative content in real-time? And, second -- and you hinted at that -- is, well, how do you know that information is reliable to begin with? Is it credible? You know, talk about the veracity and so on.

So there are two ways to -- maybe I'll talk about the ladder and then -- and we can talk more about the volume, but in terms of a veracity, the first thing to keep in mind is that, you look at the New York Times, which is sort of the pinnacle of Western journalism, and the fact is, they make thousands and thousands of factual errors every year, right? And they recognize this, for the most part. So the pros -- and the best of the best -- make thousands of factual errors in their news reporting, even when it's not real-time. So you have to keep in mind that the best of the best, they're not necessarily perfect. They're doing what they can, and that's the nature of the beast.

Now, let's talk about a -- perhaps the longest-running crisis-based crowdsourcing early warning or crowdsourcing system is 911, right? It's been around for 50 years. 999 in the U.K. has been around for 75 years. That's sourcing from the crowd, right? And if you look at the statistics, in terms of the false kinds of numbers of calls that 911, say, in New York gets or in the U.K. or in Greece, I think 911, there was a 10 million false calls or hoaxes or misdials per year in New York City alone, right?

So, again, huge amounts of false information coming into the 911 system. Does that mean we should shut down 911? Of course not. That would be lunacy, right? They're looking at better ways to manage this particular challenge.

And in the social media for emergency management space, we're also looking to manage this better. So there are two things -- I'll give you one example of a good research paper that a colleague of mine did, and it showed that basically you can predict -- using computational social science and so on, you can predict the credibility of tweets without even reading the tweets. And I can get into why that is maybe in a bit, but basically Twitter is a self-correcting mechanism, and using big data analytics, you can find out why certain tweets might be credible versus others, and a partner of ours showed that you could predict whether an image during Hurricane Sandy shared on Twitter was fake or not just by doing some big data analytics on the content of the tweet, not even looking at the picture.

That's where big data analytics really becomes very, very interesting, because we have millions and hundreds of millions of tweets that we can -- and analyze and look for patterns. And we see that certain types of tweets tend -- based on their features, their content -- tend to relay false or misleading information. So we're developing a plug-in for Twitter, a credibility plug-in, that will score your tweets based on how likely -- how credible the information shared is. That is no -- by no means a silver bullet, though, right?

GRAFF: I hope it doesn't involve Washington pundits, as a credibility plug-in for Twitter.

MEIER: And real-time, but...

GRAFF: That would be real damaging to a lot of Council on Foreign Relations Twitter accounts.

(LAUGHTER)

MEIER: And it could be -- I'm sure it'll be -- I know it'll be game-able, right? The point is, there's going to be -- it's going to be a cat-and-mouse issue, and it's going to be going back to the analytics and seeing whether we can find out how to, you know, fill a loophole or something -- or stop -- block a loophole and go back and forth. So it's not going to be a one-off, here it is, the ultimate Twitter credibility plug-in. It's ongoing research and trying to get ahead. It's a spam filter of sorts, right?

HUDSON: I mean, if I could just chuck in an additional thought, it's just -- I mean, one of the things we -- if that's all right -- we bang on about all the time is that the choices platform of communications platform is political. So not everybody in this room has a -- I'm going to go ahead and guess that not every single person in this room has a smartphone.

So not everybody is able to tweet rebuffs to everything that Patrick and I are saying, which obviously we deserve. But that means that those of you who aren't tweeting -- heavens -- are effectively excluded from any kind of backchannel debate that may be going on, on Twitter, so we're missing out on what you have to say and you're missing out on the opportunity to say something.

And I guess I would just say, so big data is enticing, and I'm not saying we should ignore it. I'm not saying this is an opportunity that we should seize. I'm rather saying that, who is not represented in these big data sets? Who is missing? It's the people who don't have any phones at all. Even adding SMS into the mix is great, but there are barriers to entry with SMS, also, which is why we suggest very often, you know, adding radio as an excellent way to get messages out and then allow people to interact by SMS or voice calls, but, you know, expanding beyond people who maybe don't have access to a phone or aren't literate.

And the other question is, so who is absent from big data? And does -- by shaping our vision of what we should be doing in response to an emergency by some of these large data sets that effectively don't represent some people, is there a risk that their voices and needs are missed?

And then the other thing I would just ask you is, there is a big -- and I'm just happy to leave this maybe for a later discussion -- but there is an issue of ownership here and informed consent to use of this data, particularly not necessarily with Twitter and stuff, but with government data sets that are beginning to be released, which is an interesting one.

GRAFF: Laura, you also touched on something that Patrick mentioned a little bit earlier, which is beginning to use some of these technologies to predict and help plan for disasters in disaster-prone areas. Can you talk a little bit about the ways that these technologies are being used in the days ahead or months or years ahead of something occurring?

HUDSON: I mean, I think for us, again, it comes down to, very simply, empowerment and preparedness. I mean, disaster preparedness, as I'm sure many of you all know, is underfunded compared to what we know could be the impact dollar for dollar against disaster response. We know that we should be spending more on disaster preparedness. And we know that there is a huge impact of that. And I think you've said -- some -- I've seen some cost-benefit analyses that say that you save $17 for every $1 spent on disaster preparedness, so we know it's important.

We also know that local responders respond first and often, and that investing their capacity very simply is disaster preparedness in and of itself. And also, you know, community resilience and capacity to respond and to be resilient to emergencies.

So I think that's important. I think we can see where there are recurring emergencies, where there are floods every year, where there is a risk of drought. Some of these are becoming less easy to predict with climate change. Of course, that is a big problem for the international humanitarian system and for governments all over the world to deal with changing -- even Australia, the changing conditions in which fires occur, for example.

So I think that's an issue. I would say for us, therefore, that, you know, this issue is sort of building capacity ahead of time is the most critical one and the easiest to address. We know how to do it. We can just do it. Let's go.

But I would also say, just from my experience -- this is where I work now -- that hazard mapping and, you know, seismology and these sciences are very evolved and advanced, but a big challenge in the often very pressured world of humanitarian response to have time to absorb what's coming out of the sciences and to feed them into everyday work.

But I would also say that that is the role of government. International disaster response law says that there is a role for government in this work and that where we see there are gaps, it's because government capacity isn't there. So we should also be investing in that, I would say.

GRAFF: Patrick, I'll ask you this question first, but I imagine Laura will have some thoughts on it, as well. Talk a little bit about the differences in responding to humanitarian crisis technology-wise in the developing world versus the developed world, I mean, how -- we had Hurricane Sandy in the United States last year versus Typhoon Pablo in the Philippines, for instance.

MEIER: Sure. And then just to also connect and echo what Laura just said, there's a great article in the Stanford Law Review, I believe just last month, called "Big Data and its Exclusions." And that should be required reading. It is phenomenal. It's very lucid, and it really gets to the heart of the issue of what happens when you look at big data. And it's not even focused on disaster response, but it has very important ramifications for humanitarian response. So that's one thing.

And also, there are other studies and guidelines now that are coming out in terms of big data and biases and who you're excluding, as well, that I can -- I can share with you after the talk.

In terms of comparing the use of humanitarian technologies in places like New York versus, say, the Philippines, well, maybe that's not the best comparison, but even if you look within New York, right, my colleague, Kate Crawford at the Microsoft research did a great op-ed in the New York Times a few months ago and clearly showed that the vast majority of tweets that were coming out during Hurricane Sandy were from affluent, rich, white areas, that there were basically no tweets coming -- very little social media activity on Facebook, as well, coming from more marginalized, low-income neighborhoods. And that's really, really serious.

We need to keep that in mind. It's not just in developing countries that we have this digital divide. It exists here, as well. Now, the thing to keep in mind -- in the U.S., you know, you still have access to 911, right? So -- and I don't think either Laura or I would say anything like, oh, you should social media or SMS at the expense of everything else. Everything else, all the traditional, established humanitarian information management systems and emergency management systems should go out the door, of course not. It's only a way to complement that and just to add another data feed, right? So that's something to keep in mind.

And also, to put in context on 911 -- and then I'll jump to the Philippines -- it took two decades for 911 to become mainstream and implemented across the United States, two decades. It was really until the early 1980s that finally the vast majority of the U.S.'s population, municipalities, towns, and so on, had their 911 system up and running. Does that mean that in 1960, when this started being -- you know, there was a federal mandate to roll this out, that people who made calls to 911 in 1962 from Oklahoma shouldn't be responded to? No, of course not. There was a huge digital divide when it came to 911, and there will always be, I'm afraid, some sort of digital divide. It doesn't mean that those who are connected should also be ignored, because that's just another form of discrimination.

What was really interesting in the Philippines is the U.N. asked us to -- within about 48 hours of the Typhoon, Typhoon Pablo making landfall, which was one of the worst storms, typhoons to hit -- devastate the Philippines in 100 years, they asked us to go on Twitter and identify pictures and video footage of captured evidence of infrastructure damage and then to map that, to identify where those images and video footage had been taken within the Philippines.

And we were able to turn this around in 12 hours, and it worked really well in this case, not as it -- you know, in Pakistan, there was no social media footprint, but here in -- the Philippines has a high, high level of social media activity and has had for decades. And here we got a lot of very relevant, credible information of pictures, of damaged houses, bridges, and so on, and we were able to map within 12 hours, using microtasking, by the way, so distributing, you know, the load to individual volunteers from all around the world would go on, look at an image, and then click on whether it had severe damage or not, and other volunteers would then try and find out where in the Philippines that image had been taken, if there was evidence in the image or on the tweet that said, oh, in that particular city.

What's interesting there is, it's -- this is a developing country, worked very, very well because it was a high social media footprint, and just talking last -- the week before with my colleagues at the American Red Cross, they're very interested in this idea of microtasking, right? So that could be applicable here, as well.

But then again -- and I'll end with this -- and that's why I led with the example of Pakistan, is if there's no social media footprint, then forget it. There may be text messages, in which case that has to be usually in place ahead of time, with a little luck, if that's possible, and then, not only that, what happens if the telecommunications company tells you, OK, we're going to blast 2 million cellphones to send text messages to these cellphones and ask them to report on damage. What are you going to do when you get even 1 percent of people replying to that? That's going to be big data; you have no way to respond to that.

And that's why, again, you're going to need to start thinking about using advanced computing, and there are free and open-source technology that are being developed now, including in my institute, to manage this kind of large volumes, whether it's text messages -- and we're even looking at IVR, integrated voice response, to quickly tag messages, voice messages left on a machine during disasters, because obviously literacy is not something you can just assume everywhere.

So we have to be creative. And it's not going to be one solution. It might be a combination. And even then, you might still fail. It may not work. And that's just something we have to be honest about and upfront and transparent, you know, just like with Pakistan. We acknowledged right away, we looked at 35,000 tweets, totally useless, right? OK, that's how you learn.

HUDSON: So I would just add in SMS length, because I can see that we're supposed to move on to the discussion. So I would say, yes, absolutely, digital divide, the platform is political. We shouldn't forget that there are people in the developed world who don't have access to all the same technology that the rest of us do, and we also shouldn't forget when we look at the developing world that it's supposed to be different. Government and the state and the community are supposed to be able to respond to all but the biggest disasters, you know, themselves, and that's what we should be working towards, disaster risk reduction, managing -- identifying and managing risk, and preparing communities and organizations for disasters so that they can respond themselves.

GRAFF: Great. So at this point, I'd like to open it up to questions for the second half of our discussion today. A reminder that we're on-the-record today and anything you say can and may be used against you later on.

(LAUGHTER)

So when I call on you, please wait for the microphone and then state your name and your affiliation. We'll start out right up here.

QUESTION: Hello. My name's Timothy Reuter, and I'm president of the D.C.-area Drone User Group, which teaches people how to build and operate their own platforms and encourages them to use them for socially beneficial purposes. And I actually think there's just a lot of potential for community-owned, low-cost aerial platforms to play a role in this.

You know, I was interested in hearing about how the technology has changed over time. I was actually a little underwhelmed when you talked about the microtasking platform, because you then said, you know, it's 12 people. I think there -- who were in Haiti translating things and tagging them?

MEIER: Yeah.

QUESTION: And so, you know, I'm wondering how you see that evolving, because I would have imagined that you could tag a much larger community to do those kinds of things. And I'm interested to hear what you think now is currently ripe for automation, maybe things like translation and that kind of tagging, what you think will soon be ripe for automation, and in the long term what you think technology will be able to do, because I'm sure it's changing every day and we need to think carefully about, you know, what do we devote humans to and what do we devote computing cycles to?

MEIER: Well, I'm glad we -- I was hoping to bump into you at some point and been talking about the D.C. Drone Group and joining it, and a few colleagues of mine -- actually, the weekend before last went out and flew a few drones around just to practice for these kinds of ideas. We're all in the development, humanitarian space, and we think there's a huge value to letting communities sort of take their own imagery and so on, so I'd love to chat with you. I'm really glad you're here. Thank you for coming.

So in terms of the microtasking, you have to keep in mind that Haiti was the very first time that we -- overnight, literally, Brian Herbert at Ushahidi created a simple web-based microtasking platform to do the translation of the text messages. Now, that worked pretty well. We had about -- we -- there were about 1,000 volunteers from the Haitian diaspora who translated tens of thousands of text messages with a turnaround time of about 10 minutes.

What we did not have a microtasking platform to do was then triage the translated information. It was all coming to the back end of the Ushahidi platform, which is as far from a microtasking platform as you can possibly think. It was -- we had to export all these text messages out of the Ushahidi platform, use a ticketing system to tag them, and then import them back. That was a complete disaster. We weren't set up. We were making it up as we went along, hadn't been done before, and that's why in many respects, on a data -- on a big data front, quote, unquote, that's a hash #fail. That's filter failure, brilliant example of filter failure. We still got about 1,500 text messages (inaudible) course of a few weeks, but there were still many more left, right? So you're absolutely right.

So one of the projects we've just launched with the U.N. -- and that goes to recruiting volunteers -- UNOCHA is called MicroMappers, which is a very first set of microtasking apps that are directly -- specifically customized from the onset for disaster response, for digital humanitarian volunteers to tag information. That's what we used or tested in Pakistan a few weeks ago. We tested out a few of the microtasking apps. We had 109 volunteers go through about 30,000 tweets in about 30 hours.

It is what it is. It was a test. It was not ready for primetime. But what -- I talked to a group called Zooniverse, and they're the -- as far as I'm concerned, they're the masters of microtasking for citizen science, right? They've got a list of a million -- just under a million -- I think it's 980,000 volunteers from all around the world, and so they tap that listserv when they have to do some citizen science and microtasking, for example, looking at images of Mars and tagging when you see a dune on your particular part of the Martian landscape. They were able to basically tag 2 million images in 48 hours. That's what I call microtasking, right? And that's what I want to get, and so we're in touch with them and running as much as I can from them, and we're growing a listserv.

But the key is really to make it as easy as possible for anyone -- if -- you know, we talk about clicktivism in the digital activism space. I want clicktivism. Let's make it as easy as possible. If you can click "like" on a Facebook page, you, too, can be a digital humanitarian volunteer. Until recently, we haven't thought enough about engaging the rest of the crowd into being volunteers.

So in terms of what can be automated and not, we're -- that's exactly the core research that we're doing at the Qatar Computing Research Institute. And at the same time, however, I don't think any of my colleagues and I are thinking that it's going to be 100 percent automation and no human involvement. What we're doing is we're combining something called machine learning with microtasking, so you always have the human teaching the algorithm what the human is looking for on Twitter. I'm looking for infrastructure damage, and I give examples to the platform, to the algorithm. You know, these are 10 tweets or 20 or 50 tweets that have to do with infrastructure damage during a current disaster right now, go fetch more. And then it's basically a hybrid approach. It's human computation with machine computation together.

Translation is getting very good for a few languages, but, you know, when it comes to other languages, then it becomes a huge challenge, as well. So for the MicroMappers, one of the apps that we're developing is a translation app that will use Google's Translate feature to translate a text message or translate a tweet, right, under the original tweet, and then have a text box for a volunteer who speaks that language to correct the machine translation or leave it, if it's good enough, because it doesn't have to be perfect, right? It's who, where, when, what. So that's some of the stuff we're working. But I would never suggest that a fully automated system -- I don't think that's what humanitarians want.

GRAFF: Laura, any...

HUDSON: Yeah, just -- I mean, just quickly, I think when you look at SMS -- so Frontline SMS, you know, deals -- it has dealt exclusively with SMS, and Frontline crowd at the moment does, also, but when you look at SMS as a platform, it is uniquely powerful and uniquely -- because of its reach and its, you know, relative brevity, it works -- it's very low cost, if you can get the -- the agreements right and so on.

So it's uniquely powerful, but also uniquely underinvested in as a professional tool. It's actually -- I don't know if you've ever tried -- but it's extremely hard to set up an SMS project. And someone's already said you need six months to get going. It shouldn't be that hard. It really shouldn't. If you can stand up an e-mail server or you can stand up an e-mail account in a day, why can't you do that with SMS?

So we're working to do that, basically. We recognize the potential power of it as a professional tool. And we want to empower organizations in places where SMS is critical to be able to use it as it should be able to be used. So we want you to be able to be -- to set up an SMS, incoming and outgoing number, in a few clicks in as many countries as possible. That's currently very difficult outside of developed countries.

But, further, you know, we recognize that -- so I lived in Kenya for a couple of years, and what you -- what you realize is you are always somewhere on a shifting spectrum of connectivity from none to -- because my phone's gone dead because there's no power or because the master's down, to, you know, having cell capabilities, you know, data, to having, you know, 4G data or 3G data, which is now rolling out in some cities.

And so -- and this shifts over time. It depends where you are. It depends if you're traveling. It depends what kind of phone you've got and many -- increasing numbers of citizens, particularly in the middle class, even not middle class, exist on that continuum and move up and down all the time. And so do offices. Sometimes they have connectivity; sometimes they don't.

So we need to build flexibility. We need to build multiple channels in there. We need to make it possible to chat using IM. And then the other thing, when you start to look at this professionalized use of SMS, is you could then potentially have management information on organizations that have never been digital before. So as the Red Cross society, say, you can imagine being able to see information about your volunteers at branch-level for the first time.

And so being able to hook that up, aggregate that information, you know, and really empower organizations -- that's what it comes down -- whether they're businesses or not. That's where we're going. So we're excited about the future, low-tech as it is.

GRAFF: And then you next.

QUESTION: So my name is Sarah Agelwat (ph). I work for Hewlett-Packard. And in the spirit of open disclosure, my company bought a software company that's proprietary that manages unstructured big data last year, so my question is a little bit around that.

And I'm just -- I'm actually honesty befuddled by why big data is not used more, even with SMS technology, because any time you have basic electronics, you can utilize that kind of technology to take what is basically a lot of unstructured things coming at you and managing it better.

And I just -- I hear you talking about open-source data and it starts to raise sort of cost questions. Is this really just a cost issue? Because what is really keeping this kind of technology from being more broadly used for humanitarian purposes?

I mean, when I look at what our company is doing, you see a lot of demand for this stuff in the security spectrum, both in developed and underdeveloped countries, right? So governments are very happy to invest in it when they can, you know, monitor Facebook usage among their citizens and know what they're saying for perhaps positive and nefarious purposes, as well, but I just wonder why -- why we're not taking this a step further and why the aid agencies themselves are not playing a bigger role in that.

And I want to just sort of add a quick comment that, I mean, you could also argue that tech companies like ourselves could play a role. I mean, my company last year invested in a humanitarian cloud that we did jointly with Microsoft and others which supported USAID and other organizations that were operating locally. So we are thinking about these things. We are doing it. But I just -- really, the cost is not that big. And so I just wonder what in your mind is keeping these kinds of technologies out of the humanitarian domain?

GRAFF: Laura, you want to take that easy question?

HUDSON: Well, so we at SIMLab and others -- I know I've seen it crop up a couple of times in the last few days -- have started talking about data feudalism, right? We are currently in a situation where we don't own our data, we are the product, and the data is a much -- many, many different data sets are concentrated in private companies and private hands or, you know, governments or mobile network operators.

So whether you're talking about Facebook and what it knows about you or Google and what they know about you and how can you get access to that or whether you're talking about being able to go through the mobile network operator to reach all the subscribers in a certain area to be able to tell them there's a chemical leak and they should evacuate, whatever you're talking about, somebody owns this data.

And so I would say, you know, we need an honest conversation about that. We need it to be informed by the perspective of private citizens and also by the potential humanitarian uses. It is relatively low cost, but I think that that's in a monetary sense. I think we have to think, also, about the cost of privacy. And, you know, it's not that there are tradeoffs around -- whether it's this immense tool would be used wisely, I think. There's a huge conversation, but that would be what I would say.

MEIER: Private sector in this space -- I mean, we've seen it's a huge multi-billion-dollar market now, I think. It just -- or real-time intelligence with social media, brand perception, and all that stuff. I did collaborate with a number of different companies to see what their platforms were, and they were very happy to let me use it for free. One of them were still using on an ongoing basis -- at least looking at it when there are disasters, see if we get any added value.

I think the barriers are -- definitely the license are too expensive. You have to keep in mind that the U.N. budget for information management is not increasing. It's being cut every year. And while the term open-source was a bad word before 2010, before the Haiti earthquake, it would just stop the conversation, the U.N. said, oh, we've got this open-source. They'd say, you're amateurs. We're going to go with proprietary private-sector and this and that.

That's completely changed now. In fact, there is for a number of reasons an interest in working almost exclusively, whenever possible, with free and open-source software, because of the transparency, because then you're not dependent on that particular company to change the code. You can hire your own developers and so on. That's -- I'm not -- I'm showing what I've learned in the process. It's not a judgment on my end.

That said, my colleagues at FEMA, who do social media monitoring, do use a proprietary platform. I don't know if they're paying -- I think they're paying for it, so maybe at the FEMA level, that will work. But when you talk perhaps that -- Kenyan Red Cross, they're not going to fork out a license. They might get something for free from IBM Kenya to use, but there's been a huge shift towards free and open-source, and now maybe a bias towards free and open-source that may or may not be legitimate.

But I think in addition to cost, it really is just ease of use. I've seen that be a huge, huge problem. A lot of these systems that have been developed by the private-sector companies are incredibly sophisticated and meant for people who are digitally literate, who use computers on a regular basis, who maybe have a master's degree and so on, and all these assumptions just don't make sense in a lot of the contexts that -- at least that I work with and that free and open-source platforms that we're developing at QCRI are really meant, first and foremost, for disaster-affected populations. And even there we're struggling. Even with that in mind starting from scratch, it's hard to think about, how do we make it easier? How do we make it more accessible and so on? So it's not just useful for disaster -- professional paid disaster responding.

So ease of use is a huge deal-breaker sometimes. People will just not use it. And I think it has to be a user-centered design. Start developing these technologies with the end users right away, not say, OK, this has been used in the private sector, now let's square the circle, and you can use it -- I don't know -- Kenya or Tanzania.

But I continue to collaborate directly with a number of private-sector companies who are in this space, because they're very innovative, and for the most part, they want to help genuinely. So I like that, so we should talk, maybe.

GRAFF: All set? OK. Yeah?

QUESTION: Hi, I'm Joel Beinin with Open Revolution, a company I cofounded, focused on mobile financial services in emerging markets. If I could build off that last point, another way of maybe thinking about it would be a return on investment question. I mean, isn't it kind of an investment strategy question? Because the dollars really are not used. A lot of capabilities out there. It'd be good if we didn't reinvent things that already exist.

To that end, what have you done? Or what are you aware of in terms of metrics that this is done? What can we show in terms of what the application -- social media analysis has done to -- to disaster response? Is that something that's mature? Is there an ROI argument to be made? And if not, why not? And I'll just add a second kind of suggestion. Is it a question of scale and adoption? A lot of the questions about sustainability kind of hinge around whether people are using the product or engaged with the product in a sustainable sort of way, so I'd suggest that as a second possible question as an answer to the first question.

GRAFF: Either of you?

MEIER: So the evidence base for these technologies and social media is way too thin. That's not necessarily because there's nothing there to assess. What is worth keeping in mind is the evaluation -- the independent professional evaluation impact assessment of the Ushahidi Haiti project from 2010 is still the only evaluation of its kind. It's independent. It was well funded. And it was very objective; it was very critical. There was lots of things that didn't work.

I haven't seen any kind of validation at that level of sophistication, because we had top-notch professors from Tulane University who carried out the evaluation. And I think, honestly, it's because of funding. That was a $50,000 evaluation project, and we got the money by chance. A wealthy family in Boston said we loved what you did, can this money be good -- put to good use? And we said, well, we'd like to have somebody independently evaluate what we did to see what went wrong and what we can learn.

So the money issue, valuation to do it really, really well and get the top notch people, because a lot of people call themselves evaluators, but it's a whole other science unto itself. That's a big, big problem. So the evidence base is still very thin.

One of the things we're trying to do with the crisis mappers network in our annual conference is to surface what we've learned every year in the humanitarian technology space and to be as upfront as possible with what worked, what didn't work. And at QCRI, we recently launched the humanitarian computing library, which has now got about 600 or 700 papers, reports, documents on humanitarian computing or the application of social media for emergency management and so on.

We're trying to collect this and putting it into one place so we can get a better sense, but it's a huge -- it's a huge problem. And at the same time, keep in mind that humanitarians have an issue evaluating their own operations without the role of social media or even thinking about social media. That majority of humanitarian interventions are not evaluated independently, either. So it's not something that's specific to social media or new technologies and so on. It's a systemic structural issue, I think.

But that's my two cents.

HUDSON: I mean, I would agree. I think that the -- we need more evidence on the impact of technologies in humanitarian aid. And I think what we've seen is, you know, humanitarians are actually pretty good at evaluating. They're pretty strong on monitoring and evaluation through programs in the main. And the science has developed quite fast, particularly around accountability to communities in humanitarian aid.

But still, when you look at technology programs, the evaluation is at the intervention and not of the impact that technology had on the intervention and its efficiency and effectiveness. So we are planning to do some work with three or four agencies to look at quantitatively with a baseline study the impact of the use of frontline in their work.

But we're going out for a modest amount of funding to do that, because it's -- you know, it's not -- while it's something we'd love to be able to afford to do (inaudible) we don't normally implement ourselves. We support other people who do so. And in this case, we'll be supporting other people to measure impact.

Which leads me on to the sustainability question. One of the things we have learned from the more qualitative studies that have come out is that using Frontline SMS -- using SMS in a professional way can save you a lot of time and money. The ActionAid guys were halving their field trips at a time when Isiolo was becoming insecure and unsafe. And we have heard time and again -- sometimes in quite robust sort of evaluations that -- that staff time was saved, that money was saved. And, you know, a $2,500 saving across the summer in a small clinic in Malawi is not nothing.

So it actually -- they can be cost -- it's like they can be definitely cost-neutral. And our system is pretty -- you can get up and running for less than $1,000 for sure. And we think that the SMS costs, you know, as long as you're not doing blast SMS, which is an expensive proposition, if you're talking more about community interaction and using it to manage staff, it's pretty cheap stuff.

But, I mean, I think, you know, we're a hybrid organization. We're a social enterprise. We're also sort of thinking about how to make ourselves sustainable. And we have run a consulting business to support that, and we also have funding, and we're looking at other ways to support the growth of our platform. So I think that there are -- lots of interlinking issues, but I think -- just to bring it back -- I think that investing in research around the impact of this kind of thing would be -- is useful and needed.

GRAFF: Next up, a question over here.

QUESTION: Fred Tipson at the Institute of Peace. Could you say a little bit more about the dark side? We all assume that if information gets out about a disaster that people will rush to help for all the right reasons, but doesn't this technology present great opportunities for people who have other kinds of agendas, but repressive governments, but also organizations that want to mobilize people in one direction or another or blame this or that group for the disaster or for the unfairness of addressing the disaster?

How do you think about the challenge going forward of dealing with gaming the system from that point of view?

MEIER: That's an easy question, huh?

HUDSON: That's you.

MEIER: How do I even -- this is a dissertation. So, first of all, you know, you can go with technology information is always two-sided, right? The double-edged sword and so on. And we all know that. And you and I have had a number of conversations in similar seminars and so on.

One thing that I don't do any more, as of about two years ago, with a digital volunteer network that I put together with some friends called the standby volunteer task force, is we don't respond to complex humanitarian emergencies, where there is an active sort of conflict, violent sort of actors who are engaging, because of the reasons and the concerns that you've just shared. We don't think that we have the necessary expertise from a data security and information security perspective to prevent sabotage, to prevent sort of some bad apples from getting in and creating a major problem.

But it is an ongoing issue. There are a number of guidelines and codes of conduct that have been developed over the past year now on how to manage this better. I think it's a huge issue. In fact, last week at PopTech, some colleagues and I released -- first published a white paper on big data and ethics and ethical resilience. It was in the context of disaster response as something we touch on, as well.

I would never say that open data, no matter what, is a good thing. In fact, even in the data surveillance -- the biosurveillance sphere, where they've been really lead in terms of using open data for response to disease outbreaks and so on, I'm also concerned about that, as well, and surfacing all that information publicly.

So I think the best thing we can do is, you know, follow the idea of a do-no-harm approach, understand the costs and benefits when people go in, and now, you know, when -- like I said, the ICRC this year for the first time ever included a chapter on crowdsourcing and social media in their data protection protocols as a result of us engaging with them over the past couple of years and saying, frankly, we are not the experts in data security, data protection. You are. And now there's a new world here called social media and crowdsourcing, and we're looking to you, the experts, to let us know what -- how we can hold ourselves accountable, really, because we can't enforce it on others, but we can at least use it to hold ourselves accountable.

So on the conflict side, I've not really been engaged in that any more, just because I think there are too many issues. And, you know, I think, Laura, you touched on this, as well. The issue of informed consent in the humanitarian space is really, really core, right? And here we are, including myself, sort of monitoring tweets, you know, and analyzing them and not necessarily making them public, but did I ask permission to use that tweet? Granting, they're putting in voluntarily, but, you know, there are issues.

Anyway, that white paper that we published with the Rockefeller Foundation and PopTech last week goes into some of these issues. I don't have all the answers, basically. Should have started with that.

GRAFF: So let me ask a follow-up question to that, which is, we've spent most of this time talking about sort of what works, what are the opportunities, what are the, obviously, huge challenges in this space ahead. What's not working? Sort of, what are you seeing that's not worth the effort in this area so far?

MEIER: That's not worth the effort?

GRAFF: Or that's not working.

MEIER: That's not -- everything? I mean, we're doing...

(LAUGHTER)

I mean, we're -- you know, we don't -- we're developing these platforms, like the MicroMappers and so on, and that will become operational, and we'll learn how well and how well it doesn't work, right? So we don't -- we are still suffering from filter failure when it comes to big crisis data. That's -- we haven't solved that yet. We're trying to. So that's still an issue.

The issue of verifying user-generated content in real-time is still very much at the experimental applied research state, which in a way is good, because that's never really happened within the context of the humanitarian response before, but it's still really, really early days. So we'll see whether any of these platforms, regardless of whether they use human computing or artificial intelligence, actually really work and help out. So, you know, I get enthusiastic and optimistic and so on, but it's still very much experimental.

And then the huge issue of data security, privacy, is -- you know, I don't want to be too optimistic and I want to say, OK, we can still win the battle and figure things out, but it's really getting worse and worse and worse in terms of more and more people getting the skills to make use of this information for ulterior motives. And that's a continuing concern. It's a huge, huge concern.

So that's also -- we had -- that doesn't work, right? We haven't really figured out everything on the data privacy, data protection. And I've approached a number of companies to see if they'd be willing to change sort of their terms of service. Fat chance, in terms of being more respectful, especially of tweets, for example, that get posted during disasters, right? And -- but it's challenging. It's technically challenging and then policy challenging, so -- I'm sorry if I'm negative, but I'm really hopeful, because I'm finally working on things that I think need to be worked on and we have to fail and fail forward. And until we experiment, we're not going to contribute anything new. So I'm looking forward to, like, you know, get -- this being messy and trying to figure it out with our humanitarian partners.

GRAFF: OK. Do you have anything to add to that?

HUDSON: I mean, I can -- obviously, everything's going really well (inaudible)

(LAUGHTER)

No, I mean, we've -- you know, we have challenges, too. We're just trying to figure out how to make software that's instantly usable for everybody, no matter what, you know, background you have or where you are or what language you speak. That's easy, obviously.

And then, you know, SMS itself is a -- mobile is a very fragmented market. And, you know, we're trying to find ways around that, too, and that's challenging. But I think specific to humanitarian response, what I would say is what I always say -- and everyone's probably heartily sick of hearing me say -- is we need to invest more in resilience disaster preparedness. We need to invest in local responders.

We spend a lot of time as technologists, you know, talking to the international humanitarian system, the big agencies, the U.N., and that's great. And, you know, we think that they do great work and we're very happy to support -- I mean, I'm very happy to support them. I don't speak for Patrick, but I'm sure you are.

But I would love to see more of the debate being about, you know, how do we get these tools usable for and in the hands of the local responders? And how do we invest more in that? And, you know, I think until we start shifting that debate a bit, we're not going to see matters change at the grassroots, but we could really easily.

MEIER: And just on that point, as well, it's, you know, the haystack and the needle challenge. It's not something that only humanitarian organizations are facing. It's disaster-affected populations themselves, as well.

And when we look at countries and disasters in places like China -- recently, over the past year or two, China, Iran, the Philippines, and others, where you have a young population of maybe -- almost digital natives who already self-organize during disasters. They'll use Facebook. They'll use SMS. They'll use Twitter. They'll use Ushahidi or Google Maps to create their own maps. That's already happening more and more, which I think is phenomenal. It's great.

First responders by definition are the local community and the disaster-affected communities, so I'm really interested in -- and am collaborating with one group, for example, in the Philippines to see how these technologies can really help them, these sort of digital activists, digitally savvy young people who respond during disasters and are learning by doing, just like we are, can make use of these technologies to identify and find those needles in the haystack so they can self-organize quicker, better, you know, and smarter, as well. So it's -- yeah.

But, again, this is not going to be -- the stuff that I'm working on is not going to be easily usable in northern Central African Republic. It's just not. And so we're focusing on one set of users and audience and trying to do the best to equip those with data technology that they haven't had.

But I really am very interested in the last mile. So we've had talks with Question Box folks on how our technologies might translate to their approach. If you look at Question Box, it's phenomenal, in terms of the last mile connectivity. And then the audio, right, interactive voice response. If we can find ways to interface the technology, sort of the low tech with the high tech, I think everybody gains. But it's a huge challenge.

GRAFF: So I think we have time for one more question. And before I take that, I'll just make the standard reminder that this hour has been on-the-record. So maybe all the way in the back here?

QUESTION: Thank you very much. My name is Jeanie Nguyen (ph) with Reuters Vietnamese American (ph). I thank Laura and Pat, and I follow up with the point you just made, because in Vietnam, we have a young population, very much high tech and digitalized, and they've been using SMS and Twitter very effectively. We did have many cases when the local people team up, national, international, to help.

Recently, we have a lady who's delivering a baby and then she lost a lot of blood, and the officials -- the hospital were not able to help, but her husband sent out a message on Facebook, and everybody came out and offered to donate blood, and she was saved. So that was a good thing.

And there was a recent flood in Vietnam in the central Vietnam. And the government did not do anything, but the Facebook teams and people sent out messages, and we all chipped in and sent in money and helped, and they came to the local team, and all these young people came out and helped their own people. So it's all from the bottom up. So it was very effective.

Now, from the dark side, I have to ask your help, because you in connection with the United Nations and many people up at the higher level, you may have heard that many bloggers and Facebook users in Vietnam are being jailed and convicted because of what they sent out on their bloggers and Facebook, just what they tweet out and send out on all of these human rights violations, just because they sent out a message. And the government was able to find them and jail them. And they even convicted them based on what was written on Facebook for very non-threatening, just humanitarians talking about who was beaten by the policeman, who got killed for no reason, all this news.

So would you help with the United Nations to get a little more effective -- what should I say -- legal, ethical conduct from the government's side? Because the United Nations can help to put some pressure on the government actions.

GRAFF: Very complicated issue involving sort of the line between humanitarian response and human rights organizations. Do you see sort of an interplay in the field between those two? Or are your products sort of most useful in the disaster response?

MEIER: No, actually, there is -- thanks for your comment and question. There is a two-day conference at the end of the week in New York at NYU Law School that focuses on sort of new technology, fact-finding in the human rights space, and I've been asked to sort of share our lessons from sort of advanced computing and the humanitarian response, so I think both communities have a lot to learn from each other on the technology front. They're using technologies in similar, but also very different ways, given the different types of mandates and areas of interest, so they have a lot to learn from each other.

And I would -- I mean, just on your point, I would -- I'm not an expert. Honestly, I would refer you to Tactical Tech, the folks at TOR (ph), who -- to (inaudible) who also -- there are the folks at the Guardian Project, who are very strong in terms of how to use communication technology securely in non-permissive environments. That's not my area of expertise. And as for the U.N., that's probably -- Security Council. I don't have -- I don't have a good contact there.

(LAUGHTER)

HUDSON: Yeah, just very briefly, I mean, Frontline SMS is used, you know, and Frontline Crowd now are used all over the world, including, you know, by activists. And we always caution to think carefully about who owns the mobile network that they're using and what that means and also whether there are actors who are sophisticated enough to intercept SMS messages, which is quite possible. It wasn't designed to be a secure channel.

And we have a data integrity guide, which we've put out in 2011, which is available from our website, which helps you to think through that. But the message for SMS is always to be on the cautious side, because it is very easy to intercept. And we've had, you know, users -- on of our very first users driving around with her modem and laptop in the back of a car so that, you know, the system couldn't be triangulated by the mobile network, just -- you know, because you have to be very careful, so it's a very real consideration for activists, absolutely.

GRAFF: Laura, Patrick, thank you for your insights today. The Council on Foreign Relations always prides itself on wrapping up on time, so we will do that and let everyone go about their day. So, thank you.

(APPLAUSE)

MEIER: Thank you very much.