Evaluating the Impact of Grassroots Citizen Media in Developing Countries

Below are some thoughts on what I will be discussing at the Global Voices Summit. Cross-posted there as well

Grassroots citizen media has been identified in a few instances as an instrument of facilitating social change, but there are also a multitude of examples of citizen media failing to meet promises. So much has been said about the potential of information technology as a catalyst for societal progress that many have conversely complained about "cyber-utopia", information overload and even cyber-threats.

What might need to be revisited or studied in more detail is how, where and whom does citizen media really impact? In the following post, I will present thoughts from my personal experience with a project in Madagascar and a few suggestions for how one could go about evaluating citizen media projects generally:

Defining the goals of a participatory media project:

The Foko-Madagascar group blogging project was set up to highlight and share on the world wide web, what associations in Madagascar were doing to affect positive social and environmental change in many Malagasy disenfranchised communities. Unfortunately, a political crisis changed the context of our work and many of the trained bloggers turned their focus to documenting the violence and political evolution of the country. Especially because mainstream media was somewhat restrained by the political context.

My  point is that participatory media projects should have a clearly defined mission, even if it is tweaked later. Here are a few examples of objectives that citizen media projects have advocated for:

- Election monitoring (Sudan)
-Health (Kenya)
-Democracy (Zimbabwe)
-Womens Empowerment (Yemen)
-Cultural Identity (Bolivia)
-Monitoring governance (India) and transparency (Brazil).
- Environment (Mongolia)

As it was the case for Foko, the objective might temporarily change (because, well, political coup's and natural disasters happen) but the objective will ultimately drive the evaluation of the project by the both project leaders and funders. The objective will also help determine who the project will reach out to for contributions, and which type of audience it seeks to attract.

Target audience:

The impact of all grassroots citizen media is highly dependent on how large of an audience it can reach. This is especially salient when it comes to projects that advocate for democracy, transparency or human rights. Having a voice in the court of international opinion is still the simplest way to oppose or denounce an authoritarian regime that strives to control the information originating from a country. Accordingly, it is usually beneficial to make content available in different languages and reaching out to international media outlets. Citizen media are by definition participatory so it is also critical to grow an audience and engage with 1) local citizens so that they can validate the published content as well as 2) experts in the projects’ fields of interest so  they can provide insights and critiques.

Context and baseline:

The context in which a project takes place is critical for evaluating true impact. Whether the contributions stem from the heart of Silicon Valley or from the depth of a tropical forest will be a determining factor in whether a project will generate a multitude of updates and comments. Therefore it is important to establish a baseline with respect to internet access, usage and data flow rate (including possible censorship and cultural acceptance of individual expression).

Monitoring and outcome measurements:

In a previous post on my own blog, I proposed a few ideas for evaluating a blogging outreach project. Those suggestions were mostly intuitive, but a more rigorous approach to evaluation might be required for academic or funding purposes. Random assignment is one option that can provide a clear contrast between citizen media users and control groups. This study design will help determine the real impact of citizen media in a given region because it will follow trained groups and control groups that are assigned by chance, hence eliminating any confounding factors. The randomization process would have to be applied to communities rather than individuals because media training cannot be selectively provided within a community. An alternative study would be to conduct a pre-post study with the start of the project as a measuring time point. This would require collecting data on the use of blogging and ICTs in a region before the project and contrasting it with data collected as the project goes along. This method is not as rigorous because other factors might affect the data but it's Measurable outcomes would  be divided into: 1) quantitative or qualitative data; 2) direct impact on internet traffic or indirect societal impact; 3) Internal and external effects.

Quantitative data:
1) Contributors’ satisfaction and internal dynamism: blogging rate, number of internal commenting/trackbacks, back links from other members, sustained blogging activity.
2) External audience/exposure: Number of outside clicks and links.
3) Societal impact: Various depending on objective: legislative change because of advocacy, quantifiable behavioral change for environmental or health advocacy (slash and burn reduction, less smoking, less censorship etc..), number of fraud reports in elections, number of human rights violations reports etc.

Qualitative data:

1) interviews with bloggers about their experience,
2) reviews of the project by other media outlets
3) quality/relevance of content on the platform.
4) authenticity of the articles.
5) civic engagement

How often should monitoring and evaluation be conducted:

One of the appeals of participatory media for citizens is that it is solely driven by their own motivation. Therefore, I believe it would be nonconstructive to impose strict deadlines on publication of content. However, project leaders can encourage regular content production by reacting often to new postings.

Final thoughts for discussion:

Although the political crisis is still ongoing in Madagascar (Reuters), the Foko project have mostly returned to its initial purpose. Children tree-planting activities led by renown Malagasy artist Razia Said have recently been documented in Alasoara, Reinsertion programs for HIV + people and disenfranchised adolescents in Fianaratsoa have resumed and ICT for development workshops have taken place in the capital city.
However, the seeds of civic engagement and participation have been planted and the Malagasy citizen media is getting ready to help oversee the scheduled national elections in May with the local Ushahidi platform.


2) An interesting conference called Failfare will discuss lessons learned from failed experiences in the field of mobile information technology. An intriguing experiment in the evaluation of citizen media initiatives.

3) In his book “Making Democracy Work: Civic Traditions in Modern Italy”, Putnam argues that the success of democracies depends in large part on the horizontal bonds that make up social capital. Grassroots citizen media is the quintessential facilitator of bonds and social capital. However, citizen media can also be used to promote social distrust if used with ill intentions.