Evaluating the impact of any given program is always challenge, even in the best case scenario. It becomes even more complicated when it is in the context of resource-limited settings that is compounded by unexpected political events.
Another dimension that has to be factored in the case of the Foko project is the fact that I am clearly involved with the project and therefore intrinsically coming with a biased perspective that would be avoided if an independent third party were to perform the evaluation.
Still I believe there is an important value in assessing the impact of a project from within and posting publicly. First, it forces a critical examination of the proceedings at a certain point and provides a perspective from the day-to-day actions. Second, one may receive valuable feedback on the assessment of the project and recommendations on how to improve.
In addition, most of the important achievements of the Foko project can be traced directly to their individual members first and second to Joan and Stephane who have carried the lion share of the workload. The circumstantial inaction on my part allowed me to view the process with a more distant perspective that may help for the sake of the evaluation.
To get started, below is the logic model that should help monitoring the progress of the project along the way:
Methods for evaluation:
Random Assignement or Randomized Control Trials (RCTs) is the favored method in social/development projects but considering the penetration of digital media users, the limited size of sample (numbers of participants) and the limited resource, this method was just not feasible at this stage of the project.
Since an equivalent control group was not readily available either for a quasi-experimental setting at the time, the absolute number of new blogs created through the project is a good indicators of the health of the project but it may not be difficult to sort out the number of new blogs that would have been created if the project did not exist. One could "guesstimate" but it would leave a fairly large margin of error.
Evidently, the context of the Foko project changed drastically because of the ongoing political turmoil. What started has a project to illustrate the day-to-day lives of Malagasy people and a platform to exchange ideas, became an critical outlet to receive news pertaining to the crisis. It goes without saying that is affected profoundly the outlook and the outcome of the project. Trying to factor the negatives and the not-so negatives of the crisis is not straightforward. I think it certainly validated the need for increased digital awareness for the majority of the Malagasy population. In fact, even though I do not have the exact numbers, it is pretty clear that the Malagasy crisis provoke a surge in the creation of political blogs. There was also the opposite effect, the ones who stopped blogging in view of more urgent matters to attend to or because it became dangerous to speak out. Whether the unavoidable increased political content was a good or bad thing for blogging remains to be seen and opinions will certainly vary. Now that the media in the rest of the world has essentially forgotten about the persisting crisis in Madagascar, it will also be intriguing to see how the dynamic of the blogoshpere will evolve in a more introspective context.
It would take too long to lay out all my thoughts on what metric/parameters one should use to evaluate digital outreach (participation, activity, social change etc...)
But here is one metric that is undeniable and easy to illustrate: story telling. This would rule out any quantitative analysis but at this stage of the game, quantity may not matter as much anyway.
At the height of the crisis, bloggers like everyone else, were concerned with transmitting what they saw, heard or felt as quickly as possible. A photo was worth a thousand words, literally. Then the dust settled. The frenzy made ways to a less urgent but still probably more sobering, grinding crisis, one that probably will have more lasting effects than the turmoil at the height of the violence. Still bloggers in general and from Foko in particular have remained. A few quit but new ones also arrived.
After an unwanted crash-course in crisis reporting, bloggers are now facing the challenge of describing more quiescent but still oh so relevant stories.
Boy are they getting good at it.
As a case in point, one of the unheralded story from the recent months from Foko bloggers was told by Tahina in a post titled "shit happens". Tahina told the story of one man who lost his job as a result of the crisis. There was no other dramatic change in his life , just a slow, gradual descent into more difficulties over a year. If you read the post, you would have a much better understanding of how crisis and poverty intersected and affected Malagasies in a more tangible manner than any economic indicators could ever explain.
Jaona from Fianaratsoa went back to reach out and tell the stories he meant to put at the forefront in the first place: the stories of disenfranchised youngsters, sex workers and the gay and lesbian communities. you can find the links to those blogs in this post.
One of the winners of the Malagasy blog contest, Moonlightgirl is an aspiring writer who teases out the drama of the everyday life events in her town.
Finally in a post titled " living from trashes", Arinaina captures with a photo and a few words the plight of homeless people in her neighborhood. She tells the tale of a basket she threw away only to find it being used a few minutes later by the impoverished.
Those are the stories of Malagasy people as clearly re-told by the bloggers who live near or witness them.
Surely there are more parameters than the power of storytelling to factor in in order to monitor digital outreach program effectively. However, I would argue that it is a crucial aspect of its success. At the end of the day, the human connection is what give sense to such programs and story telling is the best way to make that bridge palatable.
I will provide a few statistics and graphs in a follow-up post for those who are eager to see quantitative results.