A great deal of debate is taking place among development professionals related to project evaluation, including in conflict/crisis/fragile environments, where timing rarely allows for traditional formulation of activities. Besides, related parameters are difficult to track and monitor, and the question remains about what criteria and indicators should be used. Many informed professional agree that the risk is to fail to appreciate a number of projects and initiatives because their value for money is hard to demonstrate using existing tools and methodologies.
A meta analysis of many evaluation reports about past projects has been prepared which formulates a number of key recommendations. One of the emerging conclusions is that IfS/IcSP projects take place in difficult fast-paced contexts, where change is important to account for using professional judgment, which calls for better contextualization of projects and activities, and for encouraging programmatic or portfolio approaches to evaluation.
Among other challenges, the report stresses that innovative ways to capture evidence need to be developed and tested, as well as news formats to facilitate knowledge management and wider dissemination. The argument is that audiovisual tools could be best at capturing elements of context and judgment, which are at the core of the Theory of Change.
Filmed evaluation approach
Based on preliminary research, 3 key “meta” recommendations appeared most relevant to the Sudanese case, which Collateral Creations sought to address during filmed interviews with various interlocutors:
- “Contextualisation” argument: promote the value of mainstreaming of conflict analyses in project management in relation to the Theory of Change (ToC) approach.
- “Added value & coordination” argument: promote portfolio programming and evaluation from the EU perspective, linking policy and development matters beyond individual projects.
- “Judgment elaboration” argument: promote expert thinking to help shape a professional judgment about evaluation criteria.
International operations in Sudan face a number of constraints, principally related to limitations of movements as travelling inside the country to visit projects’ sites requires the preliminary agreement of the authorities, which is granted only with difficulties. In the end, the film crew could not gain access outside Khartoum and shooting was therefore restricted to the capital and to workplaces mostly, offering limited visual opportunities.
The editing phase was entirely based on the original footage of interviews shot by Collateral Creations’ team. The film was built to refer to gallery of portraits from which relevant collected comments were picked to build the narrative.
)
)