Nous avons mené une recherche évaluative réaliste dans six pays africains pour comprendre un programme complexe et protéiforme de l'OMS qui appuie techniquement et financièrement les ministères de la santé des pays à revenu faible et intermédiaire. Ce soutien vise à favoriser l'institutionnalisation du dialogue sur les politiques (‘policy dialogue’), un dispositif de gouvernance collaborative dont l’objectif est de faire progresser la couverture maladie universelle.

 

Ce cas illustre l'approche réaliste et collaborative mobilisée dans cette recherche multi-pays. Il met en évidence les défis méthodologiques et pédagogiques auxquels nous avons été confrontés et pour lesquels nous proposons des recommandations. Nous présentons notre approche de collaboration avec nos collègues chercheurs et nos collaborateurs de l’OMS, la phase de co-construction de la théorie de l’intervention, et les trois étapes successives de l’analyse des données.

 

Ce cas est destiné aux lecteurs qui ont une compréhension de base de l'approche réaliste, de l'étude de cas et de l'évaluation. Les lecteurs qui ne sont pas familiers avec ces méthodes sont encouragés à se reporter d’abord aux lectures recommandées.

​

Lire

​

Article publié dans SAGE Research Methods (anglais)

In 2011, WHO, the European Union and Luxembourg entered into a collaborative agreement to support policy dialogue for health planning and financing; these were acknowledged as core areas in need of targeted support in countries’ quest towards universal health coverage (UHC). Entitled ‘Universal Health Coverage Partnership’, this intervention is intended to strengthen countries’ capacity to develop, negotiate, implement, monitor and evaluate robust and integrated national health policies oriented towards UHC. It is a complex intervention involving a multitude of actors working on a significant number of remarkably diverse activities in different countries.

 

The researchers will conduct a realist evaluation to answer the following question: How, in what contexts, and triggering what mechanisms, does the Partnership support policy dialogue for health planning and financing towards UHC? A qualitative multiple case study will be undertaken in Togo, Liberia, Democratic Republic of Congo, Cape Verde, Burkina Faso and Niger. Three steps will be implemented: (1) formulating context–mechanism–outcome explanatory propositions to guide data collection, based on expert knowledge and theoretical literature; (2) collecting empirical data through semistructured interviews with key informants and observations of key events, and analysing data; (3) specifying the intervention theory.

​

Article published in BMJ Open (English)

Collaborative mental healthcare (CMHC) has garnered worldwide interest as an effective, team-based approach to managing common mental disorders in primary care. However, questions remain about how CMHC works and why it works in some circumstances but not others. In this study, we will review the evidence on one understudied but potentially critical component of CMHC, namely the engagement of patients and families in care. Our aims are to describe the strategies used to engage people with depression or anxiety disorders and their families in CMHC and understand how these strategies work, for whom and in what circumstances.

 

We are conducting a review with systematic and realist review components. Review part 1 seeks to identify and describe the patient and family engagement strategies featured in CMHC interventions based on systematic searches and descriptive analysis of these interventions. We will use a 2012 Cochrane review of CMHC as a starting point and perform new searches in multiple databases and trial registers to retrieve more recent CMHC intervention studies. In review part 2, we will build and refine programme theories for each of these engagement strategies. Initial theory building will proceed iteratively through content expert consultations, electronic searches for theoretical literature and review team brainstorming sessions. Cluster searches will then retrieve additional data on contexts, mechanisms and outcomes associated with engagement strategies, and pairs of review authors will analyse and synthesise the evidence and adjust initial programme theories.

​

Article published in BMJ Open (English)

Public health interventions are complex by nature, and their evaluation requires unpacking their intervention logic and their interactions with open social systems. By focusing on the interrelationships between context, mechanism, and outcome, Pawson and Tilley’s realist approach appears a promising innovation for public health-related evaluation works. However, and as expected of any methodological innovation, this approach is being constructed gradually by answering the multiple challenges to its operationalization that fall in its path. One of these challenges, users of this approach agree on, is the necessity of clarifying its key concept of mechanism.

​

We first collected the definitions of mechanism from published works of Pawson and colleagues. Secondly, a scoping review was conducted to identify the ones quoted by users of the realist approach for evaluating public health interventions (1997–2012). We then appraised the clarity and precision of this concept against the three dimensions defined by Daigneault and Jacobs “term, sense and referent.”

​

Of the 2344 documents identified in the scoping review, 49 documents were included. Term: Users of the realist approach use adjectives qualifying the term mechanism that were not specifically endorsed by Pawson and colleagues. Sense: None of the attributes stated by Pawson and colleagues has been listed in all of the documents analyzed, and some contributions clarified its attributes. Referent: The concept of mechanism within a realist approach can be ascribed to theory-based evaluation, complex social interventions, and critical realism.

​

Article published in Implementation Science

Public health interventions are by nature complex and are implemented in contexts and with social actors who have a mutual influence on each other. It thus becomes nearly impossible, and more or less irrelevant, to control the interventions or to find direct causal relationships. It follows that most public health interventions are natural experiments for which appropriate methodological fit should be the gold standard.

 

Therefore, researchers should adopt a pragmatic, non dogmatic position; be responsive; and adapt to the natural environment of the real world (as there is no such thing as a laboratory setting for public health interventions). Because of their complexity, we need to go beyond just knowing the interventions’ effects; we also want to know how those effects are produced, for whom, and under what circumstances. Methods should be selected that allow us to understand the inherent complexity of public health interventions. (...)

 

Thus the evaluation questions and concerns should guide the methods, and not the reverse. To the extent possible, we will present all these consubstantial aspects of a pragmatic and real-world evaluation approach from a conceptual perspective and then illustrate their application. 

​

Article published in Oxford Bibliographies (English)

Les méthodes traditionnelles de recensions systématiques fonctionnent moins bien pour synthétiser les données des évaluations des programmes complexes. Une nouvelle méthode de recension des écrits, la recension réaliste, semble prometteuse pour synthétiser les données issues des évaluations de programmes complexes.

 

Cet article présente la recension réaliste et sa pertinence pour synthétiser les évaluations de programmes complexes. En guise d’exemple, les recensions effectuées sur les programmes de suivi dans la communauté en santé mentale sont analysées. Une discussion suit sur la façon dont certaines lacunes identifiées pourraient être comblées par la recension réaliste. 

​

Article publié dans la Revue canadienne d'évaluation de programme

L’approche réaliste gagne en popularité auprès des chercheurs en sciences sociales, notamment auprès de ceux dont les objets de recherche sont complexes : programmes, projets ou politiques publiques qui visent à améliorer une situation sociale. En effet, l’approche réaliste a été récemment adaptée à la pratique de l’évaluation de programmes et à celle de la revue systématique.

Ces pratiques de recherche ancrées dans le réalisme critique semblent mieux adaptées aux exigences du mouvement de la prise de décision ou de l’action fondée sur les preuves scientifiques. Pourtant, l’approche réaliste est encore peu utilisée et connue, particulièrement parmi les étudiants et les chercheurs francophones en évaluation.

 

Le premier objectif de cet article est donc de présenter, de manière didactique, les fondements épistémologiques et les concepts clés de l’approche réaliste pour l’évaluation de programmes et la revue systématique. Le deuxième objectif est d’illustrer, par une étude de cas – celui d’une revue réaliste –, les apprentissages conceptuels et méthodologiques pour la pratique de l’évaluation réaliste. Cette démarche réflexive s’inscrit dans un mouvement de partage de connaissances et de pratiques sur l’approche réaliste.

​

Article publié dans Mesure et évaluation en éducation

L’approche réaliste propose une manière d’appréhender la complexité des interventions sociales en cherchant à comprendre ce qui fonctionne, pour qui, et dans quels contextes.

 

A partir de l'étude des écrits scientifiques et d’une analyse réflexive de nos expériences, nous présentons quelques défis et réflexions liés à l’emploi de cette innovation évaluative. Si l’approche Realist apparaît intellectuellement séduisante, la construction des configurations « Contexte-Mécanisme-effet » qu’elle suggère s’avère délicate à effectuer.

 

Les difficultés de trouver les informations sur les contextes ou les effets, de distinguer les éléments du contexte de ceux du mécanisme, et d’adopter le niveau d’analyse adéquat constituent autant de défis dans l’utilisation de l’approche Realist.

​

Article publié dans la Revue canadienne d'évaluation de programme

Four years prior to the Millenium Development Goals (MDGs) deadline, low- and middle-income countries and international stakeholders are looking for evidence-based policies to improve access to healthcare for the most vulnerable populations. User fee exemption policies are one of the potential solutions. However, the evidence is disparate, and systematic reviews have failed to provide valuable lessons. The authors propose to produce an innovative synthesis of the available evidence on user fee exemption policies in Africa to feed the policy-making process.

 

The authors will carry out a realist review to answer the following research question: what are the outcomes of user fee exemption policies implemented in Africa? why do they produce such outcomes? and what contextual elements come into play? This type of review aims to understand how contextual elements influence the production of outcomes through the activation of specific mechanisms, in the form of context–mechanism–outcome configurations. The review will be conducted in five steps: (1) identifying with key stakeholders the mechanisms underlying user fee exemption policies to develop the analytical framework, (2) searching for and selecting primary data, (3) assessing the quality of evidence using the Mixed-Method Appraisal Tool, (4) extracting the data using the analytical framework and (5) synthesising the data in the form of context–mechanism–outcomes configurations. The output will be a middle-range theory specifying how user fee exemption policies work, for what populations and under what circumstances.

 

Article published in BMJ Open