Clarifying a societal problem, its causes and potential impacts
This stage aims to clarify a given problem, identify potential causes, and outline potential impacts or spillover effects of it. It is structured by six different goals that may need to be achieved (A to F). In total, 16 different types of questions that may need to be answered to achieve the goal are included in this stage.
Open the goal that is mostly related to your specific query, and a list of types of questions will be displayed. More details about each type of questions will be provided, including examples and the methodological approaches to address each question.
A. Choosing and prioritizing outcomes of a problem
How can a problem be measured?
This goal aims to get insights on what potential outcomes exist to characterize or measure a problem, the different values that individuals could have regarding the outcomes, and to prioritize what are the most suitable outcomes to characterize or measure the problem.
Open the question that is mostly related to your specific query, and more details about the question, examples and methodological approaches will be displayed.
Question A1. Identifying outcomes to prioritize a problem
This type of question looks for different outcomes that are available and suitable to characterize or measure a given problem.
Some examples of this type of question are:
- What outcomes are available to measure clinical improvement in patients with COVID-19?
- What outcomes can be used to measure health inequalities?
- What outcomes or measurements are available to describe poverty in a country"
Methodological approaches to address question A1
- Review of outcomes that have been used by other studies (e.g., scoping review)
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of people’s experience
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of experts' opinion
Question A2. Understanding stakeholders' values regarding outcomes
People could have different values on the uses of different outcomes to characterize or measure a problem. Also, different stakeholders could have different values regarding outcomes (e.g., citizens may value different the outcomes than government policymakers). This type of question aims to describe the different values that different stakeholders could have regarding existing outcomes to measure a problem.
Some examples of this type of question are:
- How important is the outcome progression-free survival for patients with breast cancer?
- How do family members of patients with Alzheimer's disease value the quality of life?
- What are parents’ views on learning outcomes when choosing a school for their child's education?
Methodological approaches to address question A2
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
Question A3. Prioritizing outcomes to characterize a problem
This type of question aims to prioritize outcomes to characterize or measure a problem by looking at the identified outcomes and weighing the different values and preferences of individuals regarding them.
Some examples of this type of question are:
- What are the most suitable outcomes or scales to measure depression symptoms?
- What are the best ways to measure emergency room waiting times?
- What evaluation scale would be more reliable and accurate to measure students’ science learning?
Methodological approaches to address question A3
- Delphi study (to get consensus from experts)
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of data collected for this purpose (i.e., primary data)
- Jurisdictional scan (comparative analysis) to understand what other jurisdictions are using
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of data collected for other purposes (i.e., secondary data)
B. Describing a problem and its magnitude
What’s the problem and how big it is?
This goal aims to describe a problem in a given moment of time and to identify the population that is affected by it.
Question B1. Describing a problem at a point in time
Prevalence
This type of question aims to describe a problem or issue at a given point in time. Depending on the type of variable that would need to be measured accumulated frequencies, central tendencies, and/or distributions can be measured.
Some examples of this type of question are:
- What are the most common symptoms of people with active COVID-19 infection?
- What is the mean age of patients living with diabetes in a given country?
- What is the 90th percentile of BMI for a given population?
Methodological approaches to address question B1
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
Question B2. Describing a problem during a period of time
Incidence
This type of question aims to describe a problem or issue during a period of time, through frequencies, central tendencies, and/or distributions.
Some examples of this type of question are:
- What is the number of new patients that entered a waiting list for elective surgery during 2022?
- What is the median age of people that entered a new social program during 2022?
- What is the number of people that have been diagnosed with HIV in the last year?
Study designs to address question B2
- Prospective cohort study of individual-level data (prospective longitudinal or panel study)
- Retrospective cohort study of individual-level data (retrospective or historical longitudinal, or panel study)
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of data collected for this purpose (i.e., primary data)
- Delphi study (to get consensus from experts)
C. Understanding a problem
How and why is a problem?
This goal aims to interpret a given problem by critically and conceptually analyzing it and understanding how it is perceived by different stakeholders and the role that the context has in the specific problem.
Open the question that is mostly related to your specific query, and more details about the question, examples and methodological approaches will be displayed.
Question C1. Finding conceptual approaches to understand a problem
Depending on the complexity of a given problem, conceptual frameworks might be useful to understand a problem, which might lead to better ways on how to address it. This type of question aims to find and select frameworks or taxonomies to describe and conceptualize a complex problem or issue.
Some examples of this type of question are:
- How can we classify the reasons behind vaccine hesitancy?
- How can we describe decision-making processes and their interaction with different contextual variables?
- What factors are contributing to inaction in global warming and climate change and how do they interact with each other?
Methodological approaches to address question C1
- Review to identify existing frameworks (conceptual analysis)
- Qualitative inductive (from particular to general i.e., creating theory) methods to interpret/critically analyze a phenomenon (e.g., ethnographic approaches, phenomenology)
- Review to build a new framework (critical interpretive synthesis)
- Qualitative inductive (from particular to general i.e., creating theory) methods to describe a phenomenon (e.g., grounded theory)
Question C2. Understanding stakeholders' perceptions of a problem
Exploring the perceptions that different groups have of a problem is critical to understand the problem and its magnitude. There are many types of stakeholders that might have different perceptions of the same issue, including government policymakers, organizational leaders, professionals and citizens.
This type of question aims to explore the perceptions that different groups of stakeholders have of a problem and its magnitude. Complementary, the interpretations of the results of a study answering this type of question can be used to understand the different framing of a given problem by certain interest groups.
- What do cancer patients think about the disruption of cancer due to COVID-19?
- How do teachers perceive the educational system in a given country?
- What are citizens’ views regarding climate change?
Methodological approaches to address question C2
- Qualitative inductive (from particular to general i.e., creating theory) methods to interpret/critically analyze a phenomenon (e.g., ethnographic approaches, phenomenology)
- Cross-sectional study (survey, point-in-time or snapshot study or analysis) of people's experiences (not asking about hypothetical scenarios)
- Qualitative inductive (from particular to general i.e., creating theory) methods to describe a phenomenon (e.g., grounded theory)
- Qualitative deductive (from general to particular i.e., testing theory) methods to describe/critically analyze a phenomenon (e.g., qualitative case studies)
Question C3. Understanding the role of context in a problem
Contexts and/or settings can bring unique challenges/opportunities for a certain problem or objective to be developed or achieved. This type of question aims to understand the role that the context or the specific setting might have in the problem and its magnitude.
Some examples of this type of question are:
- What is the role that the context has in the lack of affordable care for patients with cancer in a given country?
- What contextual factors are most important to reduce the higher inflation rates?
- How does the context of a student interact with his/her learning process?
Methodological approaches to address question C3
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
D. Assessing the variability of a problem
How the problem varies over time, across populations and in relation to other problems?
The goal of these questions is to assess how the problem varies over time, across populations and in relation to other problems. While only these three variables are addressed in this goal, the variability can also be assessed by comparing multiple variables (e.g., against time and other populations simultaneously).
Open the question that is mostly related to your specific query, and more details about the question, examples and methodological approaches will be displayed.
Question D1. Assessing variability over time
This type of question aims to measure the evolution of an indicator or measurement over time. Assessing the variability of a problem or issue when compared against time could be done over limited (e.g., 2 points) or multiple (e.g., time series) point of time.
Some examples of this type of question are:
- Has maternal mortality been reduced in the last 10 years?
- How the low ratio of nurses per 1000 habitants has evolved in the last 10 years?
- How has the average earth temperature varied in the last 3 decades?
Methodological approaches to address question D1
- Longitudinal study of individual-level data (retrospective or historical longitudinal, or panel study)
- Descriptive (not predicting) time-series analysis (including trend analysis)
- Modelling to predict future scenarios (e.g., system dynamics, ARIMA models, etc.)
- Single before-and-after study of aggregated data (pre-post or pretest-posttest study)
Question D2. Assessing variability across populations and locations
This type of question aims to identify what populations or sub-populations are affected (or most affected) by a certain problem, which could entail assessing equity considerations.
Some examples of this type of question are:
- What is the average waiting time in the emergency room vary depending on patients’ age?
- What is the income difference between citizens with high and low socio-economic status?
- How have wildfires affected different geographic communities?
Methodological approaches to address question D2
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
Question D3. Assessing the importance of a problem relative to other problems
Burden of disease
This type of question aims to assess the variability or the magnitude of a problem when compared against other problems, by measuring the relative importance (i.e., weight) of an issue compared to others.
Some examples of this type of question are:
- What is the burden that hypertension creates compared to diabetes?
- For a given city, how important homelessness is compared to a poor educational system?"
Methodological approaches to address question D3
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
E. Understanding the causes and aggravating factors of a problem
What is causing or making the problem worse?
This goal aims to identify and understand the relative importance of causes and/or aggravating factors of the problem.
In this goal, the problem is set as the dependant variable (being caused by something else) as opposed to goal F, where the problem is set as the independent variable (the problem being the cause of something else). However, in many cases, the framing of the question could be unclear (i.e., whether an issue is the problem or the cause of a different problem).
Open the question that is mostly related to your specific query, and more details about the question, examples and methodological approaches will be displayed.
Question E1. Identifying associations between potential causes and a problem
Risk / protective / explanatory / prognostic factors
This type of question aims to find to what extent a certain variable can be associated with a certain problem and can be identified as a potential cause or aggravating factor Identifying potential causes or aggravating factors of a problem require assessing whether a factor is associated with a certain outcome.
- Is having hypertension associated with a higher risk of myocardial infarction?
- Is being >60 years old associated with an increase in the risk of severe COVID-19?
- Is unemployment associated with worse health outcomes?
Methodological approaches to address question E1
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
Question E2. Determining causes and/or aggravating factors of a problem
Risk / protective / explanatory / prognostic factors
This type of question aims to find to what extent a certain variable is a cause or aggravating factor of a certain problem. Different type of causes or aggravating factors might have variable effects on certain outcomes, and the level of exposure to a certain factor might also vary the effects on these outcomes.
Some examples of this type of question are:
- Does having hypertension increase the risk of myocardial infarction?
- Does being >60 years old increases the risk of severe COVID-19?
- Does unemployment produce worse health outcomes?
Methodological approaches to address question E2
Not available yet. We will upload a consensus methodological approach very soon. . Learn more about our approach to reach consensus here.
Question E3. Understanding the relative importance of causes and/or aggravating factors across population groups
Risk / protective / explanatory / prognostic factors
After having identified causes or aggravating factors of a problem, this type of question aims to assess the relative importance that causes and/or aggravating factors have based on their contribution to the problem, and to identify in what population groups the association between these causes and/or aggravating factors and the outcome is stronger.
Some examples of this type of question are:
- What are the most important risk factors that predict the incidence of myocardial infarction?
- What are the most important factors that explain a high infant mortality rate?
- What are the most important factors that explain the lack of access to prescription drugs?
Methodological approaches to address question E3
No consensus has yet been reached on what methodological approaches are most suitable to address this question. Learn more about our approach to reach consensus.
F. Understanding the impacts of a problem
What impacts is the problem creating?
This goal aims to identify and prioritize the most important impacts or spillover effects of a given problem.
In this goal the problem is set as an independent variable (the problem being the cause of something else), as opposed to goal E, where the problem is set as the dependent variable (the problem being caused by something else). A problem can create several types of impacts or spillover effects, that could be felt in one or multiple sectors (e.g., one health problem could create spillover effects on the environment).
Open the question that is mostly related to your specific query, and more details about the question, examples and methodological approaches will be displayed.
Question F1. Identifying impacts/spillover effects of a problem
This type of question aims to find whether and to what extent a given scenario is a consequence of a certain problem.
Some examples of this type of question are:
- What are the impacts of homelessness on the health system of a given country?
- What is the natural prognosis of a given disease?
- Can unemployment produce a higher cancer mortality rate?
Methodological approaches to address question F1
- Controlled before-and-after study of aggregated data (including difference-in-differences study and non-equivalent control group designs)
- Review to find causes or aggravating factors that have been identified by other studies (e.g., scoping review)
- Prospective cohort study of individual-level data (prospective longitudinal or panel study)
- Retrospective cohort study of individual-level data (retrospective or historical longitudinal, or panel study)
Question F2. Prioritizing the most important impacts/spillover effects of a problem
This type of question aims to determine what are the most important impacts or spillover effects of a problem.
Some examples of this type of question are:
- What are the most important impacts of homelessness?
- What are the most important impacts of unemployment?
- What are the most important health impacts of undernutrition?"
Methodological approaches to address question F2
- Review to find causes or aggravating factors that have been identified by other studies (e.g., scoping review)
- Delphi studies (to get consensus from experts)
- Prospective cohort study of individual-level data (prospective longitudinal or panel study)
- Retrospective cohort study of individual-level data (retrospective or historical longitudinal, or panel study)
Problems can be issues that are sitting in present or the past, or they can also be issues that are not necessarily a problem now, but they could eventually become one (future problems or existential risk). We did not create specific questions for these scenarios, but we acknowledge that the same types of questions that are included in this stage can be formulated for present or future problems.
Although problems create a decision-making situation that frames an issue in a negative way, they can also be framed in a positive way as objectives. Hence, the goals included in this section can also be framed in a positive way by replacing problems by objectives as A. Describing an objective and its size; B. Understanding an objective; C. Understanding steps and facilitators to achieve an objective; D. Choosing and prioritizing outcomes of a goal ; E. Assessing the variability of an objective; and F. Understanding the impacts of achieving the objective.
Take a look at the demand-driven approach used to create the Matching Q-M tool, as well as a list and explanation of all methodological approaches included.