There was a popular television show in the US a few years back called Myth Busters where the show’s hosts used elements of the scientific method to test the validity of rumors, myths, and news stories. Unfortunately, to my knowledge the show never attempted to bust some common myths about monitoring and evaluation — I can’t imagine why not, it’s such a thrilling topic!
Therefore, in this post I’m going to bust 3 common myths about M&E in part 1 of Myth Busters M&E Edition. I came up with too many myths that a part 2 will be posted sometime next week.
Without further ado, here are 3 common myths about M&E:
Myth 1: M&E = Data
There’s a common belief that monitoring and evaluation is the same as data. As I referred to in a previous post, M&E is core to program strategy. It is so much more than data.
M&E helps improve progress towards and achievement of results by extracting, from past and ongoing activities, relevant information that can subsequently be used as the basis for programmatic fine-tuning, reorientation, and planning. Without M&E, it would be impossible to judge if work was going in the right direction, whether progress and success could be claimed, and how future efforts might be improved.
Data is a tool used in the M&E process described above, but it’s not the entirety of it. Data are the specific quantitative and qualitative information or facts that are collected and analyzed that help us understand our impact.
The problem with equating M&E and data as the same thing is that often people jump to thinking about the tools needed to collect information before systematically developing a plan that outlines program goals, intended outcomes, and the key strategic questions to guide our efforts. Data will not tell you much if you don’t know what need to know.
Although I just debunked the myth that M&E is the same as data, the next two myths focus exclusively on data itself because I have found that data collection is often fraught with misconceptions about what is considered important and valid.
Myth 2: More data is better
Nutritionist’s often recommend a diet of 'everything in moderation’. The same principle applies to data collection. More data is not necessarily better.
The comic above depicts a common mentality — collect it just in case we need it. This approach is actually detrimental to our work. An excess of data often leads to:
- confusion about what data to look at and how we’re using that information
- wasted time and resources collecting information that is never or rarely used
We should not be collecting data that we do not use. Our data collection should be guided by information needs that really matter right now, rather than data collection for its own sake.
There is a movement towards data minimalism that is focused on purposeful data collection to facilitate decision-making. Rather than spending our time collecting hundreds of pieces of data, our time is better spent at the front-end to:
- Reflect on what we really need to know — this requires that we put on our investigative hats and ask ourselves tough questions. Do we really need to know this? How will we use it? What resources are required to collect it?
- Use the CART Principles — I love these principles developed as part of the Goldilocks: Right-Fit M&E initiative. Make every effort to ensure your data collection is:
- Credible – Collect high quality data and analyze them accurately.
- Actionable – Commit to act on the data you collect.
- Responsible – Ensure the benefits of data collection outweigh the costs.
- Transportable – Collect data that generate knowledge for other programs.
Myth 3: Quantitative data is more valid than Qualitative data
There is a prevailing belief that numbers are more important, valuable, and useful than narrative information. Quantitative and Qualitative data are different, but one is not more important or more credible than the other.
First, let’s look at how they are different:
- Quantitative data is often represented as numbers and help to answer questions such as, “How much…?”, “How many…?”, and “How frequent…?”. Quantitative data collection is best used for understanding what is happening in a program.
- Qualitative data are usually in the form of text or narrative and helps to answer questions such as, “Why are some participants more active in the program?” and “How have participant’s understanding of the law changed since the program started?”. Qualitative data collection methods are more appropriate for understanding people’s attitudes, behaviors, beliefs, opinions, experiences, and priorities.
Rather than thinking you need to choose between either quantitative or qualitative data, it is actually quite powerful to use numbers and narrative together as described here. Collecting both quantitative and qualitative data bolsters our understanding of our impact, helping us answer our questions to know what happened, why it happened, and how it happened.