Our Shared Sector: Your Evaluations May Be Biased. Here’s What You Can Do About It.

by YW Boston


Nonprofits are increasingly interested in measuring their diversity, equity, and inclusion efforts to determine whether these are actually effecting change. Whether you are evaluating efforts within your office or your community, evaluations are important tools for measuring progress. They are a valuable tool that will inform any adjustments, should you need to course correct. Yet bias often shows up in evaluations, just like it can show up elsewhere. This unaddressed bias can jeopardize the success of your DE&I efforts.

YW Boston has focused on developing evaluation tools that allow us to effectively measure individual and organizational readiness for D&I work, as well as tracking progress and areas of improvement. Here are some tips to help you identify and disrupt bias in your DE&I evaluations.

  1. Acknowledge that bias is always present and address it accordingly

The first steps towards addressing any challenge involves acknowledging the problem at hand and exploring possible root causes. People are biased, and as biased individuals, we can reproduce our biases in everything from self-assessment and decision making, to the tools and technology that we use. Prejudice and racism are institutionalized, so it is important to recognize that bias will be present within data—such as demographic data—and processes, regardless of where and how data was collected. Try pushing beyond quantitative data as it does not always tell the full story. Your quantitative and qualitative data may be communicating differently, so it’s important to gather more perspectives in order to gain deeper meaning. One common hurdle in evaluations is the inability to disaggregate. Disaggregation is critical to identifying singular data on specific identities such as race, gender identity, class, abilities, as well as intersections of identities.

  1. Examine what is being measured

In the words of Marc Miringoff, “we measure what we value.” Our environments, experiences, and institutions will impact our ideas about what is important and what should be measured. Therefore, institutionalized bias, prejudice, and racism will impact choices about the data we gather, the findings we prioritize and the meaning we ascribe to them. One way to mitigate racism within evaluations is to examine who is performing them. Evaluators have a lot of power, so ask: Are your evaluators diverse? Do they have an understanding of power and privilege? What data collection methods are being prioritized? Is your default indicator a white male?

  1. Consider who determines outcomes

It’s equally important to consider who’s involved in the post-evaluation process. After assessments are complete, someone will interpret the data and decide how to move forward. Consider shifting power dynamics from institutions and “experts” to communities and individuals most affected by the research. Evaluators can ask themselves, “Who is not included?” Define your theory of change and what it would take to achieve your outcomes. Identify your timetables and gather input from those who will be involved in making them happen. Think about what a negative or positive outcome might mean and who will frame those results.

Nonprofits should prioritize an equity lens throughout the process, even after evaluations have concluded. Be intentional about how your share data and make sure participants know what is being measured and why.

About YW Boston 

As the first YWCA in the nation, YW Boston has been at the forefront of advancing equity for over 150 years. Through our DE&I services—InclusionBoston and LeadBoston—as well as our advocacy work and youth programming, we help individuals and organizations change policies, practices, attitudes, and behaviors with a goal of creating more inclusive environments where women, people of color, and especially women of color can succeed.