Episode highlights

00:01:40 – About SL 

00:04:08 – Why is all research biased?

00:07:23 – Examples of how people bring bias into research 

00:11:32 – Is some research better than none? 

00:16:49 – Recruitment and empowering communities 

00:22:59 – Giving away the power 

00:29:46 – Final advice 

About our guest

SL Rao (She/They) is a strategist who co-creates new futures in education, health and climate through design inspired system transformation. They bring a system-oriented, equity-centered approach to their work and to how they build, nurture, and grow teams and organizations.

They are curious, and driven by the need to find solutions to seemingly intractable problems. We’re seeping biases in all the way from the beginning, how we thought about research, what we decided to research, how we do it, how we analyze it… And then how we publish it, how we see the people we’re researching. 

Podcast transcript

[00:00:00] SL Rao: I don’t think it’s like one person or the entire field trying to be malicious, right? We all. See ourselves as objective, which is incorrect. We’re seeping those biases in all the way from the beginning.

[00:00:20] Tina Ličková: Welcome to UXR Geeks, where we geek out with researchers from all around the world and topics that they’re passionate about. I’m your host, Tina, a researcher and a product manager, and this podcast is brought to you by UXtweak, the UX research platform for recruiting, conducting, analyzing, and sharing insights all in one place.

This is UXR Geeks and you are listening to a conversation I had with SL, a designer with a background in engineering and theater who spent the last years navigating everything from project design to public policy. In this episode, we are pulling back the curtain on something that the research world loves.

To pretend we don’t have our own biases, from who we recruit to how we interpret data. SL walks us through why objective research is mostly wishful thinking and what we can actually do about it. Tune in.

Hello, SL. This is the tradition of our podcast. Who are you SL and what is important to know about you right now in this phase of your life? 

[00:01:39] SL Rao: Great question. Thank you so much, Tina, for having me here today. Excited to be here. I, um, as you said, Shree Lakshmi. I also go by SL. For this phase of my life, I really am the culmination of all of these experiences that I’ve had, which is I have an engineering background, but I actually got into design through theater, doing a lot of children’s theater, and being able to bring people’s experiences into the storytelling.

So I take those pieces with me as I think about designing solutions. Doing research. I’m also an improviser right now. I went back to my theater roots and I bring a lot of that in the facilitation and the work that I do, and really centering the people who are actually going to be impacted by the solutions.

As part of the research work I do, the strategy work I do, and maybe the one thing that’s changed since when I first started on this journey is I started specifically in product, but over time thinking about what kinds of areas have the most amount of impact. I’ve moved towards like, you know, global health, public policy, government education.

So I’ve started taking on a more systems lens to my design practice and I hope to continue that. The theater 

[00:02:55] Tina Ličková: part resonates with me a lot. This is something everybody. Who went through theater, kind of takes with them into their jobs where you’re on the stage or behind the stage. That’s maybe one of the questions.

[00:03:08] SL Rao: Yeah. I’ve been, I’ve been on the stage as an actor for, for many years that I started an undergraduate and then I took a long pause and now I’m an improviser. So again, I’m back in front of the audience members. But you know, I’ve taken on a small like side hobby project where I am writing screenplays for film.

And so now this is my first time going in the backstage and thinking about how to tell stories in a different way. I think the idea of creating these visual stories of people who we don’t always get to see, and I think that kind of excites me. So a lot of the front end, a little bit of the back. 

[00:03:48] Tina Ličková: Okay. Telling stories in a different way. This is where I will bridge to our topic. When we were brainstorming what to talk about, you came with a topic of biased research. That is a big problem. I don’t even wanna sugar coat it. I would ask the typical subway question. What’s your take on it? 

[00:04:08] SL Rao: Yeah. Bias research.

I don’t think it’s like one person or the entire field trying to be malicious. Right. The roots of the research practice come from anthropology. Like historically, if we look at who were the people doing research, they didn’t necessarily have like big degrees, but predominantly wealthy white men during colonialism could travel to other countries.

And research people with this expectation that they are very objective, but they’re really applying their own lenses to the communities and cultures that they’re seeing for the first time and then saying, oh, this culture is backwards. This culture is savage because they don’t follow X, Y, Z. These are clearly biased perspectives.

And we have lots of threads of that. When I learned how to do research, this idea that I can just come into any space, you know, I went to grad school for two years. I know there are other people who spend their PhD five to seven years studying research and really going into the ethics of research. Also, a lot of us who come into UX are a couple of years a year, and then we go into places where we might not have any background history.

Shared lived experiences. Other than that, we’re all human beings. We all see ourselves as objective, which is incorrect, right? We’re not objective people. We bring our own cultural baggage and culture doesn’t necessarily have to be like, oh, that person is so exotic. They have a culture. The culture is in the ways we make sense of the world.

What are the values that we were brought up with? How are the ways those have changed over time when we’ve been exposed to schooling the communities that we’re part of our grandparents, how do we communicate? What do we do? How is gift giving done in our communities? We have these small, little cultural pieces that we forget about because we’re told we have to be objective, we have to assimilate, and we take those.

Cultural baggages perspectives. Biases with us when we do the research, who we decide to research and how we interpret it, right? And we publish our reports based on our synthesis or our analysis of someone else’s completely different lived experiences. So we’re seeping those biases in all the way from the beginning, how we thought about it, what we decided to research.

How we do the research, how we analyze it, and then. How we publish it, how we see the people we’re researching. We’re not doing research with the people. We’re researching people. 

[00:06:48] Tina Ličková: That’s a good point that I will probably go back to. One of the threats that I see there is what you are saying about the culture, about need of being objective, that it actually brings you towards.

Being less objective if I understand it, because you are unconsciously suppressing your cultural bias. My question of if you have examples, either from your practice or from practice of others, where you could see, oh, okay, this is where the objectivity just was throw out of the window. 

[00:07:23] SL Rao: Great, great point.

I think I have a couple of different examples, but I can tell you one from education. You can look at the data, I’ll talk about it in the context of the us. You look at the data and we’re seeing students who are maybe black, indigenous. Latino, Latinx, and low income students who are quote unquote underperforming. Right?

That we are also using that language saying underperforming, which we’re adding a label. That’s also a perspective, a bias, that someone is better, someone is worse. And then we say, okay, let’s develop a piece of technology. And there is a biased perspective of like, we think the problem is that they don’t get enough practice.

Or they’re not smart enough. I know this, like in today’s context, this sounds really bad to be like, oh, people of a certain race or gender are not smart enough. But those biases are also still there in the ways we think about technology. We might not say it, but we’re still thinking it, and that’s a lot of what the technology is.

So it’s a lot of technology that recreates practice and practice problems for students. And then there’s research that shows when they look at. The same or other technologies being implemented in high income schools. A lot of it is focused on creativity, harnessing students’ creativity. Where we see this deficit around schools, you know, students of color, students experiencing poverty is a lot of like practice because we feel like they’re not good enough.

Right. That gets really seeped in. So what happens is like even doing the research, you are going to be using those same analysis factors of like. The student isn’t meeting that mark, so they must not be good enough. But actually if we take a more systems lens and take it a the broader and really understand the context within which these students’ education is being experienced, are the schools well resourced?

Are the teacher supported? Are the students parents being supported? What is the historical context of schooling for these students in the United States? Indigenous students for. Decades, their grandparents were taken to Indian boarding schools where they were stripped of their culture or forced assimilation into the community.

So there’s this historic pieces around schooling. So without having that awareness, we’re bringing our bias and saying, oh, they’re not good enough. Versus being like, what else might be impacting it? Can we take a broader lens and can we shift our own biases? Like what are the biases we’re bringing? And we’re already coming in saying, these students are not good enough.

So we’re gonna develop technologies to make them good versus actually. Develop solutions that are going to be more supportive and actually support creative thinking because that is actually the things that are going to support students in closing the opportunity gap. 

[00:10:09] Tina Ličková: I get it. I get it. I’m thinking about one situation I encountered, I’m following on LinkedIn.

A very famous teacher in Slovakia. Slovakia is in middle Europe. Very small country, 5 million people. Uh, we have a strange. Very heavy political situation in the last couple of years and it’s very much showing in educational healthcare, not to branch out too much, but he, this teacher was telling, there is some governmental project where they figured out, oh, we should get to ghettos and Roma family.

It’s living in this ghettos internet because they somehow need to be connected. And my point looking at it, it was like, has. There ever been a person from this institution proposing such a thing because they sometimes don’t have access to clean water, like maybe some water, but not clean water. They have to go to other houses, or the children don’t have spaces where to ditch themselves

Very different set of problems. And sometimes you hear, especially in the business field that, oh, maybe even some research would help. And I’m wondering, what’s your opinion on looking at this bias as a noob, objectivity or objectivity without the awareness of the cultural biases that we are bringing into it?

Would you go to the root of, oh, at least some research is good, or would you be like, oh, better not?

[00:11:32] SL Rao: It’s a really good question because research can also be a tool of liberation. When we have understanding and awareness, it can actually be used as a tool of liberation. Right. It all depends on how we are approaching it.

Are we aware of the biases? Who’s doing the research? Like should there be someone who’s not in the Roma community doing the research? There’s a lot of practices now and have been for a while in the indigenous community across the globe, having indigenous researchers do the research, they’re adapting the tools, changing it to be able to approach it and understand the impact.

Of policies, solutions, technology, and surfacing up the actual issues that are needed to be focused on. So maybe the value is actually, are there researchers who have shared lived experiences or are lived experts and are able to go in and work with the community in a way that is not harmful, not reinforcing harm, and are able to gather input, increase visibility, and bring out ideas from the community or.

Are there ways in which the political system or the city or the town is able to create spaces where the community can gather in ways that are comfortable and safe for them to be able to raise the voices? There’s that piece of like. You don’t have to have a researcher go in and come back out. Does the community feel safe to do their own research?

This happens. A lot of times communities will have leaders or people within their community who are gathering information, the information’s being gathered and held, and they are coming up with solutions themselves a lot of times, but they don’t always have a way to share that with people who have power.

So it is nuanced. It’s difficult. There isn’t a single answer. It’s not necessarily like, don’t do any research. Let’s just leave it be, research can be a tool of liberation. How is it done? Who is doing it? How can it be supported? I think those are the things that you need to consider. I think about this a lot as like, am I the person who should be doing this research?

I worked at the state level on youth housing. The mandate was to reduce youth homelessness at the state level from different systems of care. And I don’t have lived experience with housing insecurity. I was a youth, you know, a little bit ago, so I don’t have shared age experience. I’m an immigrant to the US so there’s that part as well.

So the lived experiences pieces didn’t actually exist. However, I was put in a position of power to be able to think through and support actual youth advocates in creating the public policy. So really thinking about how do I shift, yes, I’m in a position of power. And how do I use that to facilitate spaces where the young people can develop their own public policy, create spaces where they can learn about the issues that are impacting them.

Because a lot of times we don’t see the larger systems that we’re part of. So the young people are learning together, developing public policy, and I can get. Written into policy language with system side stakeholders, and then create a pathway for actual lived experts to take over my role over time.

There are ways in which we can do that, and ideally we’re working ourselves out of a job and creating more spaces for lived experts to take over. 

[00:14:55] Tina Ličková: We will be a right back after a short break with a commercial message from our Spencers, Hey you, it’s UXR Geeks. How are you doing? You know that good research isn’t just about running studies.

It’s about getting real insights without getting stuck in the process. That’s why people often use UXtweak, not just because they support this podcast, but because the tool is actually simple and easy to use. I can recruit. The right participants run everything from early tests to usability studies and actually make sense of the results without wasting time cleaning the data.

It works for big and small teams, and I even collaborate with my clients directly in the tool. So if you’re curious, go to uxtweak.com website and start for free. No credit card and no strings attached. 🐝

We already went to, okay, what is happening when we are in communities, when we are already researching. But you pointed out where, well, and this is where I encountered inclusivity, exclusivity, diversity on various levels and did it many times really. In a wrong way on projects and that’s recruitment.

When I was thinking about what are we going to chat about, I realized, A) the years of living in German speaking countries, I had probably few Muslim respondents in my researches in the studies, and I had a one woman in hijab. And I’m not saying like we should recruit women in hijab because that’s bias as well.

But it’s just how do we actually make sure that when we are researching for businesses or for governments, we have the right people on board because this is where the problem starts. 

[00:16:49] SL Rao: Recruitment is always a hard piece, right? I don’t know, Tina, if you experienced this. Um, but when I went to school, it was a lot of like human experience.

Anyone and everyone, right? Like just to recruit any user, they’ll help you. And it might be true to some extent, but then you start digging into and you realize, oh, we don’t have anyone who is disabled. So we created an entire product, shifted entirely to the entire, the world. And we didn’t realize that it’s not accessible and now you need to go backwards and actually figure out how to make it more accessible.

So I think really with recruitment, what I think about a lot is looking at the data, doing secondary research to understand like, you know, in the education space I kind of mentioned what is the historical context? Why do I need to know that? I really do actually need to know that because that’s going to define.

Who has the least access to informing solutions? So a lot of times what happens is like startups, you know, they’re very scrappy working in the education space or health space. They’ll say, oh yeah, we’ll get user feedback. Let’s just ask our friends’ kids. Your friend’s kids are going to be in the same socioeconomic status, potentially.

Same racial background, same religious background like you were saying. Maybe not. Not someone with a disability potentially, right? Like you’re not actually looking for it. You just think you know all students. Every student is fine. Great. You got some feedback. Some feedback is better than no feedback.

However, what did you miss, right?

[00:18:19] Tina Ličková: Speaking about some feedback is better than none. Recently, more and more companies have started relying on AI personas to quote unquote collect user feedback instead of conducting proper user research with users. But to what extent should we rely on AI and UX research? UXtweak recently published a great report summarizing how UX professionals feel about AI applications in our field.

So if you’re interested in that, there are portal link as in the description. Now, let’s go back to Sal.

[00:18:54] SL Rao: So looking at the research to say, okay, who has the least access to power? And sometimes I actually, I know you said you don’t wanna recruit more of a particular group, but sometimes to undo the existing bias, I over-recruit a little bit for the groups that we don’t usually get information from, because traditionally we get.

A lot of information from quote unquote the average student. There’s no average user. There’s no average student, right? We’ve developed this idea, so really breaking that down to say, okay. We need more input from people who are neuro divergent, people who are maybe in rural areas because I don’t know what I don’t know.

I don’t know what they’re experiencing to an extent that might inform how they interact with something. Sometimes you might get information that might be outside of the range of what your product can actually service, and that’s okay. You put in your back pocket. For innovations later. Do students who experience poverty have a different experience?

You don’t know until you are actually recruiting for that and you actually realize, yeah, there’s a very different way in which these intersections of those experiences cause it to, you know, learn or you understand something completely different. Sometimes a product designer will design a product in the education space for the classroom and then not realize that it needs to also be able to translate, or the UX needs to be in multiple different languages because the children’s.

Caregivers speak a multitude of languages, but we’ve made this entire product in English and now the teacher is trying to figure out how to communicate with the parents. Right? So sometimes Oversampling is okay, but also recognizing even within that oversampling, we might still have one thing. We might be like, um, Muslim woman, but there might be queer Muslim woman with a disability.

I’m not saying recruit 20 of that specific. But really recognizing that sometimes you set these targets of saying, okay, we don’t have a lot of Muslim women, we don’t have a lot of people with disabilities. We don’t have a lot of queer people. Let’s say we want at least five of each. We have these variety of experiences, so recognizing there’s gonna be intersectional experiences, and I do that on purpose, like sometimes setting those targets forces me as a researcher to be more intentional about who I’m recruiting and making sure those voices are there.

I have recognized and notice that we get. Different solutions and they’re more innovative, more interesting, and have more opportunity to using tech terms, disrupt the marketplace because these ideas are outside of what a traditional more objective user would share. 

[00:21:38] Tina Ličková: I love the idea of Oversampling because then you can relook into an area and be more specific in it and get to know it.

Actually, and it brings me to the idea. I remember a discussion with the VP of product at HR Tech that I was working for, and he was just like, oh, let’s do a. Diversity study and it was very well meant with our users. And I was like, do we know that our users are diverse? And he was like, oh, okay, good point.

And he, he took it very well. And then we were discussing, ’cause the typical idea would be, I am the user of a website where I am buying treats for my dog, but who is the real user of those treats? And I’m now ex degenerating. This is where I would say yet even going to the people and potential users or potential audiences is really how it could be done.

And that might be oversampling of the audiences that we don’t know so much about. So I really love the idea. Really good one. Giving away power. You mentioned it already. It’s, oh, I would love to, but I have an assignment to do either for my clients or for my stakeholders. I have this goal in my mind, so giving a way power if I have to be in control.

And it goes also into the direction of researching, not on users, but with users or with people. How does it work? 

[00:22:59] SL Rao: Tell us, giving away the power. I think it wouldn’t be possible if it’s only the researcher and the designer who has to do that, right? You can’t just put them on the spot and be like, you’re empathetic.

You’re going out doing this work, you’re doing really good recruitment. Let’s say you’re oversampling, you’re really pushing the way things are done, and we want you to give up power. But we also know, I don’t know if you experience this, Tina, but for a lot of times when I’ve been inside product companies as a designer and a researcher, I used to get pushed around, right?

Like the product team, the product manager has these ideas and you’re like, Nope. That’s not what the users ask for over and over again. We’re like, let’s do research. And they’re like, no, but I have this idea. So you’re already pushed around and then you’re expected to move. So I think part of it is it is an organizational level change, right?

Like it is that movement towards. Thinking more holistically and less paternalism about the people who are using your product. And I also understand within a capitalistic system, the product company is still making profits and money off of this. It’s not going to be a fully power shared relationship. If you’re in government, you should really consider how you’re sharing power.

You are a government researcher, government stakeholder. You are in service of the citizens or the people in your community who should be holding a lot of that power. There’s that nuance. However, as researchers, we are the front lines. We have the ability to continue to help our team. Shift their practices.

So participatory research, our methodologies where you can generate ideas, co evaluate research. You can do it on a spectrum of how participatory it is, and then where it happens inside your design and research process. So you could also be co defining the problem. You can be co generating ideas, you can be coating, but the code can also happen on a spectrum of who’s doing most of the generating.

Who’s taking in the ideas how much of that is being implemented. You could go into a space where it’s very structured and say, you only get to design within this space and you might get some ideas. You take that back, or you have a more open-ended space where you’re like, what are the problems that you’re seeing in the health space?

If your product is in the health space, and then you are more honest about, okay, these are great ideas. Our product can impact these things. Let me take this aspect back to my team. I’ll put the rest in my backlog and take it to your team. You are acting like an advocate, so you are leveraging your power.

We all have power. Sometimes I think we don’t see that we have power because of our identities. You know, we’re a woman in tech. You know, even if you’re older, sometimes in tech you are less coveted because there are younger people. There’s ageism. You are a queer, you’re a person of color. We all still have power because we’re in that position where we’re getting paid to do research.

People might be more likely to listen to us because of these intersections of our identities. So we recognize that and we leverage that to say, what and how much do I share? Where do I shift my practices? So instead of going out and I’m researching the Roma community, I’m researching children of color. I sit with them and create a space where participating together and they’re sharing insights of things that they see and I’m less researching them, right?

You’re trying to reduce that as much as possible. It can happen on multiple spectrums and there are resources to help people kind of think through those pieces. ’cause it can get kind of complex. But there are ways in which you can identify where you hold power. Who decides what is a problem worth solving?

Who’s doing the research? Who’s setting the agenda? All of these pieces, you can decide how much power you wanna hold versus you design the protocol, you research, you come back, you synthesize. Can you synthesize with the people who have been part of this research process? You’re looking at the research and saying.

Is this ringing true? We did this with a project where we were developing a new version of this idea of personas, so we moved away from a traditional persona where it’s static, it can be isolating. It creates kind of stereotypes, especially when we’re thinking about communities that are marginalized. And instead, we created a more interactive toolkit where we can identify the systemic things that impact.

Students who are black, indigenous, Latin, and low income in colleges in the United States, and what are their mindsets? What are their structural things that might impact them in accessing things or navigating their college experience in specific ways? Based on all of the research we did, we synthesize part of the research just to make it less complex, and we brought students together to say.

Do you see yourself reflected in these experiences? What are we missing? What else would you like people to know? And we co-created the toolkit with them so that they felt comfortable sharing these pieces about how complex the student experience can be in navigating college. So are there places where the final product, the synthesis, can be done with the people who are impacted by the work?

All in all, you should also think about compensation. How much is your team making? How much are they getting compensated? Are there other ways that people can be compensated? So do we make referrals for the students we work with? Do we help with like tutoring, resume, skill building outside of the traditional?

We give you money and it’s a transactional experience. We give you money, you’re done. We move on. Incentivization. 

[00:28:34] Tina Ličková: Then the co-creation with the users or people, which I hear a lot in the last couple of months. This kind of like auditing from side of the people we research with. It’s having a stronger thing and I think it’s, that’s real participation.

So kudos to that. And what I really loved, and I know you are not waiting for my verification of your thoughts, but I would just wanna point it out because it strike me. When you were talking about we all have power, we are being paid for what we do, either research or product manager, whatever. If you feel empower, it’s most likely and it’s definitely way.

Easily to lose power or to let go of power when doing research because the power is somewhere else and you are already in power. So I wanna emphasize that because I know folks are feeling powerless when you just scroll through LinkedIn or some blocks. So, uh, yeah. Thank you for this thought. To close it up, do you have anything what wasn’t said and you still feel like I also have to mention this or get into this?

[00:29:46] SL Rao: I think we covered all of the things that I was hoping to talk about and I think the one thing I will say, I mentioned, you know, lived experiences earlier. Really the lived experiences are honoring your research participants for their full experiences and something that came up when we were talking about users, average users, by creating a space where you are actually trying to understand their full demographics.

More users might come out and share other things that are going to be valuable for your research. So if you’re just saying the average user and you’re saying, I’m just gonna look at gender and location, you get a bunch of people, they’re gonna mask a lot of other experiences, cultures and things. But when you create a space where.

You are actually identifying, okay? We wanna understand your culture, your background, your location, because that’s also going to help us identify nuances of variations of experiences that might impact how someone might use our product. So we did for a little bit was think about, you know, in the United States race is looked at as like white, Hispanic or Latino, African American, Asian.

And I think there might be one more. What we did was we expanded that. We said, okay, thinking about breaking down African American and North African, south African, middle Eastern, south Asian, east Asian, and then we saw students and people identifying themselves across the spectrum. They were much more open to coming into the research study because they’re like, this is a space where I can bring a lot of my identities.

We ask about neurodivergence, we ask about disability. When it is relevant. This is the other thing I want to be sure to say, is like, we don’t wanna collect too much demographic information if we don’t think it’s as important. If we want to be able to have a really good representation, I would collect it to the level that I’m like.

This is actually gonna inform the research, but you also don’t wanna collect so much that it actually ends up harming people because some of the marginalized identities can be used against people to harm them. So we just need to be thoughtful. Again, this is a place of power, right? We don’t know how someone might be harmed if that actually happens.

That means we’re in a place where we have the power too, unfortunately, influences someone else’s life. Thank you. Thank you for bringing 

[00:32:03] Tina Ličková: this topic, and thank you for speaking about it. 

[00:32:06] SL Rao: Thank you so much. Thank you for having me here. I had a really great time talking with you.

[00:31:15] Tina Ličková: Thank you for listening to UXR Geeks. If you enjoyed this episode, please follow our podcast and share it with your friends and colleagues. Your support is really what keeps us going. 

If you have any tips on fantastic speakers from across the globe, feedback, or any questions, we would love to hear from you, so reach out to geekspodcast@uxtweak.com.

Special thanks goes to my colleagues, to our podcast producer, Ekaterina Novikova, our social media specialist, Daria Krasovskaya, and our audio specialist, Melissa Danisova.

And to all of you, thank you for tuning in.

💡 This podcast was brought to you by UXtweak, an all-in-one UX research tool.