Episode highlights
00:01:48 – Why UX Research should not sit under Product
00:08:08 – Servant leadership
00:11:40 – How companies benefit from having UX researchers in the field
00:16:27 – Will UX Research lose influence if moved under Operations?
00:21:23 – AI in UX research
00:26:10 – Connecting UX research with retention and sales metrics
00:28:12 – Summing up
About our guest
Joel is the Founder of FloatGoat, a Researcher, Service Designer, and an Accessibility Specialist. He is also a father to three dogs, three tortoises, 11 chickens, and more plants than you might expect. In his off hours, he’s on a river kayaking, camping, sometimes fishing. He cares deeply for the environment and especially rivers and the natural world. Joel is a mentor to many, and a friend to all.
Right now UX is fighting for its very life and until we get past the whole Kumbaya thing and start having real conversations about the value of measuring and understanding the user experience, we're going to die to the robots. Because we're unwilling to have conversations. 
Podcast summary
In this episode of UXR Geeks, Petra Rajkov talks with Joel Barr, a UX researcher, about his argument that UX research should move from under product into operations.
Joel’s thesis is that UX got folded under product during COVID and tied its value to product metrics, a positioning he calls misplaced. He argues research belongs closer to sales, implementation, and customer success, where researchers work in the field rather than remotely.
He describes a warehouse project where a new garage-door and ramp interface saw near-zero adoption: the touchscreens weren’t glove-friendly, lacked language support, and had an alert beep loud enough that workers taped cups over the speakers. Joel’s team routed the findings to operations and engineering, and the ramp’s usage rose to 95 percent.
Joel contrasts this with a shortcut he sees growing: relying on AI-generated insights or synthetic participants instead of users. He tested AI tools by feeding them transcripts of his interviews and examining how prompting influenced their responses, finding that the AI would always take on whatever emotional tone he asked it to give, which led to poor analysis and made him question the integrity of AI-driven UX research.
What this means for UX research and the future of the discipline
For UX researchers
Get on the travel schedules of sales, implementation, and retention teams, and frame your involvement around helping them hit their own KPIs — even if it takes months and doesn’t fit a sprint cadence. Tie your value to sales and retention numbers rather than CSAT or NPS, and treat pushback on field access as a signal to look elsewhere.
For UX research leaders and managers
Staying in the field yourself, rather than moving into a purely strategic role as you rise in seniority, keeps your team’s insight grounded and your credibility with direct reports intact.
For product managers and product leaders
If research reports into operations rather than product, expect insight to arrive framed around usage, retention, and workflow breakdowns rather than product roadmap priorities — and treat that as a different, not lesser, kind of evidence.
For teams evaluating AI tools for UX research
Test any AI research tool the way Joel did: feed it the same material with different emotional framing and see if the output shifts to match. If it does, that tool is producing whatever conclusion you prompt for, not independent analysis.
Podcast transcript
[00:00:00] Joel Barr: Right now UX is fighting for its very life and until we get past the whole Kumbaya thing and having real conversations about the value of, measuring and understanding the user experience, we’re going to die to the robots. Because we’re unwilling to have conversations.
[00:00:50] Petra Rajkov: Our guest today at UXR Geeks is Joel Barr. Joel has spent years working closely with product and design teams. He is someone who’s been thinking deeply about how UX works in real organizations beyond just methods and deliverables. What I find especially interesting about Joel’s perspective is that he’s not just focused on how we do research, but on where research sits, and how that shapes the impact it can actually have.
[00:01:22] Petra Rajkov: In a recent piece, he explores the idea that UX might be much more effective if it was embedded in operations rather than product, which is what we will be unpacking today Hi, Joel, welcome to UXR Geeks podcast. I am really happy to have you here today and hello to our listeners.
[00:01:47] Joel Barr: Listeners, salutations!
[00:01:48] Petra Rajkov: So this is actually my first episode as a host, and you are my first guest. So it’s a new experience for me. But I’m excited to be talking to you today because we will be talking about a bit of a provocative topic, about your hot take, let’s call it that way. And you have argued that UX research shouldn’t sit under Product, but it should move under operations. Right?
[00:02:15] Joel Barr: Not just research, design too.
[00:02:17] Petra Rajkov: Okay, so let’s unpack it. Let’s, let’s go back a little bit. So can you tell me a little bit more, how did you come to this idea? Was there anything wrong that you see happening with UX Research the way it is positioned standardly in the organizations?
[00:02:34] Joel Barr: Actually, I wouldn’t say it’s wrong, but it certainly could use improvement, right? There’s very few times life where you have an absolute good and an absolute bad. There’s always gray areas. Any UX researcher will tell you that, right? It’s all gray, it’s all beige. I’ve enjoyed a career that when I started out, it was a field-first business.
In other words, if you were gonna be in ux, it was pretty well that you would be out in the field with your users. And then we got away from that. We started doing more remote research, and then COVID happened and we all went remote, and then we all went back to the office. But user research in UX design never really got back out into the field.
And so my thesis, if you will, is that, we got folded up underneath product during COVID more than anything. And. Tied ourselves to, product metrics, to show our value to an organization. My thesis is, that is misplaced. Where we should be is under operations with sales and implementation and such that. We tie the value of user experience into the selling of that user experience for a customer. And in my career, the most, valuable and the most impactful pieces of research that I’ve ever done with my teams has always been in the field. Always. if you do it remotely, you only get little vignette, yeah. Of a user’s life and where they are using your products. you’re in operations, if you’re out with sales and implementation and retention. You get to see the entire user ecosystem of how they’re accessing your product and why they choose to access your product in a given context. You don’t really get that, looking through a window looking through a window on a computer. If you can even do that anymore. Of course, nowadays there’s tools everywhere that prefer to do all that work for you. And then we have the rise of the machines who say that you don’t even need to look at, you don’t even need to talk to humans anymore, we can just generate synthetic humans. And and you can just do research on that.
[00:04:42] Petra Rajkov: That’s pretty scary. I want to focus a little bit on that move. From product to operations, because in some organizations it’s the way it’s done, it’s the way people are used to working, and in some organizations it works, right?
Because the researchers work closely with the designers, with the product managers, with the development teams, and they’re enjoying this way of working, but what we see happening is the democratization of research. So they’re asked to do less and less. So what would you recommend to these researchers who are in the middle of this change?
Because just moving to the operations is not possible immediately. Are there any small steps that we can take in between.
[00:05:31] Joel Barr: So you’re right, like, it doesn’t work for everyone. if you have a completely virtual product, let’s suppose you’re Zapier or you’re in the WordPress space, which is a gigantic virtual market, where there isn’t much, physical component to product offerings, my thesis doesn’t work for them. And I’ll just be honest, they’re, it’s got limitations like any good science does. if you are sitting on a product and you have a physical product or a software as a service that goes out to customers, then your first call really is to, sales, right? and understanding. Their KPIs and understanding what they need to do to meet their metrics, and that effort. talk to people in implementation. Ask them when they go out into the field, see if you can get a seat on that plane or in that car, and just go out with them. And while they’re doing their work in product or doing retention or customer success, you’re right there, also in the field with them.
But finding out the real stressors of users in the field. when the rubber meets the road, as it were. So as people understand product, on the ground, you’re at the ground level. You are in situ. If you’re an anthropologist, you’re in the middle of the house, as it were, rather than looking at the house from the outside and just, guessing what goes on inside. So the real steps are just making inroads in inside your company and having conversations and finding out, how can UX affect outcomes of sales professionals and implementation and the rest of the people in operations. if you can work on increasing their KPIs or exceeding them, it becomes a very easy conversation to have because you’re helping them succeed. You are the conduit between them and the user, and through you, the enterprise will end up building better products because you’re on the ground, right where the products are. Instead of just in a theoretical capacity.
[00:07:21] Petra Rajkov: And you have argued that when UX sits under the product, it becomes a support function. That it’s not the ideal way for UX research to work. Right. But I like this term servant leadership, and that’s how I see UX research in organizations. We are not the leaders telling people what to do, we should connect with other teams in the organization.
We should connect to sales, we should connect to marketing. We should be connected with, developers and so on. And through the servant leadership, through asking them questions and collecting the information, we should create these insights and give that to them so they can make smarter decisions and work faster and so on.
And, having said that. when I read your thesis, I was taken aback a little bit because when I saw the notion like, UX should move under operations, it was hard for me to grasp. But when I look at it this way, it makes way more sense to me. So is there anything that, that resonates with you when it comes to servant leadership?
[00:08:27] Joel Barr: They say that if you get above a certain level, your role becomes strategy, and then you just accept that you’re not gonna be in the field anymore. I don’t find that to be true. I think the best leaders lead from the front and not from the rear. and I think that,I could be a director of UX, I could be a Vice President, Grand Poobah of UX right at some gigantic enterprise. And I’m still gonna be in the field with my direct reports who are also in the field not because I am there to watch over them, but because I’m just as interested in solving the problems of humanity as it relates to enterprise product, as the rest of my researchers are.
I think you know, the best leaders I’ve ever had in my career have been field first UXers. other words, we can design anything we want on paper and we can design it in some kind of software, Figma or whatever. But until we’re out in the field and we actually watch humans use our work, have no idea. Really what we’re designing for. it used to be, gosh, before COVID large enterprises regularly had laboratories could simulate real life in the use of our products.
We’ve gotten far away from that Now. and and you really only see it in the largest of enterprises, right? some of them have, mock stores that they bring customers into just to watch ’em. I think the best work is done out, like in situ, even if it’s just. Watching people, in an ethnographic kind of way, just watching how people behave, learns tons ways. We can affect product of ways, and so I,think there’s validity in the servant leadership standpoint.
I think there’s validity in disagreeing with the entire thesis I didn’t want everybody to just be like, oh, I totally agree, and just pass it off. I wanted to have debates and discussions right now UX is fighting for its very life and until we get past the whole Kumbaya thing and having real conversations about the value of, measuring and understanding the user experience, we’re going to die to the robots. Because we’re unwilling to have conversations.
[00:10:28] Petra Rajkov: But there is still the demand for research, but it’s changing the way it works, what they want, and how fast they want it and et cetera. And some researchers are like, that’s the only thing they can do is the behind the screens test.
[00:10:43] Joel Barr: User results may vary, right? My thesis is essentially that if we’re, if you’re unwilling to or unable to fight for the ability to get in front of real physical users, then maybe you’re not at the right organization if that’s not valued. By your organization, if you espouse the same values being in the field as I do and others do, where you’re measuring, things like value and trust and stress. One of the greatest studies I ever did was around stress levels individuals running from a terminal to a rental car counter. And we measured how stressed out they were while having to use the app on the run. And we got our little stopwatches out and we measured how far the distance was and then we measured, just how well the product worked in the hands of a normal, regular, everyday user, but also how stressed out they were and how much, or how little helped alleviate their stress by its ease of use. You’d never get that remotely,
[00:11:40] Petra Rajkov: So what I’m gathering here is. It’s not only about the move from product into operations, but it’s also about the infield research. That’s something what needs to be brought back. So we are in direct touch with the users and the way they’re interacting with the products that we are developing.
If you can walk me through an example of an organization before making this shift and after.
[00:12:07] Joel Barr: I won’t tell you the name of the company, but it’s very large and, and you probably order from it. and, one of the things I was tasked to do is to go into the warehouses of this particular organization to work on, the interface of, garage doors and, lift extenders. Not the sexiest work in the entire world. like watching paint dry at times. The problem was, is that this particular organization came out with this new way of opening and closing garage doors and extending and descending ramps. I’ll just give you a little spoiler alert. Garage doors go up and down.
Ramps go in and out. It’s all you really know, need to know about what they can do.
[00:12:49] Petra Rajkov: Makes sense.
[00:12:50] Joel Barr: The way in which you get those doors to open and close and ramps to extend in descent, was this fancy, one button and it’ll take care of everything, as it turned out. They didn’t actually study a single user before they created this newfangled way of doing things. It was basically was by product thrown into engineering, developed and thrown out into the field and the old fashioned product first way. And we’ll fix whatever goes wrong in the field. we’ll watch the Jira tickets as it were. it turns out that warehouse workers are not particularly, tech savvy, And so the call went out Nobody is using this new interface that’s worth millions and millions of dollars that it took to develop it. Nobody uses it. people go and do the manual way instead, which wastes on average in about a minute and 30 seconds of time.
And so they deployed myself and others out to these warehouses to find out why people aren’t using this particular interface. To make garage doors go up and down and ramps to go in and out. it became quite clear, no user was consulted in its design. And you know this because, One of the things that this particular company’s warehouse’s safety mandates is that all users, all employees, wear gloves, to protect their hands.
Unfortunately, the screens that they have to interact with aren’t glove friendly. You have to type in little tiny letters are not great for gloves. There’s no language support. So if English is not your first language, there’s not a lot of options for you to work on. as you might imagine, warehouse workers come from different places as do we all. and, a lot of them just didn’t understand the verbiage of what was being asked of them to do and then there were the other problems with notifications and that sort of thing on the screens themselves. It’s a very complicated system. It’s intended to be in its design, in its function, but when it goes wrong or when a user. does not, do things in the right sequence. the product gets out of hand rather quickly, to the point where a door goes down and is no longer functioning. In these warehouses, no air conditioning, no heating, which really got down to the grits of why these folks weren’t using this product and forwarding up our recommendations not to product. But to operations and engineering changes happened overnight.
They were rolling out updates week after week, now they have 95/98% usage rate for this one system. So that was a real eye-openers a very large enterprise, that does a lot of business and here we are, little user researchers out in this little small town in the middle of Kansas, reporting back to this, place on the West coast that their system that they invested millions and millions of dollars in, one doesn’t work for the user, two meet the user’s needs, and three needs to change or nobody’s really ever gonna use it. they were rolling out changes. In weeks, not months, not years. And it was all evidence-based and videotaped and reported on. and here’s what needs to change.
And I’ll give you another great example of this. On the doors, on the interfaces, when you’re doing opening and closing operations, there was an ear splitting beep. That would go constantly while it was doing these operations. Just, made your ears ring sound loud and just at the right tone to where it makes your ears ring afterwards for about two minutes. And the manual mode did not have that beep.
Only the new mode did. And to be honest with you, these guys were taking, styrofoam cups and taping ’em to the speakers. They were, doing all sorts of ways to get it to stop And it was a terrible quality of life thing. Engineers don’t know about that. The product, people don’t know about that.
To them, it just, it’s, that’s the way it’s supposed to work, ’cause of safety and trying to alert everybody around you that you’re operating a really big door that can crash. But if you’re a user, you’re right up against that speaker. And it is loud. and so one of the most immediate fixes that we sent back from the field, gotta either figure out a way to lessen that beep in volume or just take it out altogether or make it more pleasant because my God, and they, and it took them a week and they eventually, just took the beep out altogether.
[00:17:12] Petra Rajkov: And you have that in SaaS as well. When users find a workaround to something what was not intended that way, and suddenly they’re using your product completely differently to what you have intended and. You couldn’t anticipate that, and you get that through research.
[00:17:29] Joel Barr: So far that’s the case, yes. I’ll give you another one.same idea. and we were at a company that, sold, oh, shoot, Hondas, I think it was Hondas – foreign cars. They were big users of our products. They were probably one of our biggest customers. And we get there to Vancouver, and we find out that, they’re not using our product necessarily. They were so tired of the product working poorly that they went and built their own that sits on top of it so you want to talk about workarounds. They made an end-to-end workaround that sat on top of CDK so that it would work in the way that worked for them and be like, taking a, car that you wanted to go across the river and putting floats on it. Still driving the car, but the floats are what keep you above the water. And that was a very eye-opening experience and so we reported back, we’re like hey, you’re gonna want to study these guys more because they’re making your software work by building something on top of it, like a re-skin, a re-functionality of your software, and it’s working great for them. And I tell you that raised some alarm bells.
[00:18:30] Petra Rajkov: Well, as it should. Yeah, definitely. Okay. But if I circle back to the operations, right? So I still see in product a lot of strategic decisions happening. So if the research is moved from product, do you think that there is the risk of losing this, this, power of, the, of the impact on strategy?
[00:18:50] Joel Barr: I don’t think so because you. is only so good as the evidence allows, right? If you have incomplete evidence to back a product decision. you’re lacking research, what are you really strategizing around? I think that should be an open question. If you don’t have like on the ground research, what are we really strategizing around?
[00:19:08] Petra Rajkov: Yeah, but that’s what I’m thinking. You also mentioned, at the beginning that if the, if the lead or the CEO is not listening to the research and all the main decisions are not based on research, then you are developing for no one really. So that made me think whether this is a structural issue or a leadership issue because I have experiences with a company, where the CEO was coming to the research calls.
The GM was listening to them on a regular basis, so they stayed in touch with the user and they understood the user problems and pain points and et cetera. And that to me was leadership. Where research was right in the center of all the things.
[00:19:49] Joel Barr: My thesis does not apply to everyone. I am fully aware of that, and I make no assertions that my, thesis or my ideas are ubiquitous. other words, your mileage may vary. And if it, my point of view is something that you enjoy and you wanna put in your own organization, do! If you don’t enjoy my point of view it’s okay. I, that’s why I made it. I want you to push back on it. I want you to say, you know this doesn’t work in my organization.
Okay. That’s okay. But yeah, I like leadership, I think is a big part of it. I’ve interviewed at a number of companies lately where the head of UX was never in UX to begin with, they sat in marketing or they sat in product and they got knighted into UX is a kind of a fascination, or because they needed the chair filled by a body that they didn’t want to go out and hire. And now they’re trying to hire, Field and very experienced and designers to sit on their products some more.
And I’m just like, man, we could do so much better. We could do so much better. And I understand why decisions like that are made. but I think, where we’re getting to now, is we’re humans are beginning to be able to delineate or be able to tell the difference between companies are phoning it in, right? Or using slop as it were, to comprise their experiences and those that actually, put the work in and study. I think there’s, humans are starting to see a real difference there.
[00:21:15] Petra Rajkov: Yeah, and we touched on that the usage of AI tools when the companies are trying to substitute the researchers with the AI tools and they think like, the good enough result is all they need all or the good enough research is all they need.
[00:21:31] Joel Barr: Which is funny ’cause AI user research is so easily manipulated. every so often I’ll, run an experiment where I’ll take insights from a study I’ve done or something like that from an old study and throw it in AI transcripts, then it asked it to, analyze it and, you know, back with results. And then I’ll say to it, no, that doesn’t work for me. I really need more negative responses. And it just says, okay, and it just rewrites it negative. And I’d say, oh yeah, that’s too negative. Can we make it a little more positive? Oh, sure. that just does it. tells me AI insights, especially if you’re using synthetic users, there are very few guardrails, that account for integrity of the research evidence. In other words, if you want to use synthetic users or, artificial intelligence to back your product point of view, very easy to just ask the AI to come up with whatever you need to back your point of view. And I think that’s a real danger now, right? I think that’s a danger because, everything is all about saving money on labor.
And why wouldn’t they be? Because what’s coming out is not increasing their sales or their bottom line. So they don’t have the money to, afford to hire anybody or even the travel, For field research. yeah, I don’t blame ’em at all, but I think it’s coming to a point where, there’s a fair amount of, it’s not slop directly, but there’s a fair amount of finagling or tweaking going on, in product insights that, either UX doesn’t feel empowered to speak up or they do and they’re shown the door. And I think we’re gonna come to a point where humans, everyday humans, general population humans will be able to start telling, the difference between product-backed research or research-backed products those that are just slop.
[00:23:09] Petra Rajkov: Yeah, well, in UXtweak we actually conducted a meta-analysis on, the synthetic participants. So we have collected everything that it’s out there, all the studies that have been conducted, across different industries. There was healthcare, there was, technology and et cetera. And that’s the, the findings that we gathered is that the synthetic participants should not be used as, methodology.
There are a tools to help you get started, to come up with your research plan and so on, but it’s not something that should be your final outcome because it’s just too vague, it’s biased you cannot go into so much depth. You don’t get those random wow moments with synthetic participants.
And with AI. You cannot have AI as a human element, right? It’s still not there, so I’m completely there with you.
[00:24:02] Joel Barr: Not yet. Every day they try and replace UX research, every single day. And I knew that, I knew when I signed in, I was gonna be the underdog for life. It was just what it was part of what you accepted. didn’t have seats at the table for the longest time.
We had to fight for those. we have to fight for our very existence to be not replaced by some automated, moderator. And those robotic moderators are terrible. I was watching one, I forget what company came out with it. but the user said on, the third question that they didn’t like talking to automated systems. And the thing goes, oh, that’s nice, and just picks right up with the script right where they left off.
And I thought, oh my God, what, any human researcher would see that and be like, okay, maybe I should do you wanna end the call? If you’re not comfortable talking with me, I’m happy, ’cause people have different traumas, different triggers. We have to be mindful of that as researchers, But the robot moderator just went straight on. And it affected the whole rest of the conversation. And they put it out as a demo, it was a couple years ago… and I just, I remember going on the comment section and just going off and being like, why on earth would you do that to a human?
[00:25:08] Petra Rajkov: Yeah, the, there is so many things happening that we as researchers don’t like, and I think it, has its due date, like people eventually will realize that they cannot get the real insights, the, the good quality of research from, these AI tools, but it’s gonna probably take some time.
[00:25:28] Joel Barr: I don’t even talk to recruiters anymore that use automated systems to talk to me. I just, hi, this is such and such from some kind of recruiting company. Is this such a and no, I just hang up. I don’t care if it’s the greatest job in the world. if they’re not willing to commune with me on a human level. I just have no time for them.
[00:25:46] Petra Rajkov: Yeah. I think also, context is really important when it comes to the AI tools. I was gathering that from what you were saying as well, that AI tools do not have that context and research should be connected.
You have mentioned that research should be connected to sales and to the KPIs and to the revenue and understand the retention and so on. So what can researchers do today to better connect their work to revenue and retention and trust?
[00:26:15] Joel Barr: So the way I would advise people to go about having those conversations is to try and figure out the schedules of the folks that are going out into the field of site, and I will tell you that it will not probably fit in very neatly with a product Agile sprint, right?
Because sales and implementation, retention and customer success, they all go out at different times and, but if you can get on the schedule to, or on their travel schedule, or at least understand where they’re going, you can start having conversations ahead of time. And it may take a matter of months, it may even take years to pull off, but eventually, if you keep making the case for the need to be in the field, with your users understanding how and why and where they used products, you’re gonna change the entire way the entire enterprise at their human users.
In ways that robots can’t do. And I think that’s one of the, going back to what you said about the synthetic, using researchers, the one place robots can’t go or understand yet. ‘ cause who knows, 40 years in the future that might be different, but they can’t understand the context of where humans exist. to an ai, the world looks in the same way. that a mirror at a funny park or an amusement park looks right. It’s all topsy-turvy and all that. that’s an AI’s understanding of humanity. And when we look away from the mirror, we see the world around us. And I think that a lot of enterprises is enamored with the way the world looks in the funny mirror.
[00:27:40] Petra Rajkov: I want to turn it to something positive because we were talking about a lot of dangers, a lot of risks and et cetera. But I do think that there is a future for research, and I was gathering that from you as well, right? That eventually the management will learn that developing the products without talking to the actual users and seeing them use their products and so on will just not make sense, right?
So, to sum it up, you have advocated for the infield research, for moving research from under product to the operations so they can support all the different teams and gather all the information that is in the company and so they can be in touch with the users, right? Is there anything else that I am missing here?
[00:28:33] Joel Barr: Yeah, so you’re tracking correctly, but the metrics of the value of UX, both design and research being tied to sales and not CSAT or NPS or something like that, something arbitrary, is where really we show our value in a way that companies cannot ignore. For so long we’ve been asked to show our value to what are your, what is your value to the organization?
We’ve struggled really hard as a field try and make that salient. And I feel like if you can tie it to things like customer retention sales and implementation numbers, suddenly the value of the user experience becomes very real for a lot of companies, and I saw it at the companies that I talked about.
[00:29:19] Petra Rajkov: Well, perfect. Thank you so much Joel! I think we have covered, a lot and we have also some takeaways and some practical applications of what the researchers can start doing today to be ready for this change and for everything, what’s coming to us. So thank you so much for coming, Joel. And I will see you around.
[00:29:40] Joel Barr: My pleasure.
[00:29:45] Petra Rajkov: Thank you for listening to UXR Geeks. If you enjoyed this episode, please follow our podcast and share it with your friends and colleagues. Your support is really what keeps us going.
If you have any tips on fantastic speakers from across the globe, feedback, or any questions, we would love to hear from you, so reach out to geekspodcast@uxtweak.com.
Special thanks goes to my colleagues, to our podcast producer, Ekaterina Novikova, our social media specialist, Daria Krasovskaya, and our audio specialist, Melissa Danisova.
And to all of you, thank you for tuning in.
💡 This podcast was brought to you by UXtweak, an all-in-one UX research tool.
