Insight Leaders Weigh in on Democratized AI for Research

In a recent discussion with insight peers across industries, from Tech to Travel & Hospitality to Healthcare, C Space and Escalent explored democratizing AI for research. The participants were at different stages in their AI journeys, from early pilots with third-party tools to more advanced collaboration with custom, proprietary solutions.

Together, we examined AI approaches, unpacking experiences and learnings, and discussed how best to navigate the future. Here are the three big takeaways from our conversation.

1. AI is being used in insights as a copilot to extract more value from existing workflows.

During the last year, the narrative around practical applications for AI in research and insights has shifted significantly. Use cases that seemed utopian are fast becoming a tangible reality, and predictions about the shifting, ever-more strategic role of humans now ring true.

The discussion revealed common AI use cases embedded along the insights funnel that support:

• Synthesis of existing intelligence from disparate sources to join the dots, generate new insights and plug information gaps to better target new research briefs
• Desk research to build foundational understanding of new explorations and jumpstart hypothesis generation
• Insights at scale with open-end verbatim analysis, facial coding, tonal cues and more, delivering rich texture from hard quantitative data that can be statistically examined
• Concept pretesting and performance predictions with synthetic personas before in-market testing

While AI supports these use cases, concerns remain around sporadic inconsistency, information security and data privacy, and generic answers, calling for a strategic and measured approach to AI adoption for insights workflows.

2. AI is proving itself as an accelerator and force multiplier, creating space for researchers to focus on high-value activities: landing the story, driving implications and fueling actionable insights. 

“Given the current high demand for insights research and the smaller size of our teams, it’s impossible to meet all needs.” – Health Insights Leader

AI is empowering research teams to do more with less. Not only is demand for insights high, but smaller team sizes can make it impossible to support business needs at scale without embedding AI into existing workflows.

Rather than putting insight jobs at risk, AI may in fact reemphasize the importance of human intervention and consumer research by driving stronger ROI and making the case for more customer-centric, human-led-and-AI-enabled approaches.

Part of our discussion zeroed in on use cases that are still out of bounds, where guardrails around what is recommended, acceptable or impermissible have been established, and the factors governing such decisions.

“Interpreting the will and needs of executives is something that AI is not going to be able to do anytime soon, so the work of the research department remains vital.” – Tech Insights Leader

For most, it is too soon to rely on AI for storytelling or summarization. Accuracy, hallucination and comprehensiveness concerns are much too prevalent; AI explainability issues persist; and ultimately, humans have the unparalleled nuance and experience to craft the narratives from insights that land with executives.

As one insight leader puts it, there’s simply too great a risk of “bad deliverables, or deliverables that make good research less compelling.”

For now, at least, the premium on human nuance, context and experience is here to stay.

3. A journey from third-party to custom tooling can help drive organizational comfort, buy-in and adoption to reap greater utility, privacy and scale benefits.

What began as a broad provocation around ethics quickly narrowed to the topic of confidentiality and data privacy, from the competitive implications of synthetic datasets to the benefits and drawbacks of third-party tools vs. custom, proprietary solutions.

Third-party tools, which are often plug-and-play, are quicker to adopt and serve as a solid starting point for researchers early in their AI journey. However, such tools can operate as black boxes, making it difficult for organizations to understand how data inputs are being used and retained, raising a multitude of problems for compliance and security. This affects utility, of course, and poses challenges for training and useful scalability.

While custom tools might entail higher upfront costs, they can be tailored to specific business needs, providing enhanced data security and allowing businesses to address unique requirements and use cases at scale.

The answer to the build vs. buy debate might be a hybrid approach that balances the benefits, drawbacks and cost implications of third-party vs. custom tools against the use cases and the business impact AI is expected to address.

Mitigate the prevailing challenges of bias in AI.

Diversity, Equity and Inclusion (DEI) remains a complex topic. The bias problems of AI persist beyond their early signals, sometimes with profoundly damaging consequences.

Earlier this year, Meta’s AI image generator was criticized for its inability to envision an Asian man with a white wife, and in 2023, a study in Nature reported that “four major commercial LLMs all had instances of promoting race-based medicine.”

While we might praise the significance of facial coding or tonal cues to support transcript analysis and enhance qualitative research, there are issues with inputs that lack diversity. They may fail to correctly interpret local vernacular, code switching, or the emotion(s) associated with an expression. As one recent study from Cell notes, there are “some cases where the same facial expression could convey different emotions to different individuals or cultures.” Interview participants with autism, for example, may find it difficult to maintain eye contact. Physical disabilities may affect how emotions are expressed on the face or through body language.

The impact of these developments seems net positive, but there are critical limitations to the current state of tools that will have profound effects on the way we approach inclusivity. These must be factored into the ongoing design and development of this technology if we’re really to make rich use of it.

What’s next for AI in research and insights?

The rapid evolution of AI brings forth exciting opportunities, as well as critical challenges for the insights industry to address.

At the Escalent Group, we’re taking a hands-on, inclusive approach to the way we adopt AI and build AI-focused solutions. To learn more, contact us.

Want to learn more? Let’s connect.