From Gallup to Generative AI: Why Research Methods Still Need Accountability

I was lucky enough to be part of the last class during grad school at UConn to hear from the founding families of survey research: Bud Roper, George Gallup Jr., and Helen Crossley, the direct successors of the three families that founded scientific polling and market research in America.

They didn’t just teach methods. They taught why those methods existed: because getting it wrong has consequences.

The breakneck speed of transformation in market research invites us to look back at the major changes that brought us here.

A Brief History of Research Transformation

1. 1936 – Gallup Proves Sampling Beats Scale
   Literary Digest polled 2.4 million people and got the election wrong. Gallup polled thousands with better sampling and got it right. The founding lesson: representativeness beats volume.
2. 1934-1950s – Probability Sampling Is Formalized
   Neyman’s stratified random sampling theory, followed by Hansen, Hurwitz, and Madow’s operational frameworks at the Census Bureau, gave survey research its statistical foundation: the infrastructure every margin of error depends on.
3. 1940s – The Focus Group Emerges
   Merton and Lazarsfeld at Columbia developed focused group interviews to study wartime propaganda. 80+ years later, it remains one of the most widely used qualitative research methods in the world.
4. 1950s – Dichter’s Motivational Research
   Ernest Dichter brought psychoanalytic theory into consumer research. Controversial, but he opened the door to the insight that people often can’t tell you why they buy. It’s an insight the field has validated over and over since.
5. 1960s-70s – The Computer Age
   Mainframes enabled multivariate analysis, segmentation modeling, and syndicated scanner data. Research went from hand tabulation to true data science.
6. 1971-2000s – Conjoint and Discrete Choice Modeling
   Green and Rao introduced conjoint in 1971. McFadden provided the theoretical framework, later recognized with a Nobel Prize in 2000. Bayesian estimation made individual-level modeling practical. The field’s most important methodological innovations.
7. 1979 Onward – Behavioral Economics
   Kahneman and Tversky proved that real human decision-making systematically departs from rational choice theory. Anchoring, framing, loss aversion: these reshaped how we design instruments and interpret results.
8. 1990s-2000s – The Internet Revolution
   Online data collection compressed timelines and costs by orders of magnitude. But it also industrialized non-probability sampling. The greatest acceleration and the deepest methodological crisis arrived together.
9. 1997-Present – The Response Rate Collapse
   Telephone response rates dropped from 36% to under 6%, and eventually to 0.4%. That structural break challenged the practical viability of probability sampling and forced the field to adopt methods it had spent decades resisting.
10. 2023-Present – Generative AI and Synthetic Respondents
    LLMs are now simulating research participants at scale. Adoption is surging—72% of insights professionals are evaluating GenAI tools. The promise is real for pre-testing and structured tasks. But for decision-grade market research, the methods are still unproven.

The Accountability Gap in Market Research

Here’s what I keep coming back to.

Election polling has always had a built-in accountability mechanism. The election happens. The votes get counted. You find out very publicly whether your methods held up. Nate Silver’s grade on our polling at UConn lives on in the internet archives.

Most market research has never had that same discipline.

And now, as AI promises to make everything faster and cheaper, the temptation to skip validation is greater than ever.

AI Powered Research Needs Outcome Measures

We need to build outcome measures into AI-powered research the way elections are built into polling.

Did the product sell?

Did the concept succeed?

Did customers behave the way the model predicted?

If we can’t answer those questions, we’re not doing research. We’re generating content that looks like research.

No excuses on intervening events, either.

The people who taught me this craft understood that being fast doesn’t matter if you’re wrong. That lesson is more urgent now than it’s ever been.

One last note: the benefits of the methods above haven’t come close to realizing their potential.

We are going to try to change that. Rapidly.