Strategic Benefits of Consumer Surveys in Complex Litigation | Episode 69

Hello, and welcome to the IMS Insights Podcast. I’m your host, Adam Bloomberg. Today we’re speaking with two data experts from IMS | Analytic Focus about using consumer surveys in complex litigation. Senior Advisor Dr. Charles Cowan and Associate Director of Statistical Consulting Dr. Gabriela Orsak will explore which cases benefit most from surveys and provide compelling examples that reflect the power of data in the courtroom.

(If you are listening only and would like to watch the video version, please visit our YouTube channel.)

Adam Bloomberg:

Consumer surveys used in litigation can drive your overall case strategy. Understanding the thoughts, beliefs, and opinions of a person, let alone a group of people, is an age-old challenge for the courts. Understanding these attributes with confidence helps our clients' storytelling be cogent and believable, giving all litigators an advantage and a means for jurors to render fair judgments.

Joining me today are some veterans from the field: Dr. Charles Cowan, who holds a PhD in mathematical statistics, and Dr. Gabriela Orsak, who holds a PhD in experimental psychology. Thank you both for being here.

Charles Cowan, PhD:

Thank you, Adam. And thanks very much for having us. I'm mostly informal, so please call me Chuck.

Gabriela Orsak, PhD:

Same. Thank you so much for having me here as well.

Adam Bloomberg:

Alright Chuck, can you provide some details on your education and maybe some relevant experience in designing these surveys?

Charles Cowan, PhD:

Sure. So, you already mentioned that I have a PhD in mathematical statistics, but I got there by going through programs in economics and being introduced to statistics as part of my graduate programs and learned and I enjoyed statistics, more than economics, but economics is still an excellent background for what we do. My particular areas of expertise are generally statistics, but in particular, sampling theory and the design of research. I was the Chief Statistician for two different federal agencies and also the Chief of Survey Design for the Census Bureau. I have 25-plus years in litigation, and I've co-authored two books on survey methods, and I've got numerous academic articles that I've published. I've testified in over 200 cases.

Adam Bloomberg:

Gabriela, what about your experience?

Gabriela Orsak, PhD:

I have a degree in experimental psychology with an emphasis in health psychology. And I have over ten years of experience in research methods and survey design. As a result of me working in hospital settings, I'm very well aware that not all types of populations are very easy to recruit or to do surveys or other types of research on. I've published in numerous academic journals and, you know, when I was in academia, as also currently, and I just right now I primarily focus on designing surveys for a broad range of cases in litigation.

Adam Bloomberg:

Great. Now, let's start at a macro level on the topic of surveys. What kinds of cases need a consumer survey?

Gabriela Orsak, PhD:

Surveys in litigation require very rigorous design, unbiased sampling, and expert analysis. They serve as powerful tools to inform legal strategy and provide evidence in court. Therefore, they play a crucial role in litigation when you need to gather insights from a group of individuals. When you need to know that, you need to conduct a survey. So, for example, there are intellectual property disputes. So, surveys can help establish consumer confusion, likelihood of confusion, the dilution related to trademarks, when you think about deceptive sales practices and false advertising, surveys can assess whether consumers are actually misled by the advertising claims or the sales practices, and not whether we just think that this is the case. There's also substantiation of a claim. When a claim is made, you know, "Our product is 30% more effective." Well, surveys can actually provide evidence to support or refute that claim, the same as to assess the materiality of claims as well.

Charles Cowan, PhD:

I'd like to jump in and just mention that surveys are also used in the calculation of damages. So, you have the determination of liability on the one hand or a determination of some set of facts, but then, in addition, we use surveys simultaneously by asking questions about price premiums and other information that's important. So, you might look at how much more somebody would be willing to pay for one product versus another. You might look at how much more a consumer is willing to pay for a product with a specific characteristic relative to a product that doesn't have that characteristic.

Another way of using surveys or damages is estimating the diversion of sales. So if there is a false claim being made, is that diverting sales from somebody who's also the trademark or has particular trade to us? And then finally, one of the biggest problems litigators face is determining what the class size is for the class that they're investigating because, frequently, there's no list of the people who might be affected. There's no available source of information, but a survey can be used to collect information about the general population and the subset of that population that fits the definition or the class.

Adam Bloomberg:

Chuck, you mentioned class action lawsuits. Gabriela, can you drill down on that a little bit and give some examples of how surveys are used a bit more in these cases?

Gabriela Orsak, PhD:

Sure. A survey can be used in a variety of examples. So, in class action lawsuits, the size of the class needs to be established. Simply knowing that these people exist is not enough. We often use surveys to establish who, for example, purchased a certain product during the class period. If you have no other record of the purchase that exists, for example, you need a survey in order to determine that. Another example where a survey may be warranted would be if an attorney would like to file a motion to remand. That is if they wanted to move a case from federal to state or the other way around.

Another example is if so let's consider a smartwatch and the smartwatch has a limited battery life, and with limited battery life, it renders the watch useless. You know, that is what the complaint alleges. Well, the importance of battery life for a smartwatch needs to be established. So in other words, you need to establish what value a consumer would place on the battery life. Let's say four hours, eight hours, 12, or 24 or more hours, if they care at all. In addition to all the other features offered, this is where that conjoint analysis that we mentioned earlier comes into play where we could establish this, and that can later be used to calculate damages as well, but these are just a few examples that you could use.

Adam Bloomberg:

What about intellectual property, false advertising, or deceptive sales practice cases?

Gabriela Orsak, PhD:

In trademark and trade dress cases, there's a claim that consumers are confused between two separate marks and may believe that they are the same or connected, associated, or affiliated with each other. So many times, there's two separate companies that may have very similar names, and this can cause a number of customers to go to the competitor store instead of yours.

So a survey is good here because we get to establish whether that's actually, might be the case. In false advertising and deceptive sales, an allegation is made that, for example, an advertisement is deceptive, but you really do need to test whether consumers actually believe them deceptive advertisement or not. In a similar way, you may need to substantiate a claim. So those claims, let's say four out of five dentists really recommend this product. The only way to know is to ask dentists what they actually think. You can't just throw it out there.

Adam Bloomberg:

Chuck, any other examples?

Charles Cowan, PhD:

So I have a really good example of a survey that we conducted that was very helpful. And, a large, consumer deception case. In fact, this case wound up being the largest amount of money ever awarded to plaintiffs in a consumer deception trademark dispute. The case is Monster v. Bang. Those are two energy drinks. Monster has its own presence in the market. Bang was doing some false advertising to try to build up its share of the market relative to Monster and other retailers of energy drinks, but a lot of the claims being made by Bang were false. They advertised about the contents of Bang and how beneficial they were to you. And it turned out that the claims regarding contents and the claims regarding the benefits of Bang were both false.

So, the first thing we did in the survey was ask questions about what people believed. We then presented exactly the statements that were made by Bang. And during the trial, the attorneys from Monster demonstrated that these claims were false with respect to the particular supplements available within Bang. But on top of that, some of the claims from Bang were snake oil claims. They claimed that Bang cured Alzheimer's, claimed that your hair would grow. Other claims that just were truly not correct and very misleading. And yet, the purchasers of Bang believe these claims, and they believe these claims much more frequently than purchasers of Monster.

And so, because of this, bang had carved out its own subset of consumers, who believed all of these false claims. So when you asked the consumers, number one, “Did you believe these claims?” Bang consumers mostly said yes. On top of that, though, when you then explained that these claims were false, that these supplements that were claimed to be in Bang didn't exist, the next set of questions was “If these claims weren't true, would you continue to buy Bang or would you buy a different energy drink?” And then if they said a different energy drink, we said, okay, well, “Would you buy a Monster or which drink would you buy?” And we determined the percentage of people who were bang drinkers who would shift into Monster. So those are diverted sales. And that goes to something we discussed earlier, which had to do with the economic impact of these false claims and the calculation and damages.

Adam Bloomberg:

So, what was your approach to designing and choosing which questions to include in the survey? And what benefits did this bring to the trial team?

Charles Cowan, PhD:

So in our work, what we did was we determined, first, the proportion of people who would move from being to Monster, to increase Monster sales or to bring Monster sales to where they should have been. And then on top of that, we also asked questions about how much less would you be willing to pay for Bang if it didn't contain these ingredients or it wasn't providing the benefits that they claimed. And so there are two forms of damages there. One is diverted sales, the other is unjust enrichment.

You're able, from the survey, to then calculate both of those. So the attorneys representing Monster, got two benefits from doing the survey. One was establishing liability. People actually believe these claims. And the second benefit was being able to use the exact same survey to then, determine all the inputs to the damages calculations. So that's a great example. And it was a great win for one of our clients because of the fact that, the jury who heard this in the trial, ruled that with that kind of evidence, they could only conclude that Bang was misrepresenting its product. The benefit for Monster was, not only did they get the $293 million from diverted sales and unjust enrichment, but they also got a ruling by the court for Bang to stop doing the false advertising.

Adam Bloomberg:

Gabriela, any other scenarios where these surveys can be valuable?

Gabriela Orsak, PhD:

Okay, how about cases involving unauthorized resellers? And these are third party sellers on platforms such as Amazon. These definitely benefit from surveys, so consumers may unknowingly purchase products from these resellers, and then they miss out on the warranties or guarantees that are offered by authorized sellers.

And the survey can assess the consumer can, awareness, the satisfaction and the potential harm that's caused by unauthorized resellers because they might not be aware that they are unauthorized resellers. And I just like to note that, you know, a well-designed survey that's conducted with the representative sample provides an objective data that can, you know, strengthen the legal arguments which we are able to do in those kinds of cases, because we're able to do a lot with them.

Adam Bloomberg:

What sorts of common practices do you utilize when you're creating these surveys?

Gabriela Orsak, PhD:

Sure. When appropriate, such as what these unauthorized resellers would try to incorporate a control condition, and then we structure the questions so that they are not biased, so that they're not leading in any way. We pay attention to question order, for example, so as not to influence respondents. We try to recreate realistic shopping experiences in our survey.

So in the case of the unauthorized resellers on Amazon, you would like for your survey to look exactly like the page does on Amazon. And what that involves is being able to click numerous places that you would click. Maybe you want to click the reviews, maybe you would like to see additional images, you would want to zoom in to the images, you want to read more, and just what people do. So we're able to create that realistic experience in our surveys. Yeah. We do everything to make it look like it does in real life for example.

Adam Bloomberg:

Chuck, back to you on this. What are the typical questions that concern clients?

Charles Cowan, PhD:

So there's two or three questions that we hear frequently. The number one question has to do with feasibility. Clients don't know everything that we can do within a survey. And so, for example, there are numerous sources of lists of people that we can reach out to and even people, you know, with a specialized background, like medical doctors or consumers who are above a certain age. So, by using these external sources, we greatly cut down on cost and the time to do this survey because we can get a very good list of people who would qualify to do the survey.

But over and above that, the other thing about using these types of lists and these outside sources is that it lends a greater reliability to the survey. So as an example, the bureau, the census, the Bureau of Labor Statistics, a number of federal agencies publish tons of data about the distribution of people throughout the United States. So you can validate the way your sample looks by comparing it back to very specialized census data. There's also data that you can get from property tax appraisers in every county in the United States. Electrical companies, water companies, they all have lists of consumers, so, again, you can use these outside sources to validate who it is you're talking to and be sure that the list that you're using is actually pretty much up to date, so that you're sure you're addressing the right population.

Those feasibility issues are very important because we want to sound credible. And by being able to go to court and say, not only did we conduct the survey using the proper procedures, we can also compare the results of the survey to outside sources, like the Bureau of the census, so that the Trier of fact, whether it's the judge or the jury, feels comfortable that not only did we do the right thing, but it's benchmarked against something else that is very useful.

Adam Bloomberg:

Very interesting. Chuck, what other questions do you receive from clients regarding surveys?

Charles Cowan, PhD:

We also get questions about money. What's it going to cost? We try to provide different methodologies, so because some methods are more expensive than others. We also talk about the reliability of the outcomes. Reliability, meaning it's a survey outcome, how much variation is there around the outcome? And then finally, we talk about timeline. Sometimes we get called six months in advance, so when the report is due, we have plenty of time to develop things and make sure that everybody is happy. Sometimes we get called, you know, 20 days, not six months before the report is due. And that's kind of the end point where we can actually get into the field and do a survey. Less than 20 days is kind of impossible, but we can work with the client's deadline and provide something that’s useful. The more advanced time we've got, the better off everybody is. But those are the types of questions we get because feasibility has to do, not only with whether or not we can physically do it, but also can we do it within a particular timeline and within a particular budget. Those are the types of questions we get most frequently.

Adam Bloomberg:

Gabriela, can you explain your approach in designing surveys?

Gabriela Orsak, PhD:

I'll try. Each case is unique. Depending on what suits the project best, we can do telephone, online or mail surveys. And many times, we even mix those methods. It all depends on the type of project that we are working on and what we, what the challenge is. We select the appropriate sampling methodology always. And we get to designing a survey. So many surveys warrant replication of real-world shopping experiences, like I mentioned earlier. Other projects require us to conduct focus groups, because we need to learn more about a topic before we jump into designing a questionnaire. We also conduct pretests before launching a final survey, just to make sure that we pick up on some things that could be changed before the actual survey is launched.

Questions are worked to be as free of bias as possible and whether that's through question wording or question order. We randomize questions, so some appear at different points to different people who answer the question, so it doesn't bias you in that way – what you see first or what you see last. And then after all of this, we test that the samples are representative of the population that we're trying to study, and then we analyze the data and we interpret the results using the appropriate statistical techniques.

Adam Bloomberg:

Anything else you'd like to add?

Gabriela Orsak, PhD:

Yeah. I would like to mention that we also receive many of these types of reports involving sampling and surveys and are happy to do a rebuttal on any of these topics.

Adam Bloomberg:

Chuck, what about you. Can you share some of your final thoughts?

Charles Cowan, PhD:

Well, I'd like to expand on the last point that Gabi made. We have a great deal of expertise and how surveys are properly conducted. But besides that, we also have a lot of expertise and the issues that come up in surveys, because you do the best you can for designing, but there are always some issues that may come up and there also may be some confusion about who should be surveyed, what the correct population is, what the correct time period is. All of those affect the quality of the survey that is being presented. So, we, in addition to doing our own work and own design, are frequently called on to do a review or a critique of somebody else's survey. And there are times that we see, you know, fairly good surveys being conducted. And there's sometimes when they're not so good, or they're horrific. But, you know, we, we see the whole range.

So, the skill set that we bring is not just designing and conducting a survey. It's also looking at a survey or a statistical analysis and determining, does this meet proper, professional standards? And both of us are deeply steeped in the standards of our profession. We have different backgrounds in terms of statistics and experimental psychology, but those are two professions that stress the quality and methodologies applied within surveys. So, yeah, I agree with everything that Gabi has said. And then I, you know, but it's two sides of the coin. Sometimes we're doing the survey and sometimes we're looking at somebody else's work.

Adam Bloomberg:

Well, thank you both for joining me today for this meaningful discussion about consumer surveys. If you're watching us on YouTube, please like and subscribe to our channel. If you have any other questions for Dr. Orsak or Dr. Cowan, please reach out to their addresses linked in the description below.

Thank you to Chuck and Gabriela for speaking with us today, and a special thanks to our listeners. Please join us next time and subscribe to the IMS Insights Podcast.

IMS has delivered strategic litigation consulting and expert witness services to leading global law firms and Fortune 500 companies for more than 30 years, in more than 45,000 cases. IMS consultants become an extension of your legal team from pre-suit investigation services to discovery and then on to arbitration and trial. Learn more at imslegal.com.