With contributions from Bruce Isaacson, DBA, Senior Managing Director, Litigation Surveys & Consumer Science, and Molly Spencer, PhD, Research Director, Litigation Surveys & Consumer Science
Today's marketplace is a crowded space, with products and services facing intense competition for customer attention and spending. Shelves are filled with new and improved products, consumer attention is fragmented across channels and media, the internet and mobile devices present a world of comparison-shopping opportunities, and a difficult economy creates price pressure on consumers and manufacturers.
Faced with this difficult and complex environment, products and services must communicate their benefits to consumers quickly, clearly, and effectively. The statements on packages, ads, and point-of-sale materials that communicate these benefits are called “claims.” As consumers, we encounter a multitude of claims every day, telling us which product tastes the best, which mid-size luxury sedan handles better than the competition, which laundry detergent cleans as well as the leading brand, which retailer has the lowest prices—the list is endless.
Why Do Claims Matter?
Claims are the foundation of marketplace communications. They convey the benefits of a product or service, presenting customers with a value proposition that states why they should buy that particular product. While claims can help increase sales, they can also be problematic.
Companies can face prohibitive legal fees and penalties if their claims do not adhere to the rules and regulations that are intended to protect consumers. This compliance is a serious matter that is enforced by a variety of authorities and venues.
For example, wireless companies have spent millions of dollars in litigation over how to communicate claims about who has the best coverage. Soup companies have been in disputes about how to communicate the salt content in soup. Sunscreen companies have been subject to numerous rules and regulations about the use of words such as “waterproof” and “sunblock.”
What Is Claim Substantiation?
Claim substantiation is the process of making sure that supporting evidence is in place for statements made in advertisements, point-of-sale materials, sales brochures, product packaging, etc. These claims may attract the attention of competitors who might challenge a claim in federal court or the National Advertising Division of the Better Business Bureau (NAD). Alternatively, there are a host of government agencies that protect consumers, typically by requiring that companies that make claims have a reasonable basis for asserting that the claims, in fact, are true.
Objective vs. Subjective Claims
There are two different types of claims. Some claims can be supported with scientific data. This type of data is often gathered by an objective measurement tool, which determines, for example, the number of seconds it takes for an automobile to accelerate from zero to sixty, or whether a toothpaste actually fights cavities.
The other type of claim involves subjective data, which is typically gathered from customer perception surveys. For example, it can be determined which automobile is preferred by executives, which toothpaste tastes the most minty, or whether a drugstore shampoo leaves hair as manageable as a shampoo sold in expensive salons. Substantiation cannot rely on letters from satisfied customers, reports from salespeople in the field, or other types of testimonials or anecdotes. While these items can provide support, they are not a substitute for data gathered according to accepted industry and legal standards.
How to Gather Claim Substantiation Evidence
This article explains how such evidence must be gathered for claim substantiation when consumer perceptions are involved. Drawing on our team’s extensive experience in research, surveys, and claim substantiation, we provide an overview of claim substantiation for sensory claims that includes governing authorities and standards, along with five multifaceted steps involved in claim substantiation.
Governing Authorities and Standards
A variety of agencies and authorities can potentially be involved in governing or mediating issues involving claims. Three of the most common and most important are the National Advertising Division (NAD), the court system, and government agencies.
The NAD is a forum operated by the Council of the Better Business Bureaus where advertisers can mediate differences over claims. When one company files a complaint with the NAD against another company, a voluntary process is started. The process is lower cost, faster, and more private than litigation. A company called into the forum can decline to participate, in which case the matter may be referred to other authorities.
A second forum for substantiation is the federal or state court system, where claims are often at the heart of matters involving legal issues such as false and deceptive advertising or trademark infringement. Unlike the NAD, the court process is not voluntary; it takes more time and is more expensive. For matters involving false advertising, the burden of proof for substantiation is made more complex by a two-step standard: the evidence must indicate that a claim has caused consumers to have a particular belief or attitude, and that the belief is “material” to marketplace activities such as purchase habits.
A third forum includes a host of government agencies, such as the Federal Trade Commission (FTC) or the Food and Drug Administration (FDA). The FTC has had a substantial impact on claim substantiation with a policy statement issued in 1984 called the “reasonable basis doctrine”.1 This doctrine states that:
- Marketers must have a basis for substantiation before making a claim.
- The evidence for the basis must be “reasonable” when judged by factors such as the type of claim, the risk of a false claim, the cost of developing substantiation, generally accepted principles for claim substantiation, and other factors.
The “reasonable basis doctrine” has similar counterparts in the expectations that exist in other venues. In all venues, there is an expectation that a party making a claim has evidence to support it. For claims involving sensory perceptions, which are the focus of this paper, the evidence is often in the form of consumer surveys. Although the standards, expectations, and processes can vary from venue to venue, and even across geographies within a venue, all these forums have leeway in determining how much weight to give a survey.
For example, the NAD may decide that a survey is flawed and take the authority to “step into the shoes of the consumer” and use its own experienced judgment to determine what messages are conveyed to a reasonable viewer of an advertisement. Similarly, a court of law can accept or reject a survey based on how it was done and can assign the survey greater or lesser weight.
Critical Research Steps and Core Principles
Figure 1 illustrates the five steps for substantiating claims involving consumer perceptions. Because claim substantiation is complex and somewhat unpredictable, we have also provided a core principle as a guide for each step described below.
Step 1: Decide which claims to test.
Core Principle 1: Different claims provide different levels of support for product strategy and different burdens for substantiation.
The first step in the claim substantiation process is to decide which claims to test. This involves evaluating the product to decide what claims to make. The core principle guiding these selections is that different claims create varying levels of support for product strategy and different burdens for substantiation.
A claim applied to a product or service should reference an attribute or quality that 1) the product or service can deliver, 2) differentiates that product or service in the marketplace, and 3) supports and reinforces product strategy and positioning. For example, if we were substantiating claims for an automobile, it would only make sense to make claims referencing performance for a car that had exceptional performance.
What may seem like minor changes in the wording of a claim can cause large differences in the burden of substantiation. To describe different kinds of claims, let us use a fictional pre-packaged cake mix: Ready-Bake. Imagine that the marketing team at Ready-Bake is thinking about how to position their cake mix product in the marketplace and is considering different wording for text on the package and advertising. As Table 1 shows, there are a variety of claims they might consider, each with different implications for developing evidence for substantiation.2
The easiest claim is a non-comparative claim, where there is no claim against another product. The most difficult claim is a superlative claim, which states that our product is better than all other competitive products.
This principle holds whether we are looking at the cake overall or specific product characteristics. Think again about our Ready-Bake cake mix and consider some possible claims that our hypothetical marketers are considering to describe the rich chocolate flavor of the cake. Statements might appear similar but have very different implications for the burden of proof required in substantiation (see Table 2).
As mentioned previously, the phrasing of a given claim affects the burden for substantiation.
Step 2: Select the products or services to test.
Core Principle 2: Always match the claim with the marketplace as much as reasonably possible.
The next step in the process is to select the products or services to test. Sometimes this is not an easy task, particularly in categories with a range of products and product forms. The second core principle states that, “as much as reasonably possible,” the survey should match the conditions under which real customers encounter the product in the marketplace. The quoted phrase is important for matching the marketplace, because a survey is always an abstracted environment where respondents encounter items (like products, packages, or ads) under conditions that are carefully controlled to allow measurements.
Core Principle 2 has important implications for many aspects of substantiation research, such as:
- Product forms should match the claim.
- All product forms tested should match the claim. For example, our Ready-Bake cake mix would be tested against other pre-packaged mixes rather than pre-baked cakes.
- Other product attributes should match the marketplace and other products tested.
- Other product attributes should match the marketplace as well, and they should be as similar as possible for each product tested. For example, attributes such as package size, package type, and means of product usage (such as pump versus aerosol for hair spray) should match the typical conditions under which consumers referenced in the claim encounter the product.
- Product formulas and versions should be current.
- The survey should test the product formula or version that currently exists in the marketplace, rather than an obsolete or older version.
- Product conditions should be similar to the conditions that consumers encounter.
- The survey should test a product that is as similar as possible to the product that consumers encounter in the marketplace. For our Ready-Bake product, which is sold in supermarkets, it is best to purchase the mix for testing from a supermarket so that the product tested is identical to the product a consumer could purchase at retail. If we were to obtain the Ready-Bake product for our test from the factory, we might test a mix that is fresher than the mix that consumers could buy.
- Markets and geography should match the footprint of the claim.
- Substantiation is market-specific. If a product is sold nationally, then the claims testing should be done nationally. If a product is only sold in certain regions, the testing should be done in those regions. For products sold in multiple but distinct markets or countries, research must be done in each distinct market or country.
An important part of Step 2 is deciding who to survey. When conducting research to substantiate claims, one typically interviews respondents who represent the universe of customers and potential customers of the goods in question. The research sample should match the category on factors such as age, gender, and income. The researcher should be careful not to omit key segments. For example, in product categories where children are part of the purchase process, it may make sense to interview parents and children together.
Beyond matching demographics, the research should also screen for category involvement. Category involvement may not be only at the category level, but at the level of the specific product, model, or flavor in question. This is particularly true for usage or taste tests; it would not be wise to test chocolate ice cream among consumers who do not like chocolate, or to test a convertible among those who only drive SUVs. Category involvement can also involve making sure that respondents have some category experience, but not too much. The typical customer may be different than a company’s most frequent users or most expert users.
Step 3: Conduct the claim substantiation interview.
Core Principle 3: Use a research design appropriate to the claim tested.
The next step in claim substantiation is to conduct the survey. In this step, consumer data measuring perceptions of the product is gathered through a survey. Typically, in a claim substantiation interview, the respondent answers a series of questions about the product, either after trying the product or based on their recollections of the product.
The core principle for this step is that the protocol for the research must be appropriate to the claims tested. Three important aspects of research design include research mode, research protocol, and questionnaire design. Two modes of research are most commonly used for claims research: in-person and in-home use tests.
When conducting a claims study, a researcher often selects between a home-use test that is conducted at the respondent’s home, or central location testing, which is typically conducted in a research facility in a shopping mall or focus group facility. Table 3 describes the two modes in more detail and contrasts the types of considerations that each involves. As with other aspects of substantiation research, the choice between a home-use test and a central location test will depend upon the circumstances. For example, our Ready-Bake cake mix should most likely be tested in-home, where the respondent can prepare it according to the package directions, while a packaged pre-baked cake may be more appropriate to test at a central location facility.
The second aspect of research design is the order in which consumers are exposed to the products, ads, or packages tested. Monadic testing is a common protocol where each brand is tested on its own, and then the averages are compared across brands. In a monadic design, respondents evaluate one product in one interview, which provides a very clean read on that particular product, but is relatively expensive and time-consuming.
By contrast, in sequential monadic testing, respondents evaluate several products one at a time in a sequential fashion and complete a survey after each product is used. After all products have been tried, they may also compare and contrast the products tested. This technique is less costly and less time-consuming than a pure monadic design. Also, the sequential monadic design can provide direct comparisons of one product versus another. From a statistical standpoint, the comparison ratings are particularly powerful because they compare products in a head-to-head manner, providing lots of statistical power for a given sample size. However, the research must be carefully implemented to manage an interview with two products rather than one.
The third aspect of research design is questionnaire design. Questionnaires for claim substantiation are different than questionnaires for other types of research. When designing a study to track brand image or customer satisfaction, a researcher may include additional questions that are peripherally helpful to the core questions of interest. For example, a brand tracking study may also include questions about customer buying behavior.
However, the claims questionnaire must focus only on the claims of interest and nothing more. Authorities who evaluate claims research use it to decide whether the data provided substantiates the statements made. There is an inherent assumption in that process that statements measured later in a long questionnaire could potentially be biased by statements measured earlier. As a result, claims questionnaires must be short and specific to the attributes relevant to the claims being made.
Step 4: Analyze the data.
Core Principle 4: Each claim stands on its own merits.
The fourth step in claims substantiation is to analyze the data that comes from the survey. This step consists of quantitative analysis, with each claim typically standing or falling on its own merits. A claims substantiation questionnaire may test a number of attributes, and marketers can use the results that support their product, even if other results do not. For example, if we test our Ready-Bake cake against competitors for moistness, chocolate taste, and velvety texture, and it is preferred on chocolate taste only, we can still use the results to support a claim based purely on chocolate taste.
We can state claims that closely match the data even when the tested product did not “win” the test in the traditional sense of the word. For example, if consumers in our test compare Ready-Bake to homemade cakes, and one-third of consumers prefer Ready-Bake, the research can substantiate a claim that “One in three consumers prefers Ready-Bake to homemade.” The statement is true and can be used. Whether it is helpful is another matter.
Statistically, the analysis is typically done at the 95% level of confidence. This is a statistical calculation that reflects the percentage of the time, if we repeated the survey, that the numbers we measured would reflect the actual underlying data in the population. It is worth noting that the analysis of claims data can involve a number of claims-specific tests. For example, if we are measuring preference for Ready-Bake over a competitive brand, imagine that Ready-Bake is preferred 55% of the time, and the competitor is preferred 45%. The analysis of the claims data would include a statistical calculation to make sure that this preference level is statistically different from a 50/50 tie.
Step 5: Select the claim to use in the marketplace.
Core Principle 5: Involve multiple functions, probably including marketing, consumer insights, and legal.
The final step in the process of claims substantiation is to select the claim to use. This can be a challenging task. The data from a claims substantiation study can be conflicting. Some claims may be easier to substantiate, others less easy. Some claims may be more likely to invoke competitive reactions and lawsuits, others less likely.
Given these considerations, it is best to select claims using a multi-functional team with representatives from marketing, consumer insights/research, and legal. The different perspectives are critical to the task of ensuring that a claim is effective, able to be substantiated, and consistent with product strategy. Words like “contains,” “free,” or “low fat” may have very different meanings to a marketer than to a lawyer. Involving multiple perspectives helps yield a claim that is both powerful and substantiable.
What to Consider Before Conducting Research
We have described the five key steps of claims substantiation research and core principles to help guide each step. Before closing, we would also add three things to be cognizant of when planning claim substantiation activities.
Substantiate First
The first consideration is to substantiate first, then claim second. In other words, substantiation should be in place before the claim is made. This typically requires a professionally researched survey conducted by a qualified expert.
The Unpredictability of Substantiation Standards
A second consideration is that substantiation can be difficult and unpredictable. The standards for substantiation can be high. Authorities such as judges, juries, and mediators must evaluate research presented in support of or in opposition to a claim, and surveys conducted by experienced researchers are often criticized by smart lawyers, well-armed opposing experts, and inquiring judges. Standards to evaluate surveys can vary from circuit to circuit, from judge to judge, from venue to venue, and from year to year.
Reasonableness Wins the Day
This leads to the third consideration. Good research, corresponding to the principles described in this article, is more likely to hold up and carry the day. In this changing and challenging environment, reasonable and well-conducted research has the best chance of prevailing. As the American Society for Testing and Materials (ASTM) guide states, “The keyword is ‘reasonable.’ If a particular aspect of a test is not reasonable for a specific application, it should not be used.”3
References
1See “FTC Policy Statement Regarding Advertising Substantiation”, 104 F. T. C 648, 839 (1984).
2For more details, see the Standard Guide for Sensory Claim Substantiation, from ASTM International, West Conshohocken, PA, 2008.
3See the ASTM guide, previously cited, page 1.
