Survey evidence plays a significant role in litigation, particularly in cases involving consumer confusion, trademark disputes, and similar claims.
Every survey, no matter how well designed, is subject to influences beyond the questions themselves, and one unavoidable influence is human nature.
IMS Legal Strategies has highlighted how noise within surveys can seriously undermine their value as legal evidence if left unaddressed.
This noise includes answers that reflect respondents’ preexisting views, guessing, inattention, a tendency to say yes to every question, or some other form of noise.
Controls are designed to strip out these distortions, ensuring that survey results can be reliably attributed to the specific issue under examination, such as consumer confusion.
Courts have shown a willingness to reject surveys that lack proper controls, or to admit them while affording them very little evidential weight.
One legal commentator goes so far as to say that a survey without a control is “fatally flawed,” because such a survey “cannot satisfy the reliability mandate.”
A control may therefore be vital to the success of a litigation survey, particularly where the stakes are high and opposing counsel is ready to challenge methodology.
Controls become especially necessary when a survey shows significantly high measures for the issue of interest, though they may not always be required when a survey is intended to demonstrate significantly low measures.
Both external and internal controls serve to detect the types of survey noise that can corrupt results, capturing inattention, preexisting biases, and tendencies among some respondents to agree with almost everything.
Responses gathered through internal controls cannot be considered genuine answers, meaning they function purely as a measurement of background noise present across the survey.
Because all other elements of the survey remain the same across test and control groups, researchers can reasonably assume the noise rate is similar in both sets of measures.
This consistency is what allows researchers to isolate and remove the effect of noise, leaving behind results that more accurately reflect the true measure of the issue being examined.
Legal teams commissioning surveys for use in litigation should treat the inclusion of well-designed controls not as optional, but as a foundational requirement for producing credible and admissible evidence.