Opinion
CASE NO. 07 C 4718.
April 26, 2010
MEMORANDUM OPINION AND ORDER
This matter is before the Court on Defendants' motion [52] to exclude the expert testimony of Plaintiff's witness, Thomas J. Callahan, pursuant to Federal Rule of Civil Procedure 702. For the reasons explained below, Defendants' motion [52] is respectfully denied without prejudice.
I. Background
A. The Parties
Plaintiff Bobak Sausage Company ("BSC") is an Illinois corporation that manufactures, markets, and sells a variety of wholesale and retail food products. BSC owns three federally registered trademarks, but alleges infringement only of Bobak's word mark ("the Mark"). BSC provides retail grocery, deli, restaurant, and catering services under the marks, and has its principal place of business in Chicago. BSC and its predecessors have used the Mark in commerce continually since 1967, and the Mark has been federally registered by BSC on the Principal Register since 2004.
Defendant A J Seven Bridges, Inc. ("A J") is an Illinois corporation that provides banquet hall, conference center, and food catering services at its location at 6440 Double Eagle Drive in Woodridge, Illinois, under the mark "Bobak's Signature Events (and Conference Center at Seven Bridges)". Defendants (and siblings) John Bobak and Anna Zalinski operate the conference center. A J's events consist of approximately forty-five percent corporate functions and fifty-five percent family social events such as weddings and anniversaries. A J serves steak, chicken, and seafood dishes, and, prior to the commencement of this lawsuit, occasionally served BSC sausage as an appetizer. Most of A J's patrons either are customers who patronized the business before A J purchased it or are referrals from those patrons. However, A J also advertises through its website and in local wedding materials.
Stan Bobak, president of BSC, is a cousin of Defendants; in fact, their fathers are brothers and their mothers are sisters.
B. The Lawsuit
Early in 2005, BSC orally granted A J a limited license to use the Mark as part of its d/b/a "Bobak's Signature Events." The license was terminable at will and was conditioned upon A J's execution of a formal written trademark license agreement. Since at least April of 2005, A J has used the registered trademark "Bobak's" as part of its trade name for banquet and catering services. BSC alleges that since it granted the oral license in January 2005, it has repeatedly (albeit unsuccessfully) demanded that A J execute a formal written license agreement. In October 2006, BSC provided A J with a draft formal trademark license agreement with BSC, but A J did not execute that agreement. On July 10, 2007, Plaintiff sent A J a formal notice of termination of the alleged license agreement. Despite that notice and repeated demands to cease and desist, A J continues to use the "Bobak's" name.
C. The Survey
BSC hired Amplitude Research to develop and analyze a trademark confusion survey. Thomas J. Callahan is a senior consultant for Amplitude Research. Callahan has never developed a trademark confusion survey, nor has he ever served as an expert witness. However, Callahan has designed more than 100 academic, governmental, and commercial surveys. Amplitude, in turn, hired Communications Center, Inc. ("CCI") to conduct the actual survey.
The survey consisted of eight questions that were asked of 360 participants. In total, eight thousand calls were made. Two questions were targeted toward understanding the participants' familiarity with the companies and six questions measured the participants' perceptions concerning the products. The participants were selected using a random digit dial sample drawn from the population of the Chicago Metropolitan Statistical Area and reflect census data quotas. The surveyors only contacted households, not businesses.
The survey did not use threshold questions, such as "Are you in the market to buy sausage and rent a banquet facility?" in order to locate and utilize potential purchasers. Additionally, Callahan did not use a third product as a "control." According to Callahan, his controls were the "familiarity" questions and statistical controls that would reveal random guessing. Because the survey was conducted over the phone, the participants did not see the trademarks in question. The marks referred to in the survey were "Bobak's brand food products" and "Bobak's Signature Events." The surveyors did not inform the participants what products and services each company provided.
The participants answered according to a scale (provided by the surveyor), ranging from strongly agree to strongly disagree. The surveyors asked follow-up questions to ascertain why the participants answered as they did. Verbatim responses to the questions were coded and tabulated by Amplitude. After analyzing the survey, Callahan concluded that 31%-43% of the participants "agree it is likely that Bobak's brand food products and Bobak's Signature Events have the same ownership, management, or products" and 7%-13% of the participants "agreed it is unlikely that the two businesses share ownership, management, or products."
II. Analysis
Federal Rule of Evidence 702 and the Supreme Court's decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), provide the legal framework for the admissibility of expert testimony. See U.S. v. Pansier, 576 F.3d 726, 737 (7th Cir. 2009). Rule 702 permits the admission of expert testimony if "scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue." Fed.R.Evid. 702. Rule 702 requires that the district court act as a "`gatekeeper' who determines whether proffered expert testimony is reliable and relevant before accepting a witness as an expert." Winters v. Fru-Con Inc., 498 F.3d 734, 741-42 (7th Cir. 2007) (quoting Autotech Tech. Ltd. P'ship v. Automationdirect.com, 471 F.3d 745, 749 (7th Cir. 2006)); see also Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 147-49 (1999); Daubert, 509 U.S. at 589; Jenkins v. Bartlett, 487 F.3d 482, 488-89 (7th Cir. 2007).
To determine reliability, "the court should consider the proposed expert's full range of experience and training, as well as the methodology used to arrive at a particular conclusion." Pansier, 576 F.3d at 737 (citing Smith v. Ford Motor Co., 215 F.3d 713, 718 (7th Cir. 2000)); see Ervin v. Johnson Johnson, Inc., 492 F.3d 901, 904 (7th Cir. 2007). Daubert lists a number of relevant considerations in evaluating an expert's reasoning and methodology — including testing, peer review, error rates, and acceptability in the relevant scientific community. Daubert at 593-94. "[T]he test of reliability is flexible," however, "and Daubert's list of specific factors neither necessarily nor exclusively applies to all experts or in every case." Kumho, 526 U.S. at 141 (internal quotation omitted). "Rather the law grants a district court the same broad latitude when it decides how to determine reliability as it enjoys in respect to its ultimate reliability determination." Id. at 142; see also Pansier, 576 F.3d at 737 (the Seventh Circuit "gives the [district] court great latitude in determining not only how to measure the reliability of the proposed expert testimony but also whether the testimony is, in fact, reliable.") (citing Jenkins, 487 F.3d at 489); Lewis, 561 F.3d at 704-05 ("the law grants the district court great discretion regarding the manner in which it conducts that [ Daubert] evaluation.").
In addition, in considering Defendants' motion, it is important to bear in mind the Seventh Circuit's teaching about the critical distinction between a jury trial and a bench trial with respect to the Rule 702 inquiry:
Where the gatekeeper and the factfinder are one and the same — that is, the judge — the need to make such decisions prior to hearing the testimony is lessened. See United States v. Brown, 415 F.3d 1257, 1268-69 (11th Cir. 2005). That is not to say that the scientific reliability requirement is lessened in such situations; the point is only that the court can hear the evidence and make its reliability determination during, rather than in advance of, trial. Thus, where the factfinder and the gatekeeper are the same, the court does not err in admitting the evidence subject to the ability later to exclude it or disregard it if it turns out not to meet the standard of reliability established by Rule 702.In re Salem, 465 F.3d 767, 777 (7th Cir. 2006); see also United States v. Brown, 415 F.3d 1257, 1269 (11th Cir. 2005) ("There is less need for the gatekeeper to keep the gate when the gatekeeper is keeping the gate only for himself"). Under this sensible approach, where there is no jury demand, and therefore the judge will be the trier of fact at trial, the Court may choose to (i) allow the presentation of borderline testimony, (ii) subject the testimony to the rigors of cross-examination, and (iii) decide later whether the testimony is entitled to some consideration or whether it should be excluded as irrelevant, unreliable, or both.
A. Qualifications
Thomas Callahan has a B.A. in Economics from the University of Missouri, an M.A. in Psychology from the University of Missouri, an M.B.A. from Michigan State University, and a Ph.D. in Business Administration, concentrating in Organizational Behavior and Strategic Management, from Michigan State University. He is an Associate Professor at the University of Michigan. As noted above, Callahan has never developed a trademark confusion survey, nor has he ever testified as an expert witness; however, he has designed more than 100 academic, governmental, and commercial surveys. In Defendants' initial brief, they appear to challenge Callahan's qualifications to offer expert testimony in this matter. In their reply brief, they do not address his qualifications, but rather focus on the reliability of the survey. In the interest of completeness, the Court briefly addresses Callahan's qualifications.
Federal Rule of Evidence 702 allows parties to introduce expert opinions if the expert is qualified by "knowledge, skill, experience, training, or education." Anyone who has relevant expertise enabling him to offer responsible opinion testimony helpful to judge or jury may qualify as an expert witness. See Tuf Racing Products, Inc. v. Am. Suzuki Motor Corp., 223 F.3d 585, 591 (7th Cir. 2000). In assessing an expert's qualifications, a court should consider the proposed expert's full range of experience and training. LG Electronics U.S.A., Inc. v. Whirlpool Corp., 661 F. Supp. 2d 940, 951 (N.D. Ill. 2009); Rust Env't Infrastructure, Inc. v. Teunissen, 131 F.3d 1210, 1218 (7th Cir. 1997). Experts can be qualified based on experience alone. Shari Seidman Diamond, Reference Guide on Survey Research, in Federal Judicial Center, REFERENCE MANUAL ON SCIENTIFIC EVIDENCE 238 (2d ed. 2000) ("FJC REFERENCE MANUAL"). However, in regard to surveys, education in psychology, marketing, and communication, or other related fields, may be pertinent. Id. At the very least, the survey expert must understand "survey methodology, including sampling, instrument design (questionnaire and interview construction), and statistical analysis." Id.
An expert witness should not be precluded from testifying simply because he has never testified as an expert witness. In fact, many courts have expressed more concern about expert witnesses who have too much — not too little — experience as witnesses in court. See Daubert v. Merrrell Dow Pharmaceutical, Inc., 43 F.3d 1311, 1317 (9th Cir. 1995); Tokio Marine Fire Ins. Co. v. Grove Mfg. Co., 958 F.2d 1169, 1174-1175 (1st Cir. 1992); Thomas J. Kline, Inc. v. Lorillard, Inc., 878 F.2d 791, 800 (4th Cir. 1989); In re Air Crash Disaster At New Orleans, La., 795 F.2d 1230, 1236 (5th Cir. 1986).
On balance, the Court concludes that Callahan has the "knowledge, skill, experience, training, or education" that Rule 702 mandates from experts. Callahan has written questionnaires and analyzed data since 1985. In addition, Callahan's curriculum vitae states that he has written surveys for more than twelve years, and Callahan testified that he designed more than 100 surveys in his career. Although Callahan could be qualified based on his experience alone, his academic career — a Master's Degree in Psychology and a Doctorate in Philosophy in Organizational Behavior and Strategic Management — enhances his qualifications. To be sure, Callahan has never created nor analyzed a trademark survey. Callahan has performed limited research regarding trademark surveys, which detracts from his ability to transfer his experience and education to professional testimony. Nevertheless, he is educated in relevant fields and his professional experience, deposition, and survey report reflect that he has sufficient understanding of survey methodology, instrument design, and statistical analysis to be qualified for the purpose of giving opinion testimony under Rule 702.
B. Reliability
To meet Rule 702 and Daubert's standard of reliability, a survey offered to establish the likelihood of consumer confusion must "have been fairly prepared and its results directed to the relevant issues." Weight Watchers Int'l, Inc. v. Stouffer Corp., 744 F. Supp. 1259, 1272 (S.D.N.Y. 1990) (citations omitted); see also Coors Brewing Co. v. Anheuser-Busch Co., Inc., 802 F. Supp. 965, 972 (S.D.N.Y. 1992) ("The evidentiary value of a survey's results rests upon the underlying objectivity of the survey itself."). "The criteria for the trustworthiness of survey evidence are that: (1) the `universe' was properly defined; (2) a representative sample of that universe was selected; (3) the questions to be asked of interviewees were framed in a clear, precise, and non-leading manner; (4) sound interview procedures were followed by competent interviewers who had no knowledge of the litigation or the purpose for which the survey was conducted; (5) the data gathered was accurately reported; (6) the data was analyzed in accordance with accepted statistical principles[;] and (7) objectivity of the entire process was assured." Weight Watchers Int'l, Inc., 744 F. Supp. at 1272 (collecting cases). Although these criteria generally address the weight that a fact finder should give the survey, a survey method that ignores these criteria may be of so little utility as to be rendered irrelevant, and thus inadmissible. See, e.g., Evory v. RJM Acquisitions Funding LLC, 505 F.3d 769, 776 (7th Cir. 2007) ("survey evidence in debt-collection as in trademark cases must comply with the principles of professional survey research; if it does not, it is not even admissible").
Consumer surveys frequently are used by litigants as a means of attempting to show likelihood of confusion in a trademark case. Simon Property Group, L.P. v. mySimon, Inc., 104 F. Supp. 2d 1033, 1038 (S.D. Ind. 2000). To be admissible, consumer survey results must be presented through expert witnesses. Id. at 1039. "No survey model is suitable for every case. At bottom, however, a survey to test likelihood of confusion must attempt to replicate the thought processes of consumers encountering the disputed mark or marks as they would in the marketplace." Id. at 1038; see also 5 MCCARTHY ON TRADEMARKS § 32:163 at 32-237 (4th ed. 1999) ("the closer the survey methods mirror the situation in which the ordinary person would encounter the trademark, the greater the evidentiary weight of the survey results"); Lindy Pen Co. v. Bic Pen Corp., 725 F.2d 1240, 1245-46 (9th Cir. 1984) (even where marks were identical when viewed in isolation, determination of likely confusion required consideration "in light of the way the marks are encountered in the marketplace and the circumstances surrounding the purchase of the pens," which sufficed to distinguish the two marks except in context of telephone solicitation, where such distinctions were not evident).
Trademark confusion measures whether a potential purchaser who views the junior mark would associate its products with the products of the senior mark. James Burrough Ltd. v. Sign of Beefeater, Inc., 540 F.2d 266, 275 (7th Cir. 1976). A trademark confusion survey "must attempt to replicate the thought processes of consumers encountering the disputed mark or marks as they would in the marketplace." Simon Prop. Group, 104 F. Supp. 2d at 1038. However, trademark confusion also can be measured by sound, meaning, or connotation. Planet Hollywood (Region IV), Inc. v. Hollywood Casino Corp., 80 F. Supp. 2d 815, 880 (N.D. Ill. 1999) (citing Knaack Mfg. Co. v. Rally Accessories, Inc., 955 F. Supp. 991, 1000 (N.D. Ill. 1997); Henri's Food Prod., Inc. v. Kraft, Inc., 717 F.2d 352, 354 (7th Cir. 1983)).
As Judge Hamilton observed in Simon Property Group,"[n]o survey is beyond criticism, especially in the context of litigation." 104 F. Supp. 2d at 1039. That observation accords with the Seventh Circuit's teaching that survey evidence need not be perfect to be admissible. Indianapolis Colts, Inc. v. Metropolitan Baltimore Football Club, 34 F.3d 410, 416 (7th Cir. 1994). In fact, the court of appeals has stressed that only in "rare" situations will a proffered survey be "so flawed as to be completely unhelpful to the trier of fact and therefore inadmissible." AHP Subsidiary Holding Co., 1 F.3d 611, 618. Finally, consistent with the Seventh Circuit's observations about bench trials noted above, the Court may provisionally admit borderline opinion testimony and exclude it later if, upon further reflection, it is not sufficiently reliable or relevant to be entitled to any consideration at all. See Simon Property Group, L.P., 104 F. Supp. 2d at 1039 n. 3 (explaining that in cases dealing with problematic survey evidence involving an injunction hearing or a bench trial, "the safest course for the trial judge is to admit the evidence and to treat the criticisms as going to the weight of the evidence").
Callahan's survey, composed of eight questions asked to 360 participants (out of 8,000 calls made), was relatively straightforward. The first two questions gauged familiarity; the third, fifth, and seventh questions measured perceptual familiarity; and the fourth and sixth questions requested that the participant explain his or her reasoning. Plaintiffs and their lawyer requested that the eighth question be added to the survey, and Callahan claims that it did not affect his analysis. The participants had the option to choose one of six answers ranging from "strongly agree" to "strongly disagree," including "can't say," and no answer was suggested as the correct one. Closed-ended questions are thought to be useful in surveys. See FJC REFERENCE MANUAL at 253. Although Callahan did not rotate the responses — and therefore the participant might have reflexively answered "can't say," because it was always the last option in the close-ended questions — his report contains the percentages of times that the participants selected an answer. Thus, the Court can pinpoint responses that may have been suspect because of the recency effect and take that potential shortcoming into consideration in assessing the reliability of the survey. Finally, participants were directed not to guess when responding, which presumably supports the reliability of the survey because the participants would have answered knowledgably or not at all.
Despite these useful features, the survey has many significant flaws. First, the universe is too broad. Callahan's definition of the relevant universe was the entire Chicago metropolitan area (or at least anyone in that area with a telephone), without regard to the respondents' purchasing preferences. The participants were filtered according to demographic quotas and to exclude businesses. Furthermore, the survey did not use threshold questions to measure the preferences or purchasing inclinations of the participants. As such, the survey was overinclusive, as it included many participants who were not in the market for either Bobak's Sausage Company Products or A J's Signature Events products. The survey also was underinclusive because it excluded businesses, a large section of A J's market. In a trademark case, the proper universe usually is potential purchasers of the junior users' products or services. See LG Electronics U.S.A., Inc. v. Whirlpool Corp., 661 F. Supp. 2d 940, 953 (N.D. Ill. 2009). To narrow the scope of potential customers of A J, Callahan could have added two simple survey questions: (1) Have you sought to purchase banquet or conference facility services in the last twelve months? and (2) Do you plan to purchase banquet or conference facility services in the next twelve months? See id. And to broaden the survey appropriately, Callahan could have included businesses.
The survey also used leading questions. Respondents were asked if they agreed with statements such as (1) "It is likely that Bobak's brand food products and Bobak's Signature Events have the same ownership," and (2) "When planning an event, it is likely that I would choose Bobak's Signature Events based on the belief that they serve Bobak's brand food products." These questions appear skewed to obtain a desired result, by linking Bobak's brand food products and Bobak's signature events (and thereby implying that the two are affiliated) and also by using "likely" to imply the answer to the question.
Furthermore, Callahan did not use the visual marks and he took no steps to put the trademarks in a typical marketplace situation. Admittedly, as the products sold by the parties are substantially dissimilar, a marketplace situation would have been difficult to replicate. And Callahan's perceptual similarity questions marginally addressed whether or not the companies are associated with one another. Therefore, although Callahan's survey is not as probative as it would have been if it had used the actual marks, the survey is not fatally flawed simply because it was a telephone survey.
The survey also had minimal controls. Surveys typically use a control group or a control question. Nat'l Football League Props., Inc. v. Prostyle, Inc., 57 F. Supp. 2d 665, 667-73 (E.D. Wis. 1999). Trademark surveys measure how the trademark influences participants' "perceptions or understanding of a product." FJC REFERENCE MANUAL at 256. Therefore, a control group or control question is used to measure the origins of the perceptions in order to assure that the participants are not basing their answers on preconceptions. Id. In addition, the ability to evaluate the effect of the wording of a particular question makes the control group design particularly useful in assessing responses to closed-ended questions, providing an additional safeguard against poorly worded questions. Id. at 258.
Callahan claims that "principle components analysis" was his control. Although Callahan did not use a control group, he contends that the familiarity questions and the random selection of respondents were types of controls. However, Callahan did not explain if the responses to the familiarity questions affected whether or not the participants were included in the study. Otherwise, he took no steps to account for skewing factors specific to a trademark case. Plaintiff's response argues that this statistical analysis was used to "ensure the survey respondents were not giving random answers." (Pl. Resp. at 10.) However, the goal of a control in a trademark survey is not only to prevent random answers, but also to bring preconceptions to light. Therefore, in Callahan's survey, is it difficult to ascertain the baseline knowledge of some of the participants. This is apparent in some of the responses. For example, in response to a follow-up question, one participant answered, "We used to have a Bobak restaurant and there was (sic) some problems with the family and the ownership of the store." (Callahan Dep. 98:8-10.) Furthermore, the use of a control group or a control questions could have emphasized potential problems with Callahan's questions. See FJC REFERENCE MANUAL at 258. Despite these concerns, Plaintiff raises a legitimate question concerning whether an additional control (another hypothetical "Bobak" entity) would have added much to the reliability of the survey in the particular circumstances of this case.
Technical defects in a survey are a matter of degree. The judge, as a trier of fact, has wide discretion in assessing whether the cumulative effect of technical defects simply affects the weight given to the survey or renders the expert's opinion inadmissible altogether. LG Elecs. U.S.A., Inc., 661 F. Supp. 2d at 951. The Court's preliminary conclusion is that the flaws described above substantially limit the helpfulness of the proposed survey in this case to the trier of fact. Plaintiff has described the survey as "no frills," which is not a problem if at least the critical steps in survey methodology are followed. Here, for the reasons stated above, it is evident to the Court that "plaintiff could have conducted its survey more carefully." McDonald's Corp. v. McBagels, Inc., 649 F. Supp. 1268, 1278 (S.D.N.Y. 1986). Two hallmarks of a good survey include the selection of an appropriate universe of respondents and the use of non-leading questions. The Callahan survey fell somewhat short of the mark in both respects.
Cumulatively, the flaws noted above present a close question in regard to whether to exclude Callahan and his survey evidence. On balance, however, the Court cannot conclude at this time that as a result of these errors, the Callahan survey is one of the "rare" surveys that is "so flawed as to be completely unhelpful to the trier of fact and therefore inadmissible." AHP Subsidiary Holding Co., 1 F.3d at 618. The Court stresses that this conclusion is preliminary-in part because of the somewhat atypical circumstances in which this motion was brought. The admissibility of survey evidence often is resolved in the context of a battle of the experts, where both sides have developed and placed before the Court competing views on the issue of consumer confusion. Here, because Defendants chose to seek the exclusion of Plaintiff's expert before undertaking any expert work of their own, the Court has evaluated Dr. Callahan's survey with a less developed record than often is the case. In addition, because there is no jury demand in this case, the Court retains the ability to "exclude" or "disregard" Dr. Callahan's testimony and survey at a later stage — even after trial — if it later concludes that the testimony and survey are of little or no value in deciding the issues in the case. See Simon Property Group, 104 F. Supp. 2d at 1039 n. 3.
III. Conclusion
For the reasons stated above, Defendants' motion [52] to exclude the expert testimony of Plaintiff's expert witness is denied without prejudice. This case is set for further status on 5/10/10 at 9:00 a.m.