Summary
explaining that "[t]ests are valid if, and only if, they predict performance"
Summary of this case from Lewis v. City of ChicagoOpinion
Case No. 98 C 7673
September 30, 2002
MEMORANDUM OPINION AND ORDER
In this class action suit, two subclasses of minority police officers allege that City of Chicago violated Title VII of the Civil Rights Act of 1964, 42 U.S.C. § 2000e et seq., when it selected other officers for promotion to the rank of sergeant in August 1998. One subclass has moved for summary judgment. The City has cross-moved for summary judgment as to both subclasses. For the reasons set forth below, the City's motion is granted.
Some historical context is helpful. Since 1970, the City has been defending itself against allegations of racial discrimination in the hiring and promotion of police officers. United States v. City of Chicago, 411 F. Supp. 218, 225 (N.D. Ill. 1976), aff'd, 549 F.2d 415, 438-39 (7th Cir. 1977). In 1976, Judge Marshall found that the 1973 sergeant examination had a disparate impact and was not a valid predictor of performance on the job. Id. at 236-39. The City operated under promotion quotas until 1988, when the district court approved a new promotion roster based on race-formed examination results. United States v. City of Chicago, No. 73 C 2080, 1988 U.S. Dist. LEXIS 13242 (N.D. Ill. Nov. 21, 1988). After passage of the Civil Rights Act of 1991, the City abandoned this approach.
To the extent the parties object to any of the historical facts outlined below as irrelevant and immaterial, their objections are overruled.
"When the scores from the 1994 [lieutenant] examination resulted in promotions in a racial pattern significantly different from the racial make-up of the applicant pool, the City attempted to rectify the situation by combining merit promotions with the rank-order promotions." Bryant v. City of Chicago, 200 F.3d 1092, 1095 n. 1 (7th Cir. 2000). The Superintendent selected 13 sergeants for promotion on the basis of merit selection (20% of promotions) and 54 on the basis of their examination score and rank on a promotion list. A police sergeant, James McArdle, filed suit in state court seeking to enjoin merit promotions on the theory that they were not permitted by then-existing rules. The state court granted McArdle a preliminary injunction and that order was affirmed on appeal. McArdle v. Rodriguez, 659 N.E.2d 1356 (Ill.App.Ct. 1995). While McArdle was pending, a group of African-American sergeants challenged the 1994 lieutenant examination in federal court on disparate impact grounds Alter a trial on the merits, Judge Gettleman held that the examination was valid and consistent with business necessity, but that the City failed to use a less discriminatory, equally valid selection device — namely, making 20% of lieutenant promotions through merit selection. Brown v. City of Chicago, 8 F. Supp.2d 1095, 1110-13 (N.D. III. 1998), aff'd sub nom., Bryant, 200 F.3d at 1092.
While the lieutenant-promotion litigation was working its way through the courts, a group of African-American police officers filed a disparate impact challenge to the 1994 sergeant examination. After an evidentiary hearing, Judge Nordberg denied the plaintiffs' motion for a preliminary injunction and permitted rank-order promotions to be made. Adams v. City of Chicago, No. 94 C 5727, 1996 U.S. Dist. LEXIS 3567 (ND. Ill. Mar. 25, 1996), aff'd, 135 F.3d 1150 (7th Cir. 1998). Among other things, Judge Nordberg rejected plaintiffs' proposals that cut-off scores and merit promotions be used as alternative selection mechanisms. Id. at *47-49, After the 1994 examinations that gave rise to Adams and Brown, the City empaneled a task force to reassess the police department's promotion processes. The task force's recommendations (including the 30% merit component described below) were embodied in the 1998 sergeant examination that is the subject of the present litigation.
The selection process for sergeant promotions in August 1998 consisted of three components: (1) a written qualifying test, (2) an assessment exercise, and (3) a merit selection process. Only candidates who achieved a sufficiently high score on the written test were eligible for promotion either through the assessment exercise or through merit selection. Seventy percent of promotions were awarded to the highest scorers on the assessment exercise. The remaining 30% of positions were filled through merit selection, whereby trained command staff nominated up to a prescribed number of candidates (based on the number of officers supervised), a committee identified the most qualified nominees, and the Superintendent of Police ultimately selected from among this group.
This case is not the only challenge to the 1998 sergeant promotions. In Barnhill v. City of Chicago, 142 F. Supp.2d 948 (ND. Ill 2001), a group of unsuccessful sergeant candidates, all white men, claimed that the merit component "was a mask for an illegal affirmative action program" and that it "had a disparate impact on `non-minority males.'" Id. at 950. Judge Pallmeyer eventually entered summary judgment in favor of the City on a combination of standing and substantive grounds.
Plaintiffs in the case at bar attack the promotion process from the opposite direction, arguing for expansion rather than elimination of the merit component. Subclass A consists of minority candidates who failed to pass the written test; Subclass B consists of minority candidates who passed the written test, but nonetheless failed to be promoted. Allen v. City of Chicago, No. 98 C 7673, 2001 U.S. Dist. LEXIS 11103, at *12-13 (N.D. Ill. Apr. 19, 2001). Neither subclass alleges intentional discrimination. Rather, each asserts that the selection process had an impermissible disparate impact:
In a Title VII disparate impact case, the plaintiff bears the initial burden of establishing a prima facie case by showing that the promotional method in question had an adverse impact on minorities. If the plaintiff makes this required initial showing, the burden then shifts to the employer who must prove that the evaluation method is valid by showing that it is "job related" and "consistent with business necessity." 42 U.S.C. § 2000e-2(k)(1)(A)(i). The evaluation method may be shown to be job related under any one of three tests: criterion related, content validity) or construct validity. Uniform Guidelines on Employee Selections Procedures, 29 C.F.R. § 1607.5B. If the employer succeeds in validating the evaluation method, the burden shifts back to the plaintiff to prove that there was another available method of evaluation which was equally valid and less discriminatory that the employer refused to use. 42 U.S.C. § 2000e-2(k)(i)(A)(ii); see also Albemarle Paper Co. v. Moody, 422 U.S. 405, 425 (1975).Bryant, 200 F.3d at 1094. The Supreme Court and statute describe the last stage of this analysis as follows: the plaintiff must show "that other tests or selection devices, without a similarly undesirable racial effect, would also serve the employer's legitimate interest[s)," Albermarle, 422 U.S. at 425 (internal quotation marks omitted), and that the employer "refuses to adopt such alternative employment practice." 42 U.S.C. § 2000e-2(k)(1)(A)(ii).
It is undisputed that the written test had a disparate impact on African-American candidates and that the assessment exercise had a disparate impact on both African-American and Hispanic candidates. The passage rates on the written test by race and ethnicity were: White, 90.8%; African American, 73.3%; Hispanic, 82.1%; and Other, 90.2%. The success rates on the assessment exercise were: White, 9.8%; African American, 3.4%; Hispanic, 3.1%; and Other,
In light of the undisputed fact that the written test had no disparate impact on Hispanic candidates, plaintiffs' request to modify the definition of Subclass A to exclude Hispanics is granted. Cf Allen, 2001 U.S. Dist. LEXIS 11103, at *12-13 (defining Subclass A to include both African-Americans and Hispanics).
5.7%. The merit selection component, on the other hand, had no disparate impact. The success rate for each group was between 2.7% and 3.0%. All three components of the selection process are conceded to be "job related" under the content validity method and "consistent with business necessity." To succeed, plaintiffs must therefore show that there "was another available method of evaluation which was equally valid and less discriminatory that the employer refused to use." Bryant, 200 F.3d at 1094.
Understanding the concept of validity is crucial in this case. As noted above, there are three acceptable methods of demonstrating validity:
Evidence of the validity of a test or other selection procedure by a criterion-related validity study should consist of empirical data demonstrating that the selection procedure is predictive of or significantly correlated with important elements of job performance. Evidence of the validity of a test or other selection procedure by a content validity study should consist of data showing that the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated. Evidence of the validity of a test or other selection procedure through a construct validity study should consist of data showing that the procedure measures the degree to which candidates have identifiable characteristics which have been determined to be important in successful performance in the job for which the candidates are to be evaluated.
29 C.F.R. § 1617.5B (citations omitted).
Although there is no general preference under Tide VII for one validation method, Gillespie v. Wisconsin, 771 F.2d 1035, 1041 (7th Cir. 1985), it has been said that
the conceptual core of validation is criterion validation. Tests are valid if, and only if, they predict performance, although content or construct validation may be cheaper and more convenient techniques to use. Content or construct validity are simply plausible predictions that true criterion validity would in fact exist if tested for.
Mark Kelman, Concepts of Discrimination in "General Ability" Job Testing, 104 Harv. L. Rev. 1158, 1171 n. 35 (1991). Selection mechanisms are instrumentally, not intrinsically, valuable. The mere fact that a test "is representative of important aspects of performance on the job" (as content validity requires) matters only because it is reasonable to suppose that such a test will usefully distinguish among candidates — in other words, that using the test in selection will likely lead to a better performing workforce. For example, it is often said that "a typing test given to prospective typists would be validated by the content validation method." Gillespie, 771 F.2d at 1040 n. 3. The implicit assumption is that performance on a typing test is correlated with performance as a typist. If for some bizarre reason this assumption were false, then random selection would do as well and the test would not be valid, no matter how representative it might be of a typist's job duties. It is only because the past is a decent predictor of the future that the concept of content validity (or, indeed, validity generally) makes sense. Predictive validity is the gold standard — content validity, construct validity, and, given measurement error, even criterion related validity are valuable only as proxies.
This general discussion leads to a more specific point. That a content-valid test can usefully identify a small percentage of outstanding candidates or usefully exclude a small percentage of unqualified candidates does not necessarily justify its use as a rank-order selection mechanism. Guardians Ass'n of New York City Police Dep't v. Civil Service Comm'n, 630 F.2d 79, 100 (2d Cir. 1980). The reason is that "content validity is not an all or nothing matter; it comes in degrees." Id.
This last point is fatal to plaintiffs' motion. Subclass B argues that increasing merit promotions beyond the 30% ceiling would be an equally valid, less discriminatory alternative to the present system. Plaintiffs offer no specific percentage — the City assumed in its response that plaintiffs sought to shift entirely to merit selection; in their reply brief, plaintiffs use 35%, 40%, and 50% interchangeably. Even assuming that such a vague and indefinite proposal could qualify as an "alternative employment practice," 42 U.S.C. § 2000e-2(k)(1)(A)(ii), plaintiffs' proffered evidence does not conclusively establish that using the merit selection for more than 30% of promotions would be as valid as the present system. Plaintiffs rely solely on statements made by the City and one of its expert consultants to the effect that the merit selection process was "a substantially equally valid selection procedure." (Pls' Rule 56.1 Statement ¶¶ IV.13 to IV. 15.) When each statement is read in context, however, it is perfectly clear that the City was comparing the content validity of the 30% merit selection scheme to the alternative of no merit selection, not to the alternative of 35%, 40%, or 50% merit selection.
For example, the City's answer to paragraph nine of the amended complaint states: "Defendant admits only that the merit selection process that was included in and used as part of the 1998 police sergeant examination process (including, but not limited to, the fact that the merit selection promotions were limited to less than 30% of all promotions and that the remaining promotions were based on the results of the assessment exercise component), was a substantially equally valid selection procedure." (Def.'s Rule 56.1(b)(3) Resp. ¶ IV.14.)
In effect, the City was claiming that the least qualified candidate promoted via the merit route was as about as likely to perform well as a sergeant as was the candidate with the lowest successful score on the assessment exercise. Because plaintiffs concede the validity of both selection methods as applied, to fill more positions through merit selection would necessarily diminish the validity of the overall process in the sense of reducing the expected performance of the selected candidates. If, as plaintiffs suggest, each commander nominated six instead of five candidates for merit selection, the sixth nominee (assuming no ties) would be less qualified than the fifth or she would have been selected in the first instance. Perfect mathematical equality, however, cannot be the standard. A certain amount of imprecision is inevitable and must be tolerated if employers are ever to be permitted (or required) to adopt alternative practices. To succeed on an alternative employment practice claim, a plaintiff must show only that the actual practice and a refused alternative have substantially, not perfectly, equal validity. Plaintiffs here offer no evidence comparing the 30% merit selection regime to any alternative with greater reliance on merit selection. Indeed, plaintiffs admit that "[a]s the number of persons to be nominated increases, it becomes more and more difficult to select highly qualified people and the closer the system comes to facing the problems inherent in performance rating systems." (Pls' Resp. Def.'s Rule 56 Statement ¶ 165.) Plaintiffs have clearly failed to demonstrate that they are entitled to summary judgment.
The City has cross-moved for summary judgment as to Subclass B. In deciding this motion, all reasonable inferences must be indulged in plaintiffs' favor. A slight alteration in the present system would almost certainly have only a slight impact on overall predictive validity. The undisputed fact that merit selection and the assessment exercise had substantially equal validity at the 30% to 70% ratio suggests that they would have substantially equal validity at a slightly different ratio. It is also reasonable to infer from the fact that the merit selection component, unlike the assessment exercise component, had no disparate impact that a shift toward merit selection would reduce the overall disparate impact. But these inferences can be drawn whenever an employer partially modifies a selection system in order to alleviate a disparate impact. So long as some disparate impact remains, the employer can be accused of not going far enough. Given the inherent imprecision of validation, defending a particular level of emphasis on the new component is a difficult task. There is a good reason Congress placed the burden of demonstrating a valid alternative on those who would challenge an employment practice, not on the employer. 42 U.S.C. § 2000e-2(k)(1)(A)(ii). Otherwise, employers would have a powerful incentive not to consider, let alone expend resources validating, less discriminatory alternative practices: as here, the employer's own validation efforts could be used against it. At a minimum, challengers in such cases must specify a particular alternative practice to create a triable issue of fact. Plaintiffs here fail to meet even this minimal burden, so the City is entitled to summary judgment.
This conclusion is bolstered by the apparent reasonableness of the City's efforts to devise a less discriminatory alternative to pure rank-order selection based on the assessment exercise. The 1998 sergeant promotion system represents but one iteration in a process dating back at least to the 1970s. It is undisputed that the task force recommending the 30% figure knew that a 20% merit selection scheme had been successful in detective promotions, and that the task force considered 50% and 100% alternatives but were concerned about validity and practicality at those larger percentages. (Pls.' Rule 56 Statement ¶ V.05.) Plaintiffs' characterization of the 30% figure as "arbitrary" is dubious at best. (Id.) In any event, the EEOC regulations appropriately recognize that a certain degree of arbitrariness is inevitable, explaining that reasonable consideration of alternatives should generally immunize a valid selection system from immediate disparate impact challenges. "If a user has made a reasonable effort to become aware of such alternative procedures and validity has been demonstrated in accord with these guidelines, the use of the test or other selection procedure may continue until such time as it should reasonably be reviewed for currency." 29 C.FR. § 1607.3B; see also Watson v. Ft. Worth Bank Trust, 487 U.S. 977, 999 (1988) (opinion of O'Connor, J.) ("In evaluating claims that discretionary employment practices are insufficiently related to legitimate business purposes, it must be borne in mind that courts are generally less competent than employers to restructure business practices, and unless mandated to do so by Congress they should not attempt it.") (internal quotation marks omitted). The City has already (successfully) defended itself against claims that merit selection amounted to affirmative action and that the overall scheme imposed a disparate impact on whites. That both minority and non-minority officers are upset is itself indicative of reasonableness.
It is worth noting the City chose to emphasize merit selection more heavily in sergeant promotions than the level that had proven successful in detective promotions, which tends to undermine plaintiffs' contention that it did not go far enough.
To be clear, nothing in this opinion should be taken to suggest that 30% is the only defensible level, only that the City has in fact adequately defended it against plaintiffs' attack. The City can and should reexamine its sergeant promotion system with reasonable regularity. 29 C.F.R. § 1607.3B
The City's motion for summary judgment as to Subclass A requires less analysis. Subclass A, which consists of African-American officers who failed the written test, does not challenge the written test generally, only its use as a prerequisite for merit promotion. The undisputed facts, however, foreclose this claim. As noted above, the written test is conceded to be content valid, job-related, and consistent with business necessity. (Pls.' Resp. Def.'s Rule 56 Statement ¶ 70.) It is also undisputed that the merit selection process does not purport to assess specific job knowledge as is measured on the written test and "was not designed nor was its validity supported as a stand-alone procedure." (Id. ¶¶ 167, 168.) It necessarily follows that a merit promotion scheme without the written test would be less valid than the present process. The reduction in validity (as with an increase in the 30% figure) could be insubstantial, but based on the present record that is pure speculation. All of the statements in the record concerning the validity of the merit selection component assume that it will be used in combination with the written test. There is simply nothing in the record from which one can reasonably infer the merit component's stand-alone validity. The City's motion for summary judgment as to Subclass A is therefore granted.
In their response to the City's fact statement, plaintiffs argue that the choice of passing score on the written test was not supported by empirical data and that choosing a lower cut-off would have minimized the disparate impact. (Pls.' Resp. Def.'s Rule 56 Statement¶ 90.) Plaintiffs cite no record evidence to support this argument, refer to this argument nowhere in their briefs, and do not attempt explain the inconsistency with their express disavowal in a response to interrogatories of any intent to litigate the passing score issue. (Def.'s Rule 56.1(b)(3)(B) Statement Ex. 7 ¶ 5.) This argument is waived. Even if it were not, plaintiffs have not even argued that using a lower cut-off score would result in a process as valid as the present system.