Opinion
No. 84 C 2718.
August 19, 1988.
Kenneth N. Flaxman, Elizabeth Dale, Chicago, Ill., for plaintiff.
Jack M. Siegel, Jack M. Siegel Associates, Ltd., Chicago, Ill., for defendants.
MEMORANDUM OPINION AND ORDER
Findings of Fact
1. This case arises under Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq. It was certified as a class action on November 20, 1985. The class is thirty-nine women who took Evanston's firefighter physical agility test in 1983.
2. Charlene Evans, plaintiff, a woman residing in this District, took the test on August 6, 1983 and failed it. She has pending an individual damage claim under 42 U.S.C. § 1983, but both parties indicate settlement will follow decision of the Title VII claim.
3. The City of Evanston, a municipal corporation, and its Director of Personnel are defendants. Evanston has a population of 74,000 or so and employs 106 firefighters, none of whom are women though it had previously employed two women as firefighters.
4. The jurisdictional prerequisites of prior timely complaint to the Equal Employment Opportunity Commission and timely filing of this complaint after receiving the "right to sue" letter have been met with respect to the August, 1983 test.
5. Evanston begins to hire firefighters by administering a physical agility test. Those who fail are given no further consideration. Those who pass take a written test. After initial hiring the entering firefighters are trained at an Academy used by several municipalities. The Academy instructs these individuals, inter alia, in the proper techniques of performing the physical tasks of firefighting.
6. The physical agility test used in 1983 (and 1981 and 1985 as well) consisted of five events. Persons wishing to be firefighters were to attend an advance orientation session which plaintiff and others did. A written summary of information was provided. It accurately summarized the physical agility test administered in 1983. The summary stated:
Examination
Because this exam is designed to test your physical abilities as it simulates actual tasks/functions performed by firefighters, we urge you to practice. To facilitate your "work-out", listed below, please find a brief description of the exercises you will be required to perform.
1. Aerial Ladder Climb: Climb to top of the aerial ladder which will be extended to 70 ft. You will wear a coat, air pack and face mask.
2. Ladder climb: Climb extension ladder twice while carrying a hose pack. You will wear a coat.
3. Ladder Carry and Set Up: Remove ladder from the side of the engine and carry it to the wall, set up to a climbing angle, remove, carry and return it to the engine. You will wear a coat.
4. Hose Connect: Connect 2 1/2" hose to a fire hydrant, turn the hydrant completely on and off, and disconnect. You will wear a coat.
5. Agility Course:
a. Drag one section of 2 1/2" hose filled with water 50 ft.
b. Climb to the top of the ski hill while carrying a tarp and return via the stairs on the opposite side of the hill.
c. Walk through ten (10) tires while carrying the tarp.
d. Drag one section of 2 1/2" hose filled with water 50 ft.
The applicants were informed that the test would be timed. Time was kept beginning with the first event, and the clock was not stopped until completion of the final event.
6. The physical agility test was devised in 1980 in this way. The test was created by Evanston's then-Director of Personnel and Fire Chief along with David Franzen, then and now, the Division Chief in charge of training. Prior tests had used events such as pushup, pullup and rope climbs. The new tests were to be "completely job related." The method used was to examine their own knowledge of the physical tasks of firefighting which in the case of the training officer consisted of, in addition to the ordinary experience of firefighting, the experience of attending and critiquing all fires in Evanston and keeping records as to the task performed at fires. These post-fire critiques were widely attended by fire officers and individual firefighters. The group considered some of the appropriate literature on qualifications and the physical agility tests actually used throughout the immediate area. Two of the members of the group were clearly experts in firefighting. The Director of Personnel, who was not such an expert, attended several fires during the period when the test was developed. The group also conferred with other members of the Evanston Fire Department. The group prepared a test using physical actions they found to be important to firefighting. The test, once devised, was then administered to all incumbent firefighters under the age of thirty-five whose performances were timed. Timing of the tests was done because speed of performance is an important element of effective firefighting.
6A. The work of the group did not result in a written form of occupational analysis sometimes seen in this area of personnel administration. These types of analyses usually list each of the various physical (and, where appropriate, mental) operations organized in some logical fashion accompanied by an analysis of the frequency and importance of their proper performance in the occupation. Often this is done by written surveys of incumbents using a list of tasks identified by subject matter experts. See Guardian's Association v. Civil Service Commission, 630 F.2d 79 (2nd Cir. 1980). It is apparent that this method was not used or, at least, that evidence of its use does not exist. A written job analysis was not presented at trial.
7. Evans completed the 1983 test in 880 seconds (14 minutes, 40 seconds). Evanston selected a cut-off score by passing all those within one standard deviation above the mean (767 seconds). The exact number of persons taking the test is stipulated to be 839. It is also stipulated in one place that 38 women took the test and, in another, that 39 took the test. The defendants now submit that only 832 took the test. The differences in the stipulations are immaterial. The overall passing percentage rate was in the high eighties. Male applicants passed at a rate of over 90% and female applicants at a rate of less than 16%.
8. It is unclear that Evanston had consistently applied the same method of determining the cut-off point. In 1985 when the cut-off was 915 seconds, it appears to be nearly three standard deviations above the mean. In 1981 it was stipulated that the cut-off score was 890 seconds, but there is truthful testimony by the current Director of Personnel, Judith Witt, that, in 1981, she deducted a constant number (which she does not recall) from the score of the four women who took the 1981 test. This deduction enabled three of their four (including Evans at 889 seconds) to achieve passing grades.
9. The parties have stipulated that the passing rate for women on the 1983 tests is less than 80% of the passing rate for men on the same test and that the discrepancy "is statistically significant: that the likelihood that the difference in male and female passing rates arose from chance is less than one chance in a billion."
10. Evanston hired nine persons, all male, from among those who took the 1983 test; most were hired on May 7, 1984.
11. To the extent that in the following passages there are sentences which are findings of fact rather than conclusions of law those sentences shall be deemed findings of fact.
Analysis, Further Findings of Fact and Conclusions of Law
I. Here this Court must follow the standard ". . . approach to disparate impact cases set forth in Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971), and its progeny. To establish a prima facie case of discrimination, `a plaintiff need only show that the facially neutral standards in question select applicants for hire in a significantly discriminatory pattern.' Dothard v. Rawlinson, 433 U.S. 321, 329, 97 S.Ct. 2720, 2726-27, 53 L.Ed.2d 786 (1977). When a plaintiff makes this threshold showing of disparate impact, the employer bears the burden of showing that its discriminatory policy or practice is necessary and manifestly related to job performance. Id.; Griggs, 401 U.S. at 431-32, 91 S.Ct. at 853-54. If the employer satisfies its burden, the plaintiff may show that less discriminatory alternatives would also `serve the employer's legitimate interest in "efficient and trustworthy workmanship."' Albemarle Paper Co. v. Moody, 422 U.S. 405, 425, 95 S.Ct. 2362, 2375, 45 L. Ed.2d 280 (1975), quoting, McDonnell Douglas Corp. v. Green, 411 U.S. 792, 801, 93 S.Ct. 1817, 1823, 36 L.Ed.2d 668 (1973); . . ." Thomas v. City of Evanston, 610 F. Supp. 422, 427 (N.D.Ill. 1985); Evans v. City of Evanston, 621 F. Supp. 710, 712 (N.D.Ill. 1985) (Rovner, J.). Less discriminatory alternatives are not at issue in this case. As stated in Plaintiff Reply To Proposed Finding of Fact and Conclusions of Law, "Plaintiffs agree that some sort of physical agility test should be given [to firefighter applicants]." (Reply H, citing testimony of Plaintiffs' expert Spetz.)
II. Evanston contends that there is no initial showing of adverse impact noting correctly that the "80% rule" is a guideline, not a legal definition of adverse impact. The rule comes from the EEOC Guidelines (29 CFR, pt 1607) adopted in 1978 by the EEOC, the Civil Service Commission, the Department of Labor and the Department of Justice. It says adverse impact is demonstrated by a selection rate for any race, sex or ethnic group which is less than four-fifths or 80% of the rate for the group with the highest rate of selection. Violation of the guideline, however, does ordinarily establish a prima facie case of disparate impact discrimination. See Connecticut v. Teal, 457 U.S. 440, 443, n. 4, 102 S.Ct. 2525, 2529 n. 4, 73 L.Ed.2d 130 (1982); United States v. City of Chicago, 663 F.2d 1354, 1357 n. 8 (7th Cir. 1981) (en banc); Compare Clady v. County of Los Angeles, 770 F.2d 1421, 1428-29 (9th Cir. 1985). And in this case there is more than mere application of the guideline — there is the stipulation that the discrepancy is statistically significant, (Fudge v. City of Providence, 766 F.2d 650 (1st Cir. 1985)), and the report of Evanston's expert Shepherd which opined that the 1983 test had an "adverse impact on female applicants." Finally, the margin by which the four-fifth guideline was missed was not close either in terms of final selection for employment or pass-fail rates. Castaneda v. Partida, 430 U.S. 482, 496, 97 S.Ct. 1272, 1281, 51 L.Ed.2d 498 (1977). See Thomas v. City of Evanston, 610 F. Supp. 422 (N.D.Ill. 1985); Schlei and Grossman, Employment Discrimination Law 98-99 (2nd ed. 1983) (1985 Supp. at 18-19).
Judge Rovner also found disparate impact in this case. Evans v. City of Evanston, 621 F. Supp. 710, 712 (N.D.Ill. 1985). It may be that the size of the group going through the test was not large enough to lead to a statistically valid conclusion of adverse impact (See testimony of Dr. Metzcus) but it appears this argument is not pursued here.
III. Evanston next asserts that whatever its impact its physical agility test is valid. The EEOC Guidelines provide three methods by which selection procedures might be found valid, to wit, construct validity, criterion validity and content validity.
Criterion validity is established when there is a significant correlation between comparative success in the test and comparative success in job performance. Validity is shown by studies of those who take the test before employment and are hired regardless of the test scores. Job performance is rated after a period of time and job ratings are checked against test results to determine the correlation between the two. Or the test may be administered to those already employed at the job with a comparison again made between test performance and job performance.
Construct validity depends upon correlation between a test and the presence of a trait required for success in the job, i.e., intelligence, judgment.
Content validation occurs when the test involves the performance of tasks actually performed on the job. Those wishing to be typists, stenographers and court reporters usually take content valid tests.
Evanston claims its test is content valid and Evans says it cannot be, and she offers three arguments to which the Court will turn in their logical order.
A. Adequacy of Job Analysis
The principal attack on Evanston's firefighter job analysis is that it is inadequately documented as was the case with Evanston's police officer job analysis. Thomas v. City of Evanston, 610 F. Supp. 422 (N.D.Ill. 1985). In Thomas, Evanston based its job analysis on the inconsistent survey responses of three police chiefs and thirty hours of observation of actual police activities by a graduate student riding along with police officers. It is apparent the process here was more encompassing. Two members of the group developing the job analysis were expert firefighters, one of whom had the task of observing and analyzing firefighter performance at each of the fires in Evanston. The one non-expert in firefighting also observed firefighting. Other members of the fire department reviewed the tasks identified by the job analysis group, and the proposed test was administered to many incumbent firefighters.
There is no question that a job analysis could, or perhaps should, have resulted in more documents than Evanston produced here, and Evanston would not have been left in the position of having its job analysis claim dependent upon a Court's finding as to the credibility of oral recollections of the process it used. Nonetheless, this Court does accept, as indicated in Finding 6, the statements of Franzen and others as to the manner in which the test was devised. It is significant that neither Magel nor Spetz, plaintiff's experts, denied that any of the agility tests involved tasks unimportant to firefighting (although Magel did express doubt about the importance of speed in moving a water-filled hose). Nor did any expert state that the physical agility test omitted any activity important to firefighting.
Dr. Magel expressed persuasively the view that a proper job analysis would include quantification of the physiological requirements of a job and that the EEOC Guidelines are inadequate because they do not require physiological quantification. This he attributed to the dominance of psychologists in the development of the Guidelines. The difficulty with accepting Dr. Magel's testimony as a guide to decision here is that it runs contrary to accepted methods of job analysis (as described by Dr. Spetz, plaintiff's other expert) and it is difficult to reject this accepted practice on the opinion of a single authority. In fact, Dr. Magel admitted that, to his knowledge, a full physiological analysis of firefighting has never been done in this country.
The Court finds the City of Evanston has met the minimum standards for performing a firefighter's job analysis and Evans has not shown its invalidity.
B. Adequacy of the Test
Evans's argument really proceeds against the premise of content validity. Her expert, Dr. Magel, testified that the validity of the test here failed because it was constructed without an effort to quantify the physiological requirements of the job of firefighter. He then testified that the ladder climbs were not valid measurements of leg strength or aerobic capacity, that the ladder setup was not a valid measure of upper body strength, that the agility course was not a good measure of aerobic capacity, and that the hose connect measured only manual dexterity involved in hose connection and disconnection.
This testimony rests on a rejection of the concept of content validity. On this view the only appropriate method of devising a physical test is to identify important tasks, discover the physiological requirements of each and then administer whatever tests are valid for aerobic capacity, and physical strengths in the environments relevant to the job. One premise of this theory is that it eliminates whatever success there is in Evanston's testing that is dependent in part upon training in the specific skills; but, of course, training may be of equal value in success at general tests of aerobic capacity and physical strength. In any event, the fact that one's performance on a test would be improved by training is, despite the suggestion of the Guidelines (14(c)(1)), not a reason to invalidate a test. Guardians Assn. v. Civil Service Comm., 630 F.2d 79, 92 (2d Cir. 1980).
This is a fair characterization of the principles espoused by Dr. Magel, who opined that firefighters are good at identifying firefighting tasks but poor at defining construct in firefighting work.
Under Evans's theory, one might say of the typist that he or she needs hand and finger speed, and coordination, eye control and the capacity for sustained concentration; and the best way to test these is to measure the physiological requirements of each of these elements in typing and then apply appropriate tests for each. All this rather than ask the candidate to sit and type. Of course, Evans does not argue this proposition, but her evidence suggests something like it. Dr. Magel's position on job analysis and test adequacy may well represent what the Second Circuit viewed as the "danger of too rigid an application of technical testing principles", Guardians Assoc. v. Civil Service, 630 F.2d at 90.
Evans is not to be faulted for making her argument. The training officer, Franzen, on examination by the City expressed the view that the physical agility tests did show dexterity, leg strength, upper body strength and so forth. Despite that testimony this Court believes the essence of Evanston's defense is that (a) firefighters often must speedily climb ladders (wearing coats, face masks, air packs and hose packs), set up ladders, connect hoses, and carry charged hoses and other things up and down and do so while strained and tired, and (b) the City of Evanston wants to see if the applicants can do these things.
The uncontradicted evidence here is that the acts performed in the physical agility test are exactly the sorts of activities a firefighter performs while on duty. With a few trivial exceptions (using tires to serve as obstacles in the street) the tests here require the actual performance of job duties and not the performance of tasks whose analogy to the actual job duties may be more or less strained.
Evans contends that Evanston is really arguing that the tests are "face valid", and this must fail under Burney v. City of Pawtucket, 559 F. Supp. 1089, 1101 (D.R.I.). But Evans is wrong. Burney did not involve a claim of content validity — the tests there for police cadets involved routine tests of physical fitness (e.g., a run, situp, broad jump) — a claimed criterion for success in police work.
Evanston's expert Dr. Shepherd credibly expressed the opinion that the physical agility test was a good predictor of future performance as a firefighter.
The evidence in this case established the content validity of the Evanston physical agility test.
C. Adequacy of the Cut-off Score
Evans challenges the cut-off score arguing that it has no relation to a proven requirement for firefighting.
The thrust of the evidence on this point is not very clear. Evans's expert Dr. Spetz (who believed cut-off scores should be established for physical agility tests) established that in Evanston the cut-off scores could be defined in terms of the number of standard deviations above the mean score were the following: 1981 — 1.7, 1983 — 1.0 and 1985 — 2.8. Dr. Spetz then expressed the view that 2.8 standard deviations above the mean was not an accepted way of fixing a cut-off score. Dr. Spetz never expressed any view about the method used in 1983, the year in issue here. Evanston's expert Dr. Shepherd simply stated that a cut-off score of one standard deviation above the mean was a method used frequently. He did not say its use was appropriate. The only expert addressing the issue was Dr. Metzcus, called by Evanston. He expressed the view that one standard deviation above the mean usually permits a pass rate of 84% which he opined was not much of a screen at all. The cut-off score, he believed, was not only reasonable but extremely generous and should have been higher. There is little other evidence on this point. When the physical agility test was administered to incumbent firefighters (under the age of thirty-five) in 1980 the average time was 688 seconds compared to the 767 seconds cut off in 1983. Seven of the thirty-two incumbents would have failed to make the 1983 cut off and two had times slower than the plaintiff's 1983 time. The training officer believed that at least one of the incumbent firefighters was not giving full effort to the test. The training officer also testified he believed the cut off should be 600 or 660 seconds, but he offered no reason why this should be so, and there is no evidence that this opinion was taken into account when the cut off was established.
The cut-off score was not based on a determination of what would be an appropriate completion time. Its sole purpose was to establish some point below which candidates would not be further screened. A figure giving a pass rate of about 84% was selected for this purpose. The results of the incumbent firefighter tests were not used to decide the cut off. Other than the cut-off score it appears no other use was made of the results of the 1983 test. Candidates were not selected in rank order from the test result list, and the nine hired from the 1983 list did not receive the nine fastest scores.
A cut-off score need not be separately validated but should be "reasonable and consistent with normal expectations of acceptable proficiency within the work force." EEOC Guidelines 5H; Craig v. County of Los Angeles, 626 F.2d 659, 665 (9th Cir. 1980). The Supplemental Information and Overview of the Uniform Guidelines, Para. V, notes that where there is adverse impact "an employer is required to justify the initial cutoff-score by reference to its need for a trustworthy and efficient workforce."
The cases have generally rejected cut-off scores unrelated to job performance. In Guardians Assoc. v. Civil Service Comm., 630 F.2d 79, 105-06 (2d Cir. 1980) a cut-off score was rejected which was set to produce the number of passing candidates necessary to fill the available job openings. In Craig v. County of Los Angeles, 626 F.2d 659, 665 (9th Cir. 1980) a cut-off score was approved because there was evidence that candidates who barely passed the examination displayed correspondingly poor performance in academy training.
Here Evanston adopted its cut off of one standard deviation above the mean because "generally that allows for about 84 percent of the normal applicant pool to pass and that was determined to be in the interest of the City" (Witt). No absolute maximum number of seconds to complete the test for prospective firefighters has ever been established by Evanston though the subject has been under discussion (Witt). The test results of incumbent firefighters were not considered (Haynes). The post-hoc justification for the cut-off score is, in Dr. Metzcus's view, based on statistics and not on the mechanics of firefighting, that a cut-off score eliminating only 16% of applicants is, if anything, too lenient. And Division Chief Franzen opined the passing score should be in the area of 600 to 660 seconds, which opinion was based upon his experience running the incumbent firefighters through the test.
Evanston cites Gillespie v. Wisconsin, 771 F.2d 1035 (7th Cir. 1985) to support its cut-off score. There Wisconsin administered a written test for entry level personnel officers, "the cut-off point was selected in order that [Wisconsin] could interview as many minority candidates as possible while at the same time assuring that the candidates possessed the minimum skill necessary to perform. . . . In addition the cut-off score was set so that the interviewers would not be overwhelmed by the sheer number of candidates . . .", 771 F.2d at 1045. But Gillespie is of little use to Evanston. There is no claim that the Evanston cut-off score served to prevent overwhelming of an interview process. Indeed the next step in the Evanston process was a written examination. The cut-off score was not defended as an effort to maximize the inclusion of female candidates. There was no effort to correlate in any way the cut-off score with ability to perform as a firefighter. The practice in Evanston involved only the use of a statistical determination of a passing score — the score varied from test to test reaching 890 seconds in 1981 and 915 seconds in 1985. The failure of seven of thirty-two incumbent firefighters (under the age of thirty-five) to meet the cut-off score speaks (though not conclusively) against the validity of that score. The only witness to express a view on a specific cut-off score (Division Chief Franzen on direct examination by plaintiff) based his view not on the observation of a correlation between fast times and successful training or firefighting, but on his observation of the tests run on incumbent firefighters. Under Franzen's suggested upper limit of 660 seconds, sixteen of the thirty-two incumbents would have failed to make the cut off.
Evans has proved that the cut-off score chosen by Evanston was unjustified.
I do not say that the 1983 cut-off score of 767 seconds is unjustifiable. I say it was not justified by Evanston. Evans introduced substantial evidence that the score was not reasonable and consistent with normal expectations of acceptable proficiency in firefighting.
IV. Remedy. The Court has considerable discretion to devise appropriate relief where employers are found in violation of Title VII, 42 U.S.C. § 2000e-5(g). In this case the appropriate relief is that:
A. The City of Evanston shall be prohibited from using a physical agility test in a way that unlawfully discriminates against female firefighter applicants. Use of the same physical agility test at issue in the proceeding for employment decisions shall be permissible only if this Court approves in advance the establishment of the cut-off score. Notice of intent to seek such approval shall be given to plaintiff's counsel. If the City seeks to devise a new physical agility test, it shall seek court approval in advance of its use and shall notify plaintiff's counsel of its intent to seek such approval. Provisions requiring notification of counsel shall lapse as of December 31, 1990.
B. Under the rule which favors awards of backpay, Thomas v. City of Evanston, 610 F. Supp. at 432, the Court orders that all members of the plaintiff's class shall share in an award of backpay, to be distributed on an aliquot basis and compiled on the assumption that, but for the discriminatory test, the class members would have been hired in percentages that equaled their percentage of those taking the test, that is 4.68%. The award would then be 4.68% of the total wages earned by the nine males who were hired as a result of the 1983 test.
The Court has jurisdiction over the parties and over this controversy, and reserves jurisdiction to enter such orders as may be necessary to implement its decision. Accordingly, judgment shall be entered in favor of the plaintiff class and against the City of Evanston.
The plaintiff's request for priority for hiring of class members as firefighters subject to a showing of qualification is rejected as vague, neither priority nor qualification being defined. If by priority plaintiff means its class members are to be taken above all other candidates the remedy asks for more than they would have received if the test had not been used. Only 1.1% of male applicants were hired and only 1.2% of males who passed the test were hired.