Opinion
No. 70 1335.
May 9, 1973.
Robert L. Gnaizda, Mario Obledo, Miguel A. Mendez, Mexican-American Legal Defense and Educational Fund, San Francisco, Cal., for plaintiffs.
Thomas M. O'Connor, City Atty., Michael C. Killilea, Asst. City Atty., San Francisco, Cal., Barry S. Jellison, Davis, Cowell Bowe, San Francisco, Cal., for Fire Fighters Union.
William J. Murphy, San Francisco, Cal., for Intervenors Robert Cutone and others.
MEMORANDUM OF DECISION
Pursuant to this court's previous orders of January 8, 1971, D.C., 330 F. Supp. 536, February 7, 1972, D.C., 340 F. Supp. 1351, February 17, 1972, March 7, 1972, July 20, 1972, and the court's most recent order of January 18, 1973, hearings were held on February 1st, 12th and 13th to determine whether defendants have now complied with the requirements of the Civil Rights Act of 1964 (Title VII) and the Guidelines on Employee Selection Procedures (Fed.Reg. Title 29, Chap. XIV, §§ 1607.1-1607.14) issued thereunder in the preparation of a proposed written Civil Service examination for the position of Fireman H-2.
Although these Guidelines were not originally applicable to public agencies, this court (See Order of February 17, 1972) adopted them by analogy insofar as they could be reasonably complied with by a public agency. Meanwhile, the Act has been amended, 86 Stat. 103, March 24, 1972, to make them officially applicable to public agencies as well as private business.
Court's Exhibit "A" lodged with the court under seal for in camera use only.
This issue is presented because the Supreme Court of the United States in Griggs v. Duke Power Co., 401 U.S. 424, 91 S.Ct. 849, 28 L.Ed.2d 158 (1971), a case in which the employer contended that, absent an intent to discriminate, a "professionally developed ability test" could not be held to be an unlawful employment practice, rejected that contention and interpreted the Civil Rights Act of 1964 to the effect that, even when the employment test has been professionally developed without any intent to discriminate, the test is not to be given controlling force if it in fact operates to disproportionately exclude a racial group — unless it is "demonstrably a reasonable measure of job performance," adding "more than that, Congress had placed upon the employer the burden of showing that any given requirement must have a manifest relationship to the employment in question (p. 432, 91 S.Ct. p. 854). . . . If an employment practice which operates to exclude Negroes, cannot be shown to be related to job performance it is prohibited (p. 431, 91 S.Ct. p. 853). . . . The Equal Employment Opportunity Commission, having enforcement responsibility, has issued guidelines interpreting § 703(h) to permit only the use of job related tests. The administrative interpretation of the Act by the enforcing agency is entitled to great deference . . . Since the Act and its legislative history support the Commission's construction, this affords good reason to treat the guidelines as expressing the will of Congress." (p. 433, 91 S.Ct. p. 854). (See also, the recent case of United States v. Georgia Power Co., 474 F.2d 906 (5th Cir. 1973); Bridgeport Guardian v. Bridgeport Civil Service, 354 F. Supp. 778 (D.C.Conn. 1973).
It should be emphasized that the issue thus presented is not whether the City has the right to give an examination on some subject or other to applicants for Fireman H-2; certainly the City has that right; the narrower issue here is only whether the City has shown the presently proposed written examination to be "demonstrably a reasonable measure of job performance," as required by Griggs, i.e., whether it has been "validated" as "job related" within the meaning of the Guidelines issued pursuant to the Civil Rights Act of 1964.
This issue must be considered and resolved in the light of this court's earlier findings of January 6, 1971, that the City's 1968 selectivity testing for Fireman H-2 (resulting as of 1971 in a Department of 1800 men with only 4 Negroes — although Negroes comprise over 14% of the City population) has been presumptively discriminatory against minorities within the meaning of Griggs, supra.
Of the applicants who took the 1968 Fireman H-2 written examination 37% of whites, but 12% of Negroes passed.
The only alteration of this Fire Department composition (recent entry of 21 minority firemen, including 15 blacks, into the Department) was accomplished, not through the City's Civil Service process, but under a special, temporary federally funded program (called the Fireman Safety Technician Program) designed to overcome normal Civil Service requirements by helping minority applicants prepare themselves for eventual classification as H-2 Firemen.
The present issue must also be considered in the light of the fact that on February 17, 1972, this court was obliged to again hold that a subsequent written examination given for Fireman H-2 in September, 1972, purporting to correct the situation, did not do so because the City's own witnesses conceded that it had been prepared without the job analysis required by the federal Guidelines.
This case must also be considered in light of the fact that at this point the burden of proof that the City's selection test for Fireman H-2 has been validated as not discriminatory against minorities rests, not upon plaintiffs herein, but now upon the City itself. Griggs, supra.
In February of this current year the City presented its most recent proposed written examination which it now claims to be based on a proper job analysis and, further, properly "validated."
The City gives to applicants for Fireman H-2 three kinds of tests: two of them, one an athletic (physical) test (weighted 50%) and the second test an oral examination (weighted 50%) are called rating tests because on the basis of their passing scores on these two tests (and only these two tests) applicants advance to, are positioned on and are called from the Fireman H-2 Civil Service eligible list.
A third test, a written examination, which is the only part of the selection procedure challenged by plaintiffs in these proceedings, is required by the city but is used, not as a rating test like the athletic and oral tests upon which applicants proceed to the eligible list, but as a so-called pass-fail cut-off test under which applicants who fail to correctly answer an arbitrarily fixed number of questions, e.g., 70 out of 100 questions, are wholly disqualified from even taking the athletic and oral tests upon which other applicants are rated for the Fireman H-2 Civil Service eligible list. The presently proposed pass-fail, cut-off written examination consists of 100 questions — 60 questions claimed to test "written communication" and 40 questions claimed to test mechanical aptitude.
THE HEARING
At the hearings three witnesses called by the City — all members of the faculty, Department of Psychology, University of California and all claiming to be knowledgeable in the field of employee selection and the requirements of the Civil Rights Act and Guidelines — testified in substance and effect that they were engaged by the San Francisco Civil Service Commission to conduct a job analysis concerning the job of Fireman H-2 and to prepare a written examination to be given to applicants for that position in addition to other athletic and oral tests; that they did conduct such a job analysis and found that 10 traits or skills were essential to the Fireman H-2 job; that they chose two of these skills — written communication and mechanical aptitude — as appropriate for testing by means of written examination; that they then prepared the proposed written examination which in their opinion is "content validated" (as distinguished from "empirical validation" (see footnote (5)), within the meaning of the Act and the Guidelines for the two skills it is designed to test; that they made no attempt at "empirical validation" because in their opinion "empirical validation" was not "feasible."
Three other claimed expert witnesses testified: one the Chief Research Study Director, Equal Employment Opportunity Commission, Washington, D.C., called by the court upon request of both sides (this witness, under agreement of the parties, had read the proposed written examination); also two other witnesses on behalf of plaintiffs, a Senior Psychologist, Graduate School of Psychology, University of California, and another, an associate with Educational Testing Service, a private non-profit service.
All three of these last mentioned witnesses testified in substance and effect that they had familiarized themselves with the affidavit testimony of the City's experts; that in their opinion the job analysis conducted by the City's professionals was for variously stated reasons wholly inadequate and, further, that so-called "content validation" of the proposed written examination was impermissible because "empirical validation" was "feasible" and any written examination of the kind in question should be "empirically validated;" further, that in any event said written examination could not be deemed to be even properly "content validated" within the meaning of the applicable Guidelines.
On the issue of the adequacy of the job analysis, the evidence is in conflict but the court finds from the evidence, considered as a whole, that the "job analysis" was conducted by the City's professionals substantially in accordance with the Guidelines.
On the remaining question, i.e., whether the proposed written examination, prepared on the basis of that job analysis, has been properly validated as job related, within the meaning of the Guidelines, the evidence is also conflicting.
On this subject of validation the Guidelines provide (Section 1607.4) that " where technically feasible a test should be validated for each minority group with which it is used, that is, any differential rejection rates that may exist based on a test must be relevant to performance on the job in question" and, further, (Section 1607.4) that "evidence of a test's validity should consist of `empirical' data demonstrating that the test is productive of or significantly co-related with important elements of work behavior which comprise or are relevant to the job or jobs for which candidates are being evaluated;" also (Section 1607.5) that this "empirical" evidence in support of a test's validity "must be based on studies employing generally accepted methods for determining criterion related validity." (emphasis added).
Although the Guidelines (Section 1607.5) provide that evidence of "content" validity alone may be acceptable for well-developed tests (See, Davis v. Washington, 352 F. Supp. 187 (D.C.D.C. 1972), they also provide that this method of validation is appropriate only "where criterion-related (i.e., empirical) validity is not feasible" and then only "when accompanied by sufficient information from job analysis to demonstrate the relevance of the content" and, further, when the tests "consist of suitable samples of essential knowledge or skills composing the job in question," provided also that any such tests are not to "include knowledge or skills which can be acquired in a brief orientation on the job." (emphasis added).
These terms "empirical validation" and "content validation" are used in the official Guidelines and have become words of art in this developing field of examining employee selection procedures.
"Empirical" means "Relying upon experience or observation alone; originating in or based on observations or experience; capable of being verified or disproved by observation or experiment. " — Webster Dictionary.
So, empirical validation in this particular field means that, when the scores made by an applicant group on particular initial tests are found to actually square with the on-the-job performance tests of the same group, there is empirical validation of the job-relatedness of the initial tests.
"Content" validation, as distinguished from "empirical" validation, means that the test is validated by the fact that it is made up only of actual "samples" of the job to be done, e.g., typing for a typist job; sewing for a seamstress job; piano playing for a pianist job — without any attempt to prove for general knowledge or skills which might be beyond the job requirements and to that extent might constitute over testing.
The Guidelines also provide (Section 1607.4) that "The term `technically feasible' as used in these Guidelines, means having or obtaining a sufficient number of minority individuals to achieve findings of statistical and practical significance, the opportunity to obtain unbiased job performance criteria, etc. It is the responsibility of the person claiming absence of technical feasibility to positively demonstrate evidence of this absence." (emphasis added).
FEASIBILITY OF EMPIRICAL VALIDATION
The City gives several reasons for its claim that empirical validation is not feasible, e.g., (1) that a Charter provision, requiring that Civil Service tests be opened for inspection after the examination to permit applicants to prepare protests, would make it impractical to use an examination more than once; (2) that procedures for bringing about empirical validation might involve hiring people as firemen who lack the ability to perform.
It is obvious, of course, that post examination opening for inspection prevents repeated use by the City of identical question tests; however, an examination, once validated, could be used again, if not with identical questions, at least as an examination of the same, general type. This is what the City has been doing for years.
Nor would there be any significant risk of hiring unqualified firemen for San Francisco's long highly regarded Fire Department; the City always places new firemen (who, incidentally, must be at least high school graduates), on a probationary basis for one year after hiring (including an initial 8 week Fire College on-the-job testing and training period) within which it can at any time terminate any fireman who cannot measure up.
Further, the city has already recognized that "empirical" validation of its Fireman H-2 testing procedures is, not only feasible, but also necessary.
John De Soto, Director of Recruitment and Examinations for the San Francisco Civil Service Commission, who has responsibility for recruitment, examination development and certification of eligibles for employment testifying by his affidavit of 11/24/71 in these proceedings, recognized the need for just such eventual "empirical" validation of the Fireman H-2 Civil Service examination, stating: "The empirical validity determinants will take a substantial period of time. Since the present list will last at least two and perhaps four years, we fully intend to conduct an exhaustive review of the test to analyze and confirm its empirical validity. The purpose of this study will be to apply what is learned from the validity study in developing the next fireman's examination and to confirm our opinion that the current test is indeed valid." (emphasis added).
The real reason why the City's testing procedures have not been heretofore empirically validated is that the City has thus far made no attempt to do so. There is a variety of methods for empirically validating Civil Service tests. One method, called concurrent empirical validation, is to prepare types of examinations, try them out on a representative group of firemen within a fire department and compare their scores with their actual on-the-job performance tests; another method is to create a group of new applicants whose scores on an initial test can be similarly compared with their actual on-the-job performance tests during a probationary period.
A program is now in progress in which most of California's major cities, cooperating with California's State Personnel Board, are developing just such an empirically validated type of civil service examination for local police and fire departments.
For the foregoing reasons we conclude that the City has not met the requirement of the Guidelines that "It is the responsibility of the person claiming absence of technical feasibility to positively demonstrate evidence of this absence;" accordingly, the court finds that empirical validation of the City's Fireman H-2 selection procedure has been and is, or can readily be made "feasible" and that, therefore, empirical validation, rather than mere "content validation," must be required.
CONTENT VALIDITY
Even assuming that "empirical" validation were not feasible and that the alternative "content" validation would be acceptable, the question remains whether the proposed written examination has even been shown by the City to be "content validated" within the meaning of the Guideline requirements of "samples of essential knowledge and skills composing the job in question." The proposed test is not entirely limited to samples of knowledge and skills composing the job in question; it includes questions designed to test, not job samples, but skill in general, e.g., mechanical aptitude questions concerning ability to note elementary principles of physics — pressures, balances, etc. Further, many of the questions arguably call for "knowledge or skills which can be acquired in a brief orientation on the job" within the meaning of Guideline Section 1607.5, supra, and which, therefore, should not be included.
Further, the proposed examination does not meet the content validation requirement that it must consist of samples "composing the job in question" because, admittedly, it purports to test only two skills — written communication and mechanical aptitude — which, according to the evidence adduced at the hearings, are admittedly only two out of ten, and by no means, the most important skills and traits related to the Fireman H-2 job — an obvious over-emphasis on the two skills tested in the written examination.
This is important where, as in this case, notwithstanding the requirements of Guideline Section 1607.6 that cut-off scores "must be reported" and must be "reasonable and consistent with normal expectations of proficiency within the work force or group on which the study was conducted;" the City insists on reserving the fixing of a "cut-off" (passing) score for this written examination until after it has been given and then to fix a cut-off score on the single, arbitrary basis of what the City's need for fireman happens to be at some particular time — a factor which obviously has no bearing on whether such a cut-off, whatever it might be, would be "reasonable and consistent with normal expectations of proficiency within" the Fire Department. (See Guidelines Section 1607.6).
This is particularly important where the City proposes to use this written examination (unlike the athletic and oral examinations upon the basis of which applicants are rated for eligibility position on the Civil Service list), not for rating applicants on the Civil Service list, but to render wholly ineligible any applicant who fails to correctly answer the arbitrarily required number of written examination questions.
The result of such procedure would be that an applicant, who passes high, even No. 1, on all other tests (e.g., athletic and oral tests) and who possesses to a high degree other skills and traits found essential for the Fireman H-2 job, would be completely disqualified from becoming a fireman if he happens to have correctly answered one less than the arbitrarily fixed number of written examination questions dealing with only 2 out of 10 essentials.
The effect of such a test could be to provide the City of San Francisco, not necessarily with the best firemen, but with the best reader analysts and the best (elementary) physicists (pressures, balance, etc.) and to exclude others who might be, not only sufficiently capable in these fields to meet on-the-job fireman requirements, but also capable of performing in the other essential skills and traits as well as, or perhaps better than, applicants passing the written examination.
The very purpose of the Civil Rights Act of 1964 ( 42 U.S.C. Sec. 2000e-12) and of the Supreme Court ruling in Griggs, supra, and of the Guidelines is to make sure that only "job-related tests will be given and that tests, which go beyond the actual requirements of the job, will not be given. (See also, the recent case of United States v. Georgia Power Co., 474 F.2d 906 (5th Cir. 1973)). The reason for this requirement is that, otherwise, so-called job tests can be, and often have been used for (or at least have had the effect of) excluding applicants, who although well able to perform the job, have not had the opportunity to acquire a superfluity of learning and proficiency which, although desirable in itself, is more than is necessary for the particular job.
That is why the Guidelines, Section 1607.6, provide that any cut-off (passing) score must "be reasonable and consistent with normal expectations of proficiency within the work force or group on which the study was conducted."
CONCLUSION
Upon consideration of the evidence as a whole, quantitatively and qualitatively, the court finds, after full hearing, that the City's evidence on this issue of "validation" does not preponderate; at very best, the evidence is no more than evenly balanced and, therefore, the court is bound to resolve the issue against the party having the burden of proof on the issue — in this case the City defendants; further, for reasons hereinabove set forth, the City has failed to make a sufficient prima facie case of validation to justify approval of the proposed written examination on a provisional basis subject to further hearings which would entail long continued litigation and create more uncertainty among applicants for Fireman H-2. The time has come — especially after the City has twice failed to satisfactorily demonstrate the validity of its Fireman H-2 written examination tests — for the City to seriously, rather than superficially and speciously, face up to the problem presented to it by new federal law by further efforts to rebut or correct the already legally established presumptive discriminatory effect of its selection procedures.
Plaintiffs contend that the court should now impose requirements which would be, in effect, a quota system for hiring only minority group firemen, or a fixed percentage thereof, until minority firemen representation in the Department is brought into line with minority group population in San Francisco.
The court, however, considers that this extreme type of relief is unnecessary at this time. In the first place, the law does not require that all employment must proportionately reflect racial population components; it only requires that no minority job applicant (and for that matter no white job applicant), otherwise sufficiently qualified or readily qualifiable to perform a job, shall be, in effect, disqualified on the basis of an examination that goes beyond the requirements of the job. A quota system should be avoided, even if legal, unless necessary as a last resort. See, re legality of such affirmative remedies, e.g., Carter v. Gallagher, 452 F.2d 315, 327 (8th Cir. 1972); Castro v. Beecher, 459 F.2d 725 (1st Cir. 1972).
In the pending case the objective of eliminating existing rejection rates, which are adverse to minorities and presumptively attributable to the use of insufficiently validated selection procedures, can be obtained by deferring the use of the written examination as an arbitrary cut-off device until it has been once and for all empirically validated as really job related within the meaning of the Guidelines.
Since, as already noted, the City does not use that written examination for advancing and positioning applicants on the Civil Service eligible list, this deferment works no drastic change in the City's existing procedure of using only scores on the athletic and oral tests for forming the Civil Service eligible list.
It does not follow that the giving of the examination by the City, if it wishes to do so, may not serve a useful purpose.
First, since the proposed written examination purports to be a new and different examination which has not yet been given, its actual effect upon minority, as opposed to non-minority applicants is not yet known and can be conclusively determined only by giving the examination to the present Fireman H-2 applicants on a provisional basis. If, perchance the results of the new examination surprisingly show that this written test does not in fact work adversely against any minority group, as have the written examinations in the past, the City may wish to move for reconsideration of the court's present ruling on the ground of newly obtained and not previously available evidence.
Secondly, the giving of the proposed examination to the present Fireman H-2 applicants is one way to obtain a test group whose scores thereon can be compared with actual on-the-job performance of the same applicants and thus provide a basis for the empirical validation study which must be made in any event.
ORDER
Upon the findings and for the reasons hereinabove set forth the court orders that the proposed written examination may be given and used by the City only upon the following conditions:
(1) The proposed written examination shall not for the present be used by the City defendants as a cut-off device or otherwise, to disqualify Fireman H-2 applicants from taking or being rated on the oral and athletic tests or from advancing, according to the City's present procedures, to a position on the Fireman H-2 Civil Service list on the basis thereof unless and until there is a further finding by this court (i) that the proposed written examination, or some other written examination in lieu thereof, has been empirically validated on the basis of studies employing any generally accepted method for determining criterion related (empirical) validity (see Guidelines §§ 1607.4, 1607.5, 1607.6, et seq.), or (ii) that, pending such empirical validation of the City's selection procedures, the City has worked out an acceptable cut-off score for the proposed written examination fixed at such a point and in such a manner as will assure no adverse effect of the examination on minority applicants.
(2) The City shall proceed with reasonable diligence to empirically validate its Fireman H-2 selection procedures and in this connection may give and use the proposed written examination, or another proposed examination in lieu thereof, for the limited purpose of providing a group of applicants whose scores thereon may be useful in connection with such eventual empirical validation.
(3) Meanwhile, in order that there shall be no interruption or delay in meeting manpower requirements, the City will continue to fill its current and developing Fireman H-2 manpower requirements from a Civil Service eligible list to which present applicants will proceed in accordance with the City's already existing practice of placement thereon on the basis of their scores on the athletic and oral tests (as already authorized in this court's previous order of July 20, 1972).
(4) In order to assure continued high quality and efficiency in the Department, nothing in this order shall be deemed to impair the right of the City to terminate any Fireman H-2 who, during the initial 8 week Fire College on-the-job training and testing period or during the one year probationary period, or at any other time allowed by law, is found to lack any job related knowledge, skill or trait.
(5) This order is made pursuant to the power and responsibility of this court to require demonstration by the City that all of its Fireman H-2 selection procedures comply with the Civil Rights Act of 1964, the Guidelines issued thereunder and the principles enunciated in Griggs v. Duke, supra, and the court reserves power to make such further orders herein as may be necessary or advisable to effectuate this purpose.