Job Validity Studies
American Psychological Association
U.S. Department of Labor
Equal Employment Opportunity Commission (EEOC)
Canadian Psychological Association (English and French)
The Public Service Commission of Canada
Research & Federal Compliance
In order for a personality assessment survey be acceptable under the standards and guidelines of the American Psychological Association, the U.S. Equal Employment Opportunity Commission (EEOC), the Canadian Psychological Association, and the British Psychological Society various steps have to be taken.
First, the items, whether they are phrases, words, questions, pictures, etc., have to measure what they purport to measure – they must be valid. For example, is choosing the word “self-confident” to describe oneself actually a measure of the personality trait ego-centeredness? Does someone who prefers to go to a party rather than read a book more of an extrovert than an introvert? Men pick red as their favorite color more often than woman. Does this mean something? Test developers have to use a statistical technique called factor analysis to sort and categorize items.
Also, the items or words have to be investigated for any bias. In other words, is an item more attractive or unattractive to a specific group – gender, race, ethnicity? For example, men often equate the word “aggressive” with the word “assertive”. Women, however, more often see these words as having very different meanings. Biased items should be eliminated or accounted for in test construction. Assuming that an item measures a particular trait (construct) or is unbiased is insufficient. Documentation should be provided. If you believe biased testing practices have affected you, it’s crucial to gather evidence and file a time-sensitive discrimination claim to protect your rights and challenge discriminatory practices.
Second, the items, both individually and collectively, have to be reliable. That is, the items are stable and are not subject to whim, mood, or other easy influences. For example, the word “happy” in describing oneself says nothing about the trait makeup of an individual. He or she may be happy this morning and, due to a minor incident, be unhappy in the afternoon. Another example, a multiple-choice arithmetic test given to a class of 6th graders generates an average test score of 80% on Monday. On Friday the same test generated an average score of 60%. Obviously, the test is not measuring actual student knowledge; they are guessing. Again, reliability coefficients (.70 is the generally accepted cut-off number) should be published.
Third, the items or, once grouped, constructs must have a practical value; they must be job relevant. The Supreme Court of the United States has repeatedly affirmed the legality and value of “testing” starting in 1971 with Griggs v. Duke Power Co. In that decision the Court stated that as long as test results correlated with job performance they were not only permissible but also “desirable”. A test or survey that measures irrelevancies or which are prohibited by law (e.g. “What race are you? Do you plan on having children?”) clearly should not be used. OAD conducts job validity studies for its clients; statistically correlating traits and job behaviors with relevant performance measures.
Publishing factor analyses, validity, and reliability results do not compromise trade secrets. Reputable test developers publish their findings and don’t hide behind a veil of “proprietary information”. Test publisher claims that “over 95% of our test results are valid” is nonsense. They are simply stating that most people agree to some extent that the results are accurate. This is face validity and is not an acceptable substitute. Finally, the number of translations of tests provided is not a mark of acceptability. Translations have to be separately validated as words and phrases take on different meanings in different cultures.
Click below to download “The OAD Survey – Taxonomy of General Traits” document :
For additional information, please contact Paul Chadwick at pchadwick@oadllc.com or 262-369-0987.