Use of the CAPS for Predicting Employee Performance
One of the main reasons companies use aptitude testing is to make better hiring and promotion decisions. Tests are often much better than interviews in predicting whether a person has the potential to do a job well. When designed properly, aptitude tests can fairly and objectively compare the potential of different candidates. One of the most important aspects of a test is that it is both valid and reliable, so you can be assured of a fair process and that your hiring practices are legally defensible if challenged. The Career Ability Placement Survey (CAPS) is a comprehensive, multi-dimensional battery designed to measure abilities related to performing a job. Typically, this test is used in conjunction with values and interest assessments in the COPSystem VIA package, but it can also be used by organizations to support hiring decisions. The following presents a case study of an organization that used the CAPS for employee selection and shows that it differentiated between high and low performers.
Job performance is one of the most important outcomes at work and has been defined as the measurable proficiency of work behavior that is under employees’ control and contributes to organizational goals (Campbell, McCloy, Oppler, & Sager, 1993). Since performance ratings are associated with employees’ salary and promotion, studying their predictors is of high interest for both organizational researchers and practitioners. Schmidt and Hunter (1998) established mental ability as one of the best predictors of overall job performance. In another meta-analysis on the predictive validity of specific cognitive abilities for job performance, Bertua, Anderson, and Salgado (2005) distinguished between seven occupational groups (clerical, engineer, professional, driver, operator, manager, and sales) and their results showed significant validities for predicting job performance within all those groups. As mental ability has been shown to be a predictor of job performance, we expected that it would be a strong predictor of job performance ratings.
Research on the main aspects of mental ability has often consisted of the following characteristics: learning, problem solving, information processing, and reasoning. For example, Gottfredson (1997) described mental ability as “the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience” (p. 13). According to Jensen (1989), learning has occurred when a change in a specific response to a given stimulus, situation, or problem was observed. Problem solving can be defined as successfully transferring a given actual state to a target state by overcoming barriers (Dunbar, 1998). Information processing involves transforming information and creating new information (Oberauer, Süß, Wilhelm, & Wittman, 2003). Finally, reasoning “is a process which may occur at any point in a thought-movement and consists in the appreciation of likeness and differences between old experiences and a new situation” (Skaggs, 1930, p. 439). Each of these mental abilities can be measured individually to consider a person’s aptitude, but the optimal use is for them to be used as part of a comprehensive battery to measure ability level related to work potential.
The Career Ability Placement Survey (CAPS) helps examinees relate their current levels of ability to career clusters. It is a comprehensive, multidimensional battery designed to measure abilities that are related to performing a job. There are eight ability dimensions keyed to entry level requirements for occupations in each of the 14 COPSystem VIA Career Clusters. Measures included in the CAPS are mechanical reasoning, spatial relations, verbal reasoning, numerical ability, language usage, word knowledge, perceptual speed and accuracy, and manual speed and dexterity. Scores are combined from each individual test depending on skill requirements for each career cluster to determine level of ability within each career cluster. The Technology, Professional career cluster is comprised of scores from the mechanical reasoning, spatial relations, verbal reasoning, numerical ability, and word knowledge tests. The CAPS is a valid and reliable battery with test-retest reliability coefficients ranging from .70 to .95. To establish concurrent validity, correlations with the General Aptitude Test Battery were obtained and ranged from .63 to .80 between conceptually similar tests. Correlations were obtained between CAPS tests and grades in specific subject areas. These correlations ranged from .30 to .60 between the CAPS tests and the subject to which it was most closely related. For example, the CAPS numerical ability test had the highest correlations with grades in math. These results are significant and demonstrate the validity of the CAPS. Predictive validity studies show that ability scores are significantly related to subsequent career choice and may also predict job-performance associated with that career choice (Knapp, L., Knapp, R. & Knapp-Lee, 2009).
Employee data from an organization were analyzed to assess the effectiveness of the CAPS as a selection tool in predicting job performance. A sample of 128 employees were included in the analysis. All employees had completed the CAPS and had supervisor job performance ratings which were based on a scale of 1-5 (1 = highest performers, 5 = lowest performers; scores were then reverse coded so that higher rated employees received higher scores for analysis). Employees were categorized into one of three groups. In order to distinguish between high and low performers the top two tiers, rated as 1 and 2, were combined (N = 40), and the lowest two tiers, rated as 4 and 5, were combined (N = 29). Average rated employees, rated as 3, (N = 61) were their own category (see Table 1 for means and standard deviations of test scores). Predictive validity of the scores on the CAPS was assessed by comparing categories of employee job ratings based on their CAPS scores.
The CAPS scores are presented in two profiles. One profile the CAPS Ability Profile reports scores in terms of individual scores on each of the eight subtests. The other report, the CAPS Career Profile, compares the CAPS scores to probability of success in terms of ability within a career cluster. When examining the scores on each of the eight CAPS subtests, the highest rated employees had higher average scores on all the eight CAPS tests than the average rated employees and the lowest performers. Additionally, the employee data from the CAPS Career Profile were analyzed to determine how well the CAPS predicted success within a career cluster. All the employees listed job titles were classified within the Technology cluster. Specifically, examining the Technology, Professional cluster, the high achievers were all predicted to succeed in that occupational cluster whereas, the middle and low performers were below the cutoff score for success in that cluster. In addition, when examining the scores for each group across each career cluster the higher rated employees had uniformly higher scores than both the medium and low performers.
A linear regression was calculated to predict employee rating scores based in Technology Professional scores. A significant relationship between employee rating and Technology Professional scores was found, F(1, 127) = 88.302, p < .001 (see Table 2). Technology Professional scores accounted for 41.2% of the employees’ performance rating (see Table 2). Technology Professional scores significantly predicted employee rating, as Technology Professional scores increased so did employee rating scores. A univariate analysis of variance compared employees rated in performance levels 3-5 as compared to employees rated at levels 1-2 on the Technology Professional cluster. The analysis showed a significant difference between the groups ranked as lower performing as compared to the groups ranked as higher performing F(1, 128) = 67.14, p < .001 (see Table 3). The employees that had performance ratings of 1 and 2 were significantly higher than those rated at 3,4,5 showing that the CAPS is a good predictor of success on the job. A second analysis showed a significant difference between high rated employees (1-2) and average employees (3). High rated employees scored higher on the Technology Professional cluster than average rated employees, F(1, 128) = 26.458, p < .001 (see Table 3).
The purpose of the current study was to evaluate the CAPS when used as a selection tool for employee performance. An organization provided employee performance scores as rated by the employee’s supervisor, and CAPS scores for each employee. The current study provides evidence that the CAPS can be used to predict job performance, specifically when the individual tests are combined to match a profile that aligns with the ability levels and requirements of job categories. Analysis of the employee performance data and CAPS scores show additional validity that the CAPS measures job-related abilities that are relevant and applicable to the Technology, Professional career cluster, and can be used in other settings such as different career clusters.
References
Bertua, C., Anderson, N., & Salgado, J. F. (2005). The predictive validity of cognitive ability tests: A UK meta-analysis. Journal of Occupational and Organizational Psychology, 78(3), 387-409.
Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. Personnel selection in organizations, 3570, 35-70.
Dunbar, R. I. (1998). The social brain hypothesis. Evolutionary Anthropology: Issues, News, and Reviews: Issues, News, and Reviews, 6(5), 178-190.
Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79-132.
Jensen, A. R. (1989). The relationship between learning and intelligence. Learning and Individual Differences, 1(1), 37-62.
Oberauer, K., Süß, H. M., Wilhelm, O., & Wittman, W. W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31(2), 167-193.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.
Skaggs, E. B. (1930). The essential nature and definition of reasoning. The Journal of General Psychology, 3(3), 435-442.
Table 1. Means and standard deviations for the two CAPS cluster scores of Technology and of each CAPS subtest score by employee rating.
Table 2. Regression results of employee performance predicting Technology Professional scores.
Table 3. ANOVA results comparing high vs low rated employees, and high vs average rated employees.
