Time Accommodations and the CAPS
Each of the eight CAPS ability battery subtests are administered with a strict five-minute time limit. The time limit adds an extra dimension to how the CAPS is interpreted- not only is it an assessment of a test-taker’s abilities in each category, but also a measure of how well a test-taker can process information in a quick and accurate manner. For test-takers with certain disabilities, conventional time limits may present an undue encumbrance. For those individuals that require an accommodation, EdITS offers time-and-a-half (7.5 minutes per subtest) and double-time (10 minutes per subtest). In accordance with many other strictly timed psychological assessments, test-takers are not expected to answer every question within each subdimension of the CAPS. To determine if these time accommodations present an advantage, statistical analyses were employed to assess the average score differences between the standard, time-and-a-half, and double-time allocations.
Score information from 44,222 US test-takers was assessed and is presented in Table 1. For the sake of simplicity, level of education was ignored in these analyses. Future analyses could parse out any time-based advantages within or between specific grade levels. As sample sizes were unequal across time accommodations, homogeneity of variance could not be assumed. Accordingly, rather than an ANOVA, Welch’s t-tests were utilized to determine the statistical significance of the mean differences in question. Afterwards, the effect size, or “practical significance,” of each statistically significant mean difference was calculated. While measures of statistical significance determine if a particular set of group differences are (or are not) due to chance, measures of effect size illustrate the magnitude of a group difference.
In terms of effect size, an advantage based on time allocation was only observed in two subtests: Verbal Reasoning (VR) and Perceptual Speed and Accuracy (PSA). It should be noted that these advantages were only observed between the time-and-a-half group and the standard time group and were of medium magnitude (d = -.29) for the VR and of large magnitude (d = -.92) for the PSA. It is not surprising to observe a time-based advantage for these two subtests. The Verbal Reasoning subtest can more accurately be described as a test of logical reasoning. Test-takers are given a passage and must determine if a possible conclusion about that passage is true, false, or uncertain. When allotted extra time, test-takers are able to read and reread the passages more carefully, granting them extra confidence in their conclusions. The Perceptual Speed and Accuracy subtest requires test-takers to point out small differences within short sets of numbers, letters, and/or symbols. As shown in Table 1, the column labeled “% of Correct Attempted Questions” illustrates that test-takers across time allocations correctly answer similar proportions of questions. Those receiving time-and-a-half simply answered more questions than those receiving five minutes.
In conclusion, these time-based advantages were only observed when comparing the time-and-a-half group to the standard time group. Any other existing practically significant differences between time allocations were observed in the other direction. Simply put, apart from the VR and PSA subtests, those receiving extra time either performed as well or worse than those under the standard time limit. It should be noted, however, that these results are wholly dependent on the samples (and the test-providing organizations within them) that the data were extracted from. Rather than random assignment, the extra-time subjects were placed in each group according to the accommodation required for their disability, but the details behind each accommodation are unknown. In sum, these observations support the notion that nearly all the temporal accommodations are functioning as they should, rather than offering any undue advantages. However, extra care should still be taken when interpreting subtest scores in Verbal Reasoning or Perceptual Speed and Accuracy for those receiving a time-and-a-half accommodation.
Table 1 — Summary of CAPS Subtest Score Means Across Time Allocations (N = 44,222)
