International Journal of Education and Psychological Research

(Print and Online Peer Reviewed Journal)


Print - ISSN: 2349 - 0853
e - ISSN: 2279 - 0179

VOLUME 7 - ISSUE 1

(March 2018)

Investigating the Structural Aspect Validity of a Large-Scale Standardized Achievement Test Using Rasch and CFA of Item Parceling Analysis


Authors: Amjed A. Al-Owidha

Pages: 32-45

Abstract:

This study aims to collect evidence on the structural aspect validity of Standardized Achievement Admission Test (SAAT) scores. Over a thousand examinees were randomly selected and their responses used in this analysis. Two approaches were used to investigate the structural aspect validity of SAAT test scores; Rasch analysis and confirmatory factor analysis (CFA) of item parceling. First, Rasch analysis was applied to the SAATtest scores. Several Rasch indices were used. Overall fit statistics, item fit statistics, the person reliability index, point–measure correlations, principle components analysis of Rasch residuals, and differential item functioning were computed and used in the analysis. Second, the CFA of item parcels was applied to the SAAT. Several fit statistical indices were used to evaluate the fit of the measurement model to the SAAT data. Specifically, the Chisquare test, root mean square error of approximation, goodness of fit index, comparative fit index, Tucker–Lewis index, Standardized mean square residuals, and Akaike Information Criterion were all used to further investigate the structural aspect validity of the SAATtest scores. The findings of both paradigms lend support to the hypothesized structure of the SAATtest. Also, as an attempt to evaluate the plausibility of the current position of English items as being one major component of the SAAT composite score, both frameworks showed an agreement supports the notion that the English items of the SAATare measuring mastery level of English proficiency rather than high school achievement. However, given the fact that the current study was applied only to the SAATtests for male science majors; we suggest conducting further studies that include more representative sample of the SAATpopulation