University of Illinois at Urbana-Champaign

Software Available at Assessment Systems Corporation

• DIMTEST. DIMTEST assesses lack of unidimensionality, operating in either a confirmatory mode (DIMTEST assesses a user-selected set of items a priori judged to be possibly dimensionally distinct from the rest of the test) or exploratory mode (DIMTEST, using cross-validation, assess es a statistically selected set of items that may be dimensionally distinct from the rest of the test, the selection carried out using factor analysis, cluster analysis, DETECT, or other data driven method). A modified version can be used to assess polytomous item data (ask for Poly DIMTEST).

• Hierarchical Agglomerative Clustering (HAC). This program performs a hierarchical cluster analysis. It searches for a partition into clumps of items such that the clumps tend to display approximate uni-factor (one dimension/item) structure. (E.g., the different paragraph-based item clumps in a reading comprehension test usually display such approximate uni-factor structure.) It does so by means of a multidimensionality-sensitive customized proximity measure (PROX). Because HAC proposes many multiple-clump item partitions simultaneously, DIMTEST can be useful in assessing which of the proposed clumps of the partitioning hierarchy most likely represent a good approximation to the true underlying multidimensional structure. HAC and PROX can be used to assess either dichotomous or polytomous item data.

• DETECT. The DETECT index is based on the signs and magnitudes of estimated conditional item pair covariances, given observed score on the remaining items. DETECT may be used in either a confirmatory or exploratory mode. In the confirmatory mode, the user proposes a set of non-overlapping item clusters thought to represent the true dimensionality structure of the test. DETECT then estimates the degree to which the different specified clusters represent different dimensions and the amount of multidimensionality in the test under the assumption these clusters truly do represent the multidimensional structure of the test. Users generally will propose several sets of clusters, choosing in the end the set with the maximum amount of multidimensionality as indicated by the DETECT index. In the exploratory mode, the user employs either a customized genetic algorithm or HAC to search for the set of clusters that maximizes DETECT, viewing the result as an estimate of the multidimensional structure of the test. In general, users will find it helpful to conduct both confirmatory and exploratory analyses.

• MULTISIM. Produces multidimensionality-based IRT simulated data. It does so according to a user-specified compensatory logistic IRT model allowing up to four latent dimensions. The underlying latent ability distribution is a user-specified multivariate normal distribution.

• SIBTEST. SIBTEST assesses single items for DIF or bundles of items for simultaneous DIF. A modified version can be used to look for crossing (often called non-uniform) DIF (ask for Crossing SIBTEST). Soon, a smoothed version of SIBTEST with improved local DIF accuracy (e.g., appropriate for DIF analysis of a mastery test) and a useful graphical representation of local DIF will be available (ask for Smoothed SIBTEST). A modified version of SIBTEST can be used to assess polytomous item DIF (ask for Poly SIBTEST). In two to three months a version of SIBTEST for use when the matching subtest is multidimensional (e.g., appropriate if assessing DIF in a algebra/geometry mathematics test) will be available (ask for Multi SIBTEST). Also, a major improvement of the regression correction has been accomplished, greatly reducing false positive rejection inflation.

• DIFSIM and DIFCOMP. DIFSIM produces multidimensionality-based IRT simulated DIF data according to user specified multidimensional focal and reference ability distributions and multidimensional compensatory logistic item response functions. Multidimensional model-based DIF simulation i s seen as more authentic and informative than the usual simulation approach to DIF done by manipulation of unidimensional logistic item parameters across examinee groups.

• DIFCOMP computes the theoretical Shealy and Stout model-based index of the amount of item DIF for a given user-specified two or three dimensional DIF model. This enables the user to study the amount of DIF resulting from varying amounts of multidimensional ability distributional differences across group and varying amounts of IRF dependence on an unintended to be measured dimension.

• CONDCOV. Conditional Covariance (CONDCOV) estimates the local dependence between two items as a function of the unidimensionally estimated latent trait by means of kernel smoothing. Useful if one wishes to study lack of unidimensionality or conditional dependence of item pairs locally as a function of the dominant latent trait.