Requirements, Assumptions, @ Estimation of Parameters as well as Instrument Development based on IRT

By:

Marwa & Erlik Widiyani Styati

IRT has a number of advantages over CTT methods to assess test outcomes.  CTT statistics such as item difficulty (proportion of correct responses), item discrimination (corrected item-total correlation), and reliability are contingent on the sample of respondents to whom the questions were administered.  IRT item parameters are not dependent on the sample used to generate the parameters, and are assumed to be invariant (within a linear transformation) across divergent groups within a research population and across populations.

            In addition, CTT yields only a single estimate of reliability and corresponding standard error of measurement, whereas IRT models measure scale precision across the underlying latent variable being measured by the instrument (Cooke & Michie, 1997; Hays, Morales,  & Reise, 2000).  A further disadvantage of CTT methods is that a participant's score is dependent on the set of questions used for analysis, whereas, an IRT-estimated person’s trait level is independent of the questions being used. 

            IRT is a model for expressing the association between an individual's responses to an item and the underlying latent variable (often called "ability" or "trait") being measured by the instrument.  The underlying latent variable in health research may be any measurable construct such as physical functioning, risk for cancer, or depression.  The latent variable, expressed as theta (θ), is a continuous unidimensional construct that explains the covariance among item responses (Steinberg & Thissen, 1995).  People at higher levels of θ have a higher probability of responding correctly or endorsing an item.

.               IRT models use item responses to obtain scaled estimates of θ, as well as to calibrate items and examine their properties (Mellenbergh, 1994).  Each item is characterized by one or more model parameters.  The item difficulty, or threshold, parameter b is the point on the latent scale θ where a person has a 50% chance of responding positively to the scale item (question).  Items with high thresholds are less often endorsed (Steinberg & Thissen, 1995).  The slope or discrimination, parameter a describes the strength of an item's discrimination between people with trait levels (θ) below and above the threshold b.  The a parameter may also be interpreted as describing how an item may be related to the trait measured by the scale and is directly related, under the assumption of a normal θ  distribution, to the biserial item -test correlation ρ  (Linden & Hambleton, 1997).  For item i the relationship is:

Assumptions of Item Response Theory Models

            The IRT model is based on the assumption that the items are measuring a single continuous latent variable θ ranging from -∞ to +∞. . The unidimensionality of a scale can be evaluated by performing an item-level factor analysis, designed to evaluate the factor structure underlying the observed covariation among item responses.  The assumption can be examined by comparing the ratio of the first to the second eigenvalue for each scaled matrix of tetrachoric correlations.  This ratio is an index of the strength of the first dimension of the data.  Similarly, another indication of unidimensionality is that the first factor accounts for a substantial proportion of the matrix variance (Lord, 1980; Reise & Waller, 1990).  For tests using many items, the assumption of unidimensionality may be unrealistic; however, Cooke and Michie (1997) report that IRT models are moderately robust to departures from unidimensionality.  If multidimensionality exists, the investigator may want to consider dividing the test into subtests based on both theory and the factor structure provided by the item-level factor analysis. Multi-dimensional IRT models do exist, but its models as well as informative documentation and user-friendly software are still in development.

            In the IRT model, the item responses are assumed to be independent of one another: the assumption of local independence. The only relationship among the items is explained by the conditional relationship with the latent variable θ . In other words, local independence means that if the trait level is held constant, there should be no association among the item responses (Thissen & Steinberg, 1988). Violation of this assumption may result in parameter estimates that are different from what they would be if the data were locally independent; thus, selecting items for scale construction based on these estimates may lead to erroneous decisions (Chen &Thissen, 1997). The assumptions of unidimensionality and local independence are related in that; items found to be locally dependent will appear as a separate dimension in a factor analysis.